A mind-expanding conversation on how models actually work | Emmanuel Ameisen (Anthropic)

One of the big questions on people's minds these days is AI doing bad things. As someone that watches these models think and operate, why is it so hard to catch it thinking badly trying to deceive us? >> We trained Claw to be helpful, harmless, and honest. Is it the case that it's behaving well, but if you look inside,

actually, it's like, oh, I'm only behaving well cuz Lenny's watching right now. >> People's full-time jobs is studying the model, how it thinks, how it operates. How long has this been a skill set in a job? One of the most important things to realize if you like use language models or think about them is just this like

very bizarre fact that to make them is not to understand them. All these models do is just a lot of mathematical operations on many numbers. If you look at it, it makes no sense. You just have a list of a 100 numbers. You know, the model's doing math and then you just try

to see if there's patterns. >> You are figuring it out the same way we figure out how the brain works. Looking for patterns. You show this image. This part of the brain lights up and you're doing the exact same thing with AI. Wow, this is [laughter] so fascinating. >> There was this model we were training.

It would go crazy. It would do some math. It would get to an answer. I think it was 12. It would immediately go back and say like, "Oh no, oh my god, the answer actually is for sure 36." It would do math again. It would get to 12. Slowly, it would like start descending

into madness, writing in all caps, literally saying, "I'm possessed by a demon." Humans, if you really boil it down, like we're basically next token predictors ourselves. We're just trying to figure out the next thing to say. Daario is always out there. This is really dangerous. We got to be really careful. And everyone's just like, "No,