We Tested $200 ChatGPT and Claude on Research Math

So, at this point, we've pretty much seen it all. Last year, we saw the most powerful AI models fail at basic high school-level maths. Then, fast-forward to today, where OpenAI solved one of the millennium problems. There's also talk going around that they've made progress on other millennium problems. And just with so much going on with companies

continuously releasing new models, with them then saying to slow down, but then not slowing down whatsoever. Then, huge claims being made of like a 30 trillion total addressable market. It's becoming more and more difficult to decipher the things which are just hype and the things which are genuinely true, especially regarding these most powerful

state-of-the-art large language models. So, that's why in the last year, we've been actually trying to put these models to the test on a whole bunch of research problems that I came across during my PhD, to see whether these models are actually improving at the rate that's being claimed by these companies. But, also to see whether these models are

actually able to do this autonomously in the ways that they're being claimed upon the release of all of these proofs made by Anthropic and OpenAI, where they claim that there was essentially no help being given to these models, and that they pretty much just solved these problems all by themselves. So, how exactly are we going about doing this?

Well, we've essentially been given the two best AI models, which is the Claude Pro next account and GPC Pro. These are about $200 per month each. And on our laptop, we set up a folder. The structure of this folder is a maths problem, which includes a whole bunch of different markdown files for each AI