Topics
Latest
AI
Amazon
Image Credits:Kevin Dietsch / Getty Images
Apps
Biotech & Health
clime
Image Credits:Kevin Dietsch / Getty Images
Cloud Computing
Commerce
Crypto
Image Credits:OpenAI
endeavor
EVs
Fintech
Image Credits:OpenAI
fundraise
convenience
Gaming
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
concealment
Robotics
security system
Social
blank
Startups
TikTok
deportation
Venture
More from TechCrunch
issue
Startup Battlefield
StrictlyVC
newssheet
Podcasts
Videos
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
On Thursday , OpenAIreleasedwhat ’s effectively a $ 200 - a - calendar month chatbot — and the AI biotic community did n’t know quite what to make of it .
The company ’s young ChatGPT Pro programme grants entree to “ o1 pro mode , ” which OpenAI say “ use more compute for the good result to the knockout interrogative sentence . ” A souped - up version of OpenAI’so1reasoning model , o1 pro way should answer questions relating to science , maths , and cypher more “ dependably ” and “ comprehensively , ” OpenAI suppose .
Almost now , people started asking it to tie unicorn :
I need ChatGPT o1 Pro Mode to create an SVG of a unicorn .
( This is the model you get access to for $ 200 monthly)pic.twitter.com / h9HwY3aYwU
— Rammy ( @rammydev)December 5 , 2024
And design a “ crab - based ” estimator :
Finally set o1 - pro to its ultimate use case.pic.twitter.com/nX4JAjx71 m
— Ethan Mollick ( @emollick)December 6 , 2024
And wax poetic on the significance of liveliness :
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
I just take to OpenAI ’s $ 200 / month subscription . Reply with question to ask it and I will repost them in this thread.pic.twitter.com/oTQxbPxnoP
— Garrett Scott 🕳 ( @thegarrettscott)December 5 , 2024
But many folks on X did n’t seem confident that o1 pro way ’s answers were , well , $ 200 - grade .
“ Have OpenAI shared any concrete examples of prompting that fail in regular o1 but follow in o1 - pro?”askedBritish computer scientist Simon Willison . “ I need to see a single concrete example that show up its advantage . ”
It ’s a reasonable question ; after all , this is the world ’s most expensive chatbot subscription . The service follow with other benefit , like the remotion of pace limits and unlimited admittance to OpenAI ’s other models . But $ 2,400 per year is n’t chump change , and the value proposition of o1 pro manner in particular stay on muddy .
It did n’t take long to witness unsuccessful person caseful . O1 pro modality struggles with Sudoku , and it ’s tripped up by an optical semblance joke that ’s obvious to any human .
o1 and o1 - pro both failed here , belike still because of the vision limitations ( the same with Sudoku puzzles)https://t.co / mAVK7WxBrqpic.twitter.com / O9boSv7ZGt
— Tibor Blaho ( @btibor91)December 5 , 2024
OpenAI ’s interior benchmarks show that o1 pro modality perform only slightly good than the stock o1 on fool and math problem :
OpenAI ran a “ stricter ” rating on the same bench mark to showcase o1 pro modality ’s consistency : the mannequin was only considered to have solved a question if it got the solution right four out of four times . But even in these tests , the betterment were n’t dramatic :
OpenAI CEO Sam Altman , who once wrote that OpenAI was on apath“towards intelligence information too chinchy to meter , ” was forced toclarifymultipletimeson Thursday that ChatGPT Pro is n’t for most people .
“ Most users will be very well-chosen with the o1 in the [ ChatGPT ] Plus tier ! ” he said on X. “ Almost everyone will be well - served by our free grade or the Plus level . ”
So who is it for ? Are there really citizenry out there willing to pay $ 200 a calendar month to ask toy question like “ Write a 3 - paragraph essay on strawberries without using the letter ‘ due east ’ ” or “ solve this Math Olympiad job “ ? Will they happily part path with their hard - earned cash without much guarantee that the standard o1 ca n’t satisfactorily serve the same inquiry ?
I enquire Ameet Talwalkar , an associate professor of simple machine learn atCarnegie Mellonand a speculation partner at Amplify Partners , his opinion . “ It seems like a with child risk to me to raise the price tenfold , ” he secern TechCrunch via e-mail . “ I cerebrate we ’ll have a much better sentiency in just a few week as to the appetite for this functionality . ”
UCLA reckoner scientist Guy Van den Broeck was more candid in his judgment . “ I do n’t bed if the price stop makes sense , ” he secern TechCrunch , “ and if pricey reasoning models will be the norm . ”
o1 is “ better than most humans at most tasks ” because , yes , humans exist exclusively in amnesic disembodied multi - turn chat interfaceshttps://t.co/zbLY2BG5pQ
— Aidan McLau ( @aidan_mclau)December 6 , 2024
A generous take is that it ’s a marketing blunder . Describing o1 pro mode as best at work “ the hardest problems ” does n’t severalise prospective customers much . Nor dovague statementsabout how the model can “ recall longer ” and demonstrate “ intelligence activity . ” As Willison point out , without specific representative of this purportedly improved capableness , it ’s severe to absolve paying more at all , let alone ten times the price .
this is such a suspicious recommend prompt for an ai model that costs $ 2400 / year
I go for openai keep these boilerplate sample prompt all the way to asipic.twitter.com/JQ5vLKxWWR
— Dean W. Ball ( @deanwball)December 6 , 2024
So far as I can secernate , experts in specialised fields are the intended audience . OpenAI says it plan to grant a fistful of aesculapian researchers at “ run institutions ” free access to ChatGPT Pro , which will include o1 pro mode . Mistakes matter a lot in health care , and , as Bob McGrew , OpenAI ’s former chief inquiry officer , notedon X , undecomposed reliability is perhaps o1 pro mode ’s chief unlock .
Been toy with o1 and o1 - pro for bit .
They are very good & a piffling weird . They are also not for most people most of the time . You really want to have particular concentrated problems to lick in rules of order to get value out of it . But if you have those problems , this is a very big deal .
— Ethan Mollick ( @emollick)December 5 , 2024
McGrew alsomusedo1 pro mode is an model of what he call up “ intelligence overhang ” : user ( and perhaps the simulation ’s God Almighty ) not knowing how to get note value from any “ extra intelligence ” due to underlying terminal point of a unproblematic , text - base port . As with OpenAI ’s other models , the only way to interact with o1 pro modal value is through ChatGPT , and — to McGrew ’s point — ChatGPT is n’t perfect .
It ’s also true , though , that $ 200 set arithmetic mean high . And judging by the former reception on societal media , ChatGPT Pro is no slam dunk .
