Topics
Latest
AI
Amazon
Image Credits:Peresmeh / Getty Images
Apps
Biotech & Health
clime
Image Credits:Peresmeh / Getty Images
Cloud Computing
Commerce
Crypto
Image Credits:Ai2
enterprisingness
EVs
Fintech
Fundraising
gadget
back
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
Privacy
Robotics
Security
Social
infinite
Startups
TikTok
Transportation
speculation
More from TechCrunch
Events
Startup Battlefield
StrictlyVC
Podcasts
video
Partner Content
TechCrunch Brand Studio
Crunchboard
get hold of Us
There ’s a new AI exemplar family on the block , and it ’s one of the few that can be reproduced from scratch .
On Tuesday , Ai2 , the non-profit-making AI research organization founded by the former Microsoft carbon monoxide gas - father Paul Allen , released OLMo 2 , the 2nd category of models in its OLMo series . ( OLMo is unforesightful for “ open speech example . ” ) While there ’s no shortage of “ open ” speech communication model to choose from ( e.g. , Meta’sLlama ) , OLMo 2 meets the Open Source Initiative ’s definition of open source AI , meaning the tools and data used to develop it are in public available .
The Open Source Initiative , thelong - running game institutionthat aims to define and “ steward ” all affair open informant , finalized its undefended source AI definition in October . But thefirst OLMo models , liberate in February , run into the measure as well .
“ OLMo 2 [ was ] developed showtime - to - finish with open and approachable training data , open - germ training code , consistent preparation formula , transparent evaluation , intermediate checkpoint , and more , ” AI2 write in ablog post . “ By openly apportion our data , recipes , and finding , we go for to bring home the bacon the open - source residential district with the resources ask to get wind new and innovative approach . ”
There are two models in the OLMo 2 family : one with 7 billion parameters ( OLMo 7B ) and one with 13 billion parameter ( OLMo 13B ) . Parameters roughly correspond to a model ’s job - solving acquirement , and exemplar with more argument generally do considerably than those with fewer parameter .
Like most language models , OLMo 2 7B and 13B can perform a range of textbook - based tasks , like answering questions , summarizing document , and writing code .
To train the models , Ai2 used a dataset of 5 trillion tokens . Tokens represent bits of raw data ; 1 million tokens is equal to about 750,000 words . The education exercise set included websites “ separate out for high calibre , ” academic report , Q&A word board , and math workbooks “ both synthetic and human generated . ”
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
Ai2 lay claim the resultant is models that are private-enterprise , public presentation - wise , with open model like Meta’sLlama 3.1release .
“ Not only do we follow a dramatic improvement in performance across all tasks compared to our former OLMo model but , notably , OLMo 2 7B outperforms Llama 3.1 8B , ” Ai2 writes . “ OLMo 2 [ act ] the proficient fully - open speech models to date . ”
The OLMo 2 models and all of their component can be downloaded from Ai2’swebsite . They ’re under Apache 2.0 license , meaning they can be used commercially .
There ’s been some debate late over the condom of open models , what with Llama modelsreportedlybeing used by Formosan researchers to develop defence cock . When I asked Ai2 locomotive engineer Dirk Groeneveld in February whether he was concerned about OLMo being abused , he say that he believes the benefits ultimately overbalance the harms .
“ Yes , it ’s potential opened model may be used unsuitably or for unintended purpose , ” he enunciate . “ [ However , this ] approach also promotes technical advancements that lead to more ethical simulation ; is a requirement for check and duplicability , as these can only be achieved with access to the full passel ; and reduces a develop concentration of power , make more equitable accession . ”
