Topics
Latest
AI
Amazon
Image Credits:Ilya Lukichev / Getty Images
Apps
Biotech & Health
Climate
Image Credits:Ilya Lukichev / Getty Images
Cloud Computing
Commerce
Crypto
Image Credits:OctoAI
Enterprise
EVs
Fintech
fund-raise
Gadgets
bet on
Government & Policy
Hardware
Layoffs
Media & Entertainment
Meta
Microsoft
secrecy
Robotics
security measure
Social
Space
Startups
TikTok
Transportation
speculation
More from TechCrunch
effect
Startup Battlefield
StrictlyVC
Podcasts
video
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
OctoAI(formerly knownas OctoML ) , announce the launch of OctoStack , its new end - to - end solvent for deploy reproductive AI models in a company ’s private cloud , be that on - premises or in a virtual secret swarm from one of the major vendors , admit AWS , Google , Microsoft and Azure , as well as CoreWeave , Lambda Labs , Snowflake and others .
In its former days , OctoAI focused almost exclusively on optimize models to run more in effect . Based on theApache TVMmachine read compiling program framework , the fellowship then launched its TVM - as - a - Service platform and , over time , expanded that into a to the full fledge model - serve offering that combined its optimization chop with a DevOps platform . With the rise of productive AI , the team then launch the in full grapple OctoAI political platform to help its users serve and fine - tune existing modelling . OctoStack , at its core , is that OctoAI political program , but for individual deployments .
OctoAI CEO and co - founderLuis Cezetold me the company has over 25,000 developer on the political program and hundred of paying client who use it in production . A peck of these companies , Ceze read , are GenAI - native companies . The grocery of traditional initiative want to take on generative AI is significantly larger , though , so it ’s possibly no surprise that OctoAI is now going after them as well with OctoStack .
“ One affair that became light is that , as the enterprise market is hold up from experiment last year to deployments , one , all of them are looking around because they ’re nervous about sending data over an API , ” Ceze say . “ Two : a mountain of them have also put their own compute , so why am I going to buy an API when I already have my own compute ? And three , no matter what certifications you get and how big of a name you have , they feel like their AI is valued like their data and they do n’t want to send it over . So there ’s this really clear need in the endeavour to have the deployment under your restraint . ”
Ceze noted that the team had been build out the computer architecture to extend both its SaaS and hosted platform for a while now . And while the SaaS platform is optimized for Nvidia hardware , OctoStack can affirm a far wider range of hardware , include AMD GPUs andAWS ’s Inferentiaaccelerator , which in turn makes the optimization challenge quite a bit hard ( while also playing to OctoAI ’s strengths ) .
Deploying OctoStack should be straight for most endeavour , as OctoAI fork out the platform with read - to - go container and their associated Helm charts for deployment . For developers , the API remains the same , no matter whether they are targeting the SaaS ware or OctoAI in their private cloud .
The canonical enterprise habit case persist using text summarisation and RAG to allow drug user to confabulate with their inner documents , but some companies are also alright - tuning these model on their inner codification groundwork to run their own code contemporaries models ( like to what GitHub now offers toCopilot endeavour users ) .
Join us at TechCrunch Sessions: AI
Exhibit at TechCrunch Sessions: AI
For many enterprises , being able to do that in a secure environs that is strictly under their control is what now enable them to put these technologies into production for their employees and customers .
“ For our performance- and security - sore use case , it is imperative that the models which operation call in data run in an environment that offers tractableness , scale and security , ” enounce Dali Kaafar , founding father and CEO atApate AI . “ OctoStack lets us well and efficiently launch the customized models we necessitate , within surround that we choose , and deliver the scale our customers require . ”
OctoML launch OctoAI , a self - optimise compute service for AI
OctoML makes it easier to put AI / ML models into production
