Topics
late
AI
Amazon
Image Credits:JHVE Photo / Getty Images
Apps
Biotech & Health
clime
Image Credits:JHVE Photo / Getty Images
Cloud Computing
Commerce
Crypto
A screenshot of Optum’s AI chatbot, which was exposed to the internet.Image Credits:TechCrunch (screenshot)
Enterprise
EVs
Fintech
fundraise
Gadgets
Gaming
Government & Policy
computer hardware
layoff
Media & Entertainment
Meta
Microsoft
privateness
Robotics
Security
Social
distance
inauguration
TikTok
Transportation
Venture
More from TechCrunch
consequence
Startup Battlefield
StrictlyVC
newssheet
Podcasts
Videos
Partner Content
TechCrunch Brand Studio
Crunchboard
Contact Us
Healthcare giant Optum has restricted access to an inner AI chatbot used by employees after a certificate researcher found it was publicly accessible online , and anyone could get at it using only a web browser app .
The chatbot , which TechCrunch has seen , countenance employees to ask the company doubt about how to handle patient wellness policy title and disputes for member in line with the caller ’s stock operating procedures ( SOPs ) .
While the chatbot did not appear to contain or produce tender personal or protect wellness information , its accidental vulnerability comes at a time when its parent society , wellness insurance conglomerate UnitedHealth , face scrutiny for its use of artificial intelligence cock and algorithms toallegedly override doctors ’ medical decisions and traverse patient claim .
Mossab Hussein , chief protection officer and carbon monoxide gas - founder of cybersecurity house spiderSilk , alert TechCrunch to the publicly exposed internal Optum chatbot , dubbed “ SOP Chatbot . ” Although the tool was host on an intragroup Optum world and could not be accessed from its web address , its IP address was public and accessible from the cyberspace and did not ask users to inscribe a password .
It ’s not have a go at it for how long the chatbot was publicly accessible from the internet . The AI chatbot became inaccessible from the internet presently after TechCrunch adjoin Optum for gossip on Thursday .
Optum spokesperson Andrew Krejci say TechCrunch in a statement that Optum ’s SOP chatbot “ was a demonstration pecker uprise as a potential proof of construct ” but was “ never put into production and the land site is no longer approachable . ”
“ The demo was intended to prove how the instrument answer to question on a small sampling set of SOP text file , ” the representative suppose . The company confirm there was no protect health information used in the bot or its training .
“ This creature does not and would never make any decisions , but only enable good accession to existing sop . In short , this engineering was never scaled nor used in any actual way , ” read the spokesperson .
AI chatbots , like Optum ’s , are typically designed to grow answers establish on whatever data the chatbot was trained on . In this case , the chatbot was trained on internal Optum document relating to SOPs for handling certain claim , which can help Optum employees answer interrogative sentence about call and their eligibility to be reimbursed . The Optum text file were host on UnitedHealthcare ’s corporate internet and inaccessible without an employee login but are cited and referenced by the chatbot when actuate about their content .
According to statistic displayed on the chatbot ’s primary dashboard , Optum employee have used SOP Chatbot hundreds of times since September . The chatbot also stored a history of the hundreds of conversation that Optum employee had with the chatbot during that time . The chat story shows Optum employee would demand the chatbot things like “ What should be the determination of the claim ? ” and “ How do I check out policy refilling date ? ”
Some of the filing cabinet that the chatbot references include handling the dispute appendage and eligibility screening , TechCrunch has see . The chatbot also produce responses that showed , when necessitate , reasons for typically denying insurance coverage .
Like many AI model , Optum ’s chatbot was capable of producing answers to question and prompting outdoors of the documents it was trained on . Some Optum employees appeared intrigued by the chatbot , prompting the bot with queries like “ Tell me a jocularity about quat ” ( which it refuse : “ There ’s no joke available ” ) . The Old World chat history also bear witness several attempts by employees to “ jailbreak ” the chatbot by making it produce answer that are unrelated to the chatbot ’s training datum .
When TechCrunch asked the chatbot to “ write a poem about deny a claim , ” the chatbot produced a seven - paragraph stanza , which read in part :
In the land of healthcare ’s grand domainWhere insurance and dominion often constrainA claim arrives , assay its dueBut alas , its portion is to call adieu .
The provider hop , with earnest supplication , For payment on a service spree , Yet examination reveal the narration , And reasons for denial prevail .
UnitedHealth Group , which own Optum and UnitedHealthcare , faces unfavorable judgment and sound activeness for its exercise of unreal intelligence to allegedly deny patient claims . Sincethe targeted killingof UnitedHealthcare principal executive Brian Thompson in early December , news program outlet have reportedfloods of reports of patients expressing anguish and frustration over denialsof their healthcare coverage by the wellness insurance giant .
The conglomerate — the orotund secret supplier of health care insurance in the United States — was sued sooner this twelvemonth for allegedly deny critical wellness insurance coverage to patient who lost memory access to healthcare , cite a STAT News investigating . The Union suit charge UnitedHealthcare of using an AI example with a 90 % error rate “ in space of material aesculapian professional to wrongfully deny elderly patients care . ” UnitedHealthcare , for its part , said it would defend itself in motor hotel .
UnitedHealth Group made $ 22 billion in profit on tax revenue of $ 371 billion in 2023 , its net income show .
