Beyond latent vulnerabilities
A while ago, I highlighted Anthony Tan's moving account of how he fell into AI triggered psychosis and survived. One of the things Tan asked us all to campaign for, as a result, is for AI companies to be held to a duty of care. Tan said, "When you provide health advice or emotional support to individuals, you assume a duty of care. AI companies operating in therapeutic spaces—the most popular use case right now—must be held to equivalent standards: competence, confidentiality limits, and mandatory crisis intervention. Companies profiting from intimate access to our thoughts and beliefs should accept some responsibility for the minds they’re reshaping when it goes wrong."
Tan subsequently founded the the AI Mental Health Project to campaign on these issues.
In the same spirit of looking after the humans who use AI tools, every now and again, Cory Doctorow talks about AI and psychosis as well as some of the vulnerabilties the internet has a tendency to unearth and reinforce. Doctorow's recent post on the subject struck a chord.
The internet, Doctorow argues, is very good at locating and exacerbating vulnerabilities that could have lain dormant for a lifetime, if one was not exposed to the plethora of stimuli the internet presents. The internet is a supercharger of paranoid delusions and other types of poor mental health. AI companies argue that people have always suffered from poor mental health and AI is not to blame. To which Doctorow counters that:
Even if you accept the AI companies' argument that they aren't inducing AI psychosis in their users, but rather, only surfacing latent vulnerabilities that were there all along, that shouldn't be the end of the story. Even if only a small percentage of the people who use your product experience harm as a result, if your product is intended for widespread deployment (as chatbots are), you will end up harming a lot of people unless you take measures to counteract even those rare events.
This is what Tan means when he talks about a duty of care.
⚘