As Gary Marcus writes in Politico this week, "AI itself is not one thing but many." We dig into exploratory AI applications in healthcare (successful and less so), elections (good and bad) and a wealth of new data in the 2024 AI Index.
Elections 2024: Tracking AI use in Global Elections | Rest of World
Very cool tracker from Rest of the World that aims to "create a database of examples that can be used to understand the many ways in which AI is being deployed around elections." We're used to hearing about these incidents on a case by case basis, but seeing them all compiled (to include efforts at manipulation, positive uses of AI and memes) raises some real questions about how truth will fare in the AI era.
Tech Impacts Every eEection. | Elections and Tech
In light of the above, we wanted to call out the guidelines from the International Foundation for Electoral Systems (IFES), developed in cooperation with tech companies, urging greater cooperation between technology companies and election officials to protect election integrity.
ChatGPT bombs test on diagnosing kids’ medical cases with 83% error rate | Ars Technica
One of the more exciting/promising use cases of AI is in medicine. This study suggests it has a long way to go to deliver on its promises. With the disclaimer that an LLM may not the best integration of AI into diagnosis, the study found that "ChatGPT got the right answer in just 17 of the 100 cases. It was plainly wrong in 72 cases, and did not fully capture the diagnosis of the remaining 11 cases. Among the 83 wrong diagnoses, 47 (57 percent) were in the same organ system." Also in the healthcare space, early reporting found that use of AI in reading mammograms (a distinctly more technical, less clinical, use) could increase breast cancer detection by 20%.
Opinion | How to Protect Americans from the Many Growing Threats of AI | Politico
Gary Marcus provides a direct and compelling summary of where AI regulation who focus its efforts, the harms we've seen to date, and the potential for those harms to grow absent meaningful regulation. "Perhaps the single greatest policy challenge," he muses, "is that no single law could possibly suffice to contain the risks of AI, because AI itself is not one thing but many, with many potential uses, many of which probably haven’t been envisioned yet. We need many different types of laws to combat human criminals — and in a similar way, we will need a layered approach to deal with the broad array of potential misuses of AI."
Measuring the Persuasiveness of Language Models | Anthropic
This new paper from Anthropic explores Claude's persuasiveness. Although in controlled settings (the model had a single writing prompt by which to persuade people, with no back and forth conversation), they found Claude 3 to be roughly as persuasive as humans (it's worth noting their disclaimer that "persuasion is difficult to study in a lab setting – our results may not transfer to the real world.") The visual illustration showing the increased persuasion abilities of subsequent models is deeply striking. The paper's findings (and implications) are also discussed in Ezra Klein's recent podcast with Anthropic CEO Dario Amondei.
AI Index Report 2024 – Artificial Intelligence Index (stanford.edu)
Clocking in at 500+ pages, the 2024 AI Index from Stanford Institute for Human-Centered Artificial Intelligence was released this week. Lots to dig into here – we'll try to explore individual chapters of interest in future weeks.