May 09, 2024
Every now and then we do a show that is just densely packed with learning and insight and new ideas, where I realize I can’t even really call out highlights because the highlights are the whole thing. Today’s show is one of them. My guest is Nell Watson, who is an AI Ethics Maestro at IEEE, and author of the just-released book, Taming the Machine.
I had a chance to read an advance copy of the book and to talk with Nell, who was in Northern Ireland. She is the rare expert who balances the tech world and the regulatory/controls world and appreciates both. Her story about her own early entrepreneurial AI experience reminded me of the plot line in the old Silicon Valley TV series about Jian Yang’s AI that was trying to recognize a hot dog. (And that, in turn, is a reminder about how far this technology has come, how fast!). Nell’s book, and this conversation, offer a balanced picture of what AI is going to mean for us all, clear-eyed about the enormous risks ahead, and equally open-eyed about the massive possibilities that AI offers for solving our problems.
And, best of all, more than most sources I find, she has done the work to think through how we can cut down the risks and open up the opportunities.
In our conversation, Nell talks through what this control framework is going to look like and describes the foundations we need to put in place to build legal, regulatory and ethical systems to channel this fiercely powerful new technology away from the bad, and toward the good. As she notes, this is important not only to prevent harm, but also to head off scandal and blowback that would cut off chances to use AI where we should.
Nell covers the gamut of challenges, including how to get reliable, high quality data for AIs to use, how to prevent and eliminate bias, how to address algorithmic management, and the role of open source. She talks about AI’s cost dynamics and ability to democratize huge swaths of human society (like finance). She talks about the privacy challenges and the role of encryption, and some of the tradeoffs between assuring security and addressing bias. She talks about the “branding” of algorithms, and how better to track AI “incident” databases, and how to do better horizon scanning for risk.
Speaking of risks, Nell gives examples of the kinds of terrifying financial frauds and scams that are emerging. She also calls out the need to differentiate among AI use cases and focus sharply on high stakes ones that need tighter standards because bad AI can do so much real and irreparable harm. The most obvious examples are medicine and, yes, financial services.
Nell says we’re currently in the early “wild west” stage of change, and she looks to models in history for ways to bring order to chaos. She mentions the Peelian Principles, set forth by Sir Robert Peel, who established London’s Metropolitan Police in 1829. And she calls for something like the Geneva Conventions to ban dangerous AI uses throughout the world.
One of the most exciting possibilities we explore in our talk is the potential to create “agentive AIs.” Let me read a quick quote from the book. While you listen, imagine deploying these agents on the hard problems we deal with in financial and financial regulation.
Agents are AI that can spawn other versions of themselves to work together to solve a common goal. These models exhibit features like long- and short-term memory, internet access, goal-oriented planning and the ability to spawn sub-agents to which it can delegate specialized tasks… Agents have evolved to become more intelligent and proactive, capable of offering suggestions even before being prompted. They possess the ability to execute tasks across various applications and continuously enhance their performance over time.
So, think about this, and think about the challenges we work with in financial services and financial regulation. I’m picturing a bank examiner, or a bank compliance officer, with an AI agent. The agent is gathering and integrating and analyzing risk data or patterns of possible noncompliance. I’m even picturing a consumer, with an AI agent helping her choose the best credit card or mortgage, or helping her build up savings. If you can be sure that these AI agents would do things right, they could help us solve financial system problems that have never been solvable before.
I think this show will be especially interesting to policymakers in the U.S. and other countries that have not yet enacted legislative frameworks for AI, but will probably do so in the next few years.
So, this introduction is proving out what I said at the start – that the whole show is packed with highlights. I’ll stop talking and let you hear it from Nell.
As you listen, though, bear in mind that AIR is building resources for you to use in working with AI, whatever your role may be. We have published a white paper by former financial regulator Chris Calabia. We have issued a paper with Google on Model Risk Management (MRM) and have a second one coming out in the fall, even more focussed on recent AI developments. We have an online learning series you can sign up for. And we have issued a Call for Papers, with submissions due by June 28, 2024.
Taming the Machine begins with a quote by Jaan Talinn, saying, “Building advanced AI is like launching a rocket. The first challenge is to maximize acceleration, but once it starts picking up speed, you also need to focus on steering.” Nell helps us understand how to do the steering.
I found my talk with Nell reassuring, because she helps us see how, together, we might actually …well, tame these machines.
Eleanor ‘Nell’ Watson, a pioneering ethics and machine intelligence researcher, has been a driving force behind some of the most crucial AI ethics standardization and certification initiatives from organizations such as the IEEE.
We have wonderful guests coming up. We have a show coming up with Chris Calabia on generative AI — again, Chris authored our new AIR white paper on AI. We recorded a fantastic show with the South Africa National Credit Regulator and the International Finance Corporation. We also have an upcoming discussion with former U.S. GSA Commissioner Sonny Hashmi, on how to modernize government technology, and much more.
This is a busy season for AIR. Last week I saw many of you at our third convening on how to help digitize Minority Depository Institutions (MDIs). Later this month, my colleagues Mariama Jalloh-Heyward and Grace Mathebula will be in Ghana at the 3i Africa Summit. My colleague Bob Chakravorti will be speaking at FinovateSpring, holding a fireside chat with OCC Senior Deputy Comptroller and acting innovation head Donna Murphy. And in June I will be speaking at the Exchequer Club in Washington, D.C.
Please be sure to leave us a five-star rating on your favorite podcast platform so more people can find Barefoot Innovation, and please also find me on social media to continue the conversation
Stay informed by joining our mailing list
