Artificial Intelligence needs a police blotter. Wait, it already has one!

In our world of black and white, it’s difficult to be a tech skeptic without being labeled a Luddite. The Holy Grail is progress, so the thinking goes, and any pesky questions threaten to stifle innovation. Do you want us to lose to the Chinese!?!?

It’s ok, I’ve been doing this a long time. Not so long ago, my nickname among CNBC bookers was “Big Data Hater.” You remember the age of Big Data, don’t you? If you don’t, it was yet another marketing moniker that took over the tech world for a few years, stoking stock valuations everywhere it went. A mini dot-com boom, if you will. Big Data, unfortunately, often became synonymous with Bad Data, which always gives bad results, no matter how much data you shove into the GIGO machine. Today, we call these Large Language Models, which sound much more sophisticated, but suffer equally from the same garbage problem.

---

This is part 2 of a three-part miniseries on Artificial Intelligence.
Read part 1: Disarm AI, yes, but the Pope was just getting started

---

Back then, I would protest with a glint in my eye -- how can someone hate data? That’s like hating atoms! Some of my best friends are data!

I don’t hate data. Or tech. What I hate is thoughtless “progress” without discernment about side effects and collateral damage. And I really hate when the progress ... is promised, down the road -- soon! -- while the roadkill piles up today. What’s the roadkill of this never-ending tech bubble cycle? Pension funds that are crushed when the bubble bursts. Workers who are laid off in the name of cost savings needed to offset investments. Kids who end up with addiction machines in their pockets because there’s no other legitimate business model for social media. Adults who’ve sacrificed every shred of human privacy so they can be stalked by ads for items they purchased last week. And so on.

Yes, the consequences are real, and they are here -- even if the innovations are...just around the corner.

I’m not arguing that AI isn’t real. Already, it’s freed an entire generation from writing trite, jibberish-laden emails back and forth at work. AI can turn meetings that should have been an email into a summary of said email. That could be real progress -- but let me know when those meetings are actually canceled.

Can AI do a great job of writing a meeting summary for people who weren’t really paying attention anyway? Yes, absolutely. Can it pull out that one critical moment in the meeting which most attendees missed...which might very well be what was left unsaid? Ha! (You’ll read about this in part three of this miniseries.)

AI is great at writing code, getting rid of some of the grunt work of the digital age. It helps people with blank page syndrome get a start on papers and presentations. And it’ll do a fine job of summarizing large amounts of material for people in a hurry. A great application I read about recently solves that “too-much-to-read” problem for practicing physicians who have scant time to digest all the latest medical research. AI can do these things today.

As for tomorrow -- there seems good reason to believe AI will be great at finding needles in research haystacks, which could very well lead to amazing medical advances. I will be the first to cheer on this work. I’m sure I’ll need it someday.

But tech titans have a decades-long pattern of racing forward with innovations, intermediate consequences be damned. Of doing things simply because we can, not because we should -- in fact, not even asking if we should. And, specific to my main work right now, of creating tools that are easy to abuse and darn near impossible to stop.

I am not a Luddite. I think tech does more good than bad. But I think in a playoff series, “good” wins in the 7th game, and probably in overtime. It’s often a close call. We can’t ignore the bad things that AI will enable because honest evaluation might slow progress a smidge. The best thing we can do is air every single one of these side effects and work to eliminate them. That’s how penetration testing has always been done. That’s the ethos of open source software. More than ever, we need to approach the coming age of AI that way.

That’s why I was so happy to learn recently about the Artificial Intelligence Incidents Database. It is what it sounds like -- a list of mishaps caused by, or enabled by, AI. I recently interviewed one of its leaders, Harvard fellow Sean McGregor, for The Perfect Scam, a podcast I host for AARP. McGregor is the kind of plain-speaking genius we desperately need right now. We talked for an episode about a family who was targeted by an AI-generated photo of the family dog depicting him on an operating table, riddled with injuries from a car accident. (That is incident 1,478 in the incident database).

McGregor made this point: Early on, the database was full of (funny?) incidents about AI failing to work properly. But increasingly, the database is loading up on tales of fraud committed by criminals using AI. That might be the bigger problem, he suggests -- the so-called dual use problem -- as AI gets better at what it does, it gets better for the bad guys. I left our chat thinking my sarcasm about AI’s clumsy failures might very well be misplaced.

You can see a partial transcript of our chat below, if podcasts aren’t your thing. But I hope you’ll listen to our entire conversation, which you can hear at this link, or on your favorite podcast platform.

But whatever you do, don’t call someone a Luddite because they’re worried about the future. We do get to decide what kind of future we want; we don’t have to just accept what Elon Musk gives us. In fact, I’d argue, that’s a poor choice.

Tristan Harris from the Center for Humane Technology appeared on CNN this week and made a very sharp point about incentives. In the end, AI is going to become whatever the incentives nudge it to become. Right now, the only incentive on the table is shareholder value. That means AI will principally be used to eliminate labor costs. The End. But we have the chance to design other incentives right now. To reduce human suffering. To build more housing. To make mass transit far more efficient. Heck, to enable human happiness. Whoever told you that our society’s only goal is profit sold you a very shallow future. We can, we must, do better. An honest, real-time look at AI’s failings is going to be a big part of that.

-----------------------------------PARTIAL TRANSCRIPT---------------------

[00:28:16] Sean McGregor: My general reaction to the scam is while people are inventive and sometimes horrible, and as soon as you see someone, someone do the thing, the bad thing, you’re like yeah, that’s something we have to deal with in the world now, that this is something that technology has made possible. It can do many amazing, wonderful things, but it also empowers the bad actors.

[00:28:40] Bob: And, and taking a tool and abusing it or using it for abuse, that’s, that happens at every stage of technology, right?

[00:28:47] Sean McGregor: That’s right. It’s a tale as old as time.

[00:28:50] Bob: How hard is it to generate a photo using say you know I go onto someone’s Facebook page, I find a picture of their pet, and then I tell a computer, make a photo of this pet looking like it’s in dire need of surgery. How hard is it to do that?

[00:29:01] Sean McGregor: It’s very easy. You don’t need to be a computer scientist, you don’t need to have any great degree of technical sophistication, you just have to search around a little bit, find where the tool is, plug the photo in and you’re good to go.

[00:29:14] Bob: In fact, Sean helps track stories like this one in that AI Incident Database, and yes, Archer’s tale is in there. It’s Incident 1478. But back up a second. What is this database?

[00:29:28] Sean McGregor: I started a project and an organization behind it about 8 years ago that’s dedicated to indexing what are called AI incidents. And AI incidents can be a misuse, they can be scams, they can also be malfunctions. It’s basically any time someone’s harmed out in the real world we want to collect that and put it in the database because this is how you make safety. This is how you make AI safer. It’s inspired by some other databases in aviation where a plane crashes, you record the circumstances of that crash. And that collectively gives the aviation industry the charge of making sure it doesn’t happen again in that way. And so as a result of that, I’m one of the most well-read people in the world about bad things that happen with AI, and it’s a funny point to me because I more or less got my start in doing good things with AI, figuring out how to solve problems in ecology and wildfire and all environmental management problems, and here I am reading about all the bad things.

[00:30:35] Bob: Uh this sounds like the police blotter to me. I don’t know if you remember police blotter in newspapers, but it always on like on page 2 or 3, and it was just all the small bad things that happen in your town, but yet it was always the most read thing in the newspaper. This is like the police blotter for artificial intelligence it sounds like to me.

[00:30:51] Sean McGregor: Yeah, you could call us that, police blotter for AI. That fits.

[00:30:55] Bob: Why do we need a police blotter for AI?

[00:30:59] Sean McGregor: AI is increasingly everywhere in the world, and the problem is AI isn’t always all that robust to all the ways in which the world varies. It often fails and is unsafe as a result of that, and I ask the question of what is safety, how do we build effective safety culture and processes to make these systems something that we actually want to have in the world.

[00:31:22] Bob: So give me an example of like an exception condition, or something weird that happens that AI would handle badly.

[00:31:27] Sean McGregor: Sure. When I think about the physical system side of things, is I once saw a dog that was in someone’s car on the freeway and we were in stop and go traffic, and the dog jumped out of the car onto the middle of the freeway. An autonomous car hasn’t seen a lot of instances of a dog in the middle of the freeway, and people know, let’s stop driving right now. The whole freeway’s stopped to let the person get out of the car, collect the dog that looked none too happy about this situation it got itself into, and got back in the car and then freeway kept on going. But we don’t actually know how an autonomous car would react in this scenario. It’s built to try and proceed down the freeway. And this is an example of the, there’s a big, wide-open world and we need to build systems that are able to handle that.

[00:32:14] Bob: So initially, the harms that Sean logged were failures of the technology to work, to detect things like runaway dogs on highways. But that’s changing.

[00:32:25] Sean McGregor: I think that the story of AI safety is one where new technology is introduced and initially it’s unsafe because it just hasn’t gotten operational history in the world. We haven’t learned from the collective experience of the technology in society. Then through time, particularly with these systems that are based on a technique called machine learning, which is basically the system is produced by data, we accumulate data and the system gets safer to operate in the world. Then as that continues and as it moves from a place of it’s unreliable to reliable, then it gets more reliable for the bad actors and the bad people, and then you have to solve that, what’s in our community called the dual use problem of it could be used for good, and it could be used for bad. And so there’s this kind of transition from we need to care about, it’s just not working to we need to care about it’s working for the wrong people.

[00:33:25] Bob: Working too well in some cases. That’s really interesting. It’s a dual use problem, huh.

[00:33:29] Sean McGregor: Right.

[00:33:30] Bob: AI is now working for the wrong people. Sometimes anyway.

[00:33:36] Sean McGregor: North Korea, for instance, a country that really needs hard currency. This is something that will keep them, keep the government afloat and it actually has a large number of people that are trained to go out there and figure out how to do cryptocurrency scams. There was one incident where, actually we’ve seen several incidents wherein important business people were impersonated and used to instruct others to transfer funds. That the capacity for impersonation has gotten much greater than before where you might have a scam previously of someone’s grandson calling supposedly from Mexico and saying, I’m in jail and so forth, and it doesn’t sound like them but they make the line noisier so that it’s harder to tell. Now we can have people sound like the grandson, sound like the CEO, and extort funds or not extort, just command funds to be transferred. And so we’ve seen a quite a few of those as well.

[00:34:39] Bob: And since we’re talking about executives, that, that can be hundreds of thousands of dollars, right?

[00:34:44] Sean McGregor: Yeah, they, they can move around quite a lot, the funds, without it, so it’s not an unusual thing to do.

[00:34:52] Bob: Until very recently, we conclude many stories like this by saying something like, well watch for bad grammar, or stilted weird speech patterns. That’s not really useful advice anymore.

[00:35:05] Bob: If I saw a picture of my dog in pain, I, I’d do anything, right, and I, and I kind of want to make that point, like really drive it home for people that, that you can’t believe anything you see anymore, even in these emotional situations, right?

[00:35:18] Sean McGregor: Yeah, you need to approach digital information with skepticism because it’s much easier to create very convincing contact than it was previously. And there is this distinction in the community of cheap fakes vs. deep fakes that I think is good to tease out. That a cheap fake has been possible for a while. It’s using photoshop, it’s using photo editing tools. You can cre--, create those very convincingly without necessarily using a advanced AI model to, to produce it. The thing that’s different is that took some level of skill and time to produce and most scams fail. It’s a volume and a numbers game, and so what we have here is it got a lot cheaper, a lot faster to generate these kinds of highly personalized attacks, and so we see a lot more of that now and we’re going to see a lot more in the future. Another way of, of thinking of this is if you, if you plug a computer into the internet that hasn’t touched the internet for a few years, it will pretty instantly get a virus. Like it, there’s this kind of background radiation that is permeating computers that the computers don’t get into trouble just because they’ve been hardened through time; they’re robust to that background radiation, and we are moving that kind of radiated state, that scale of just there’s always something roaming around trying to attack systems. We’re moving that into the human space because we’re making the human space available to the machine and making it possible for AI to generate and interact with people more directly.

[00:37:04] Bob: And, and so that’s, that’s a fascinating comparison actually. None of us have this sort of built up immunity that our computers do to computer-related crimes.

[00:37:13] Sean McGregor: No.

[00:37:14] Bob: Has all this work on the police blotter changed any of Sean’s views on AI?

[00:37:19] Sean McGregor: I don’t think so. I think I’ve always been somewhat measured and just in awe of what we can do and how we can work with these mag--, magical chips that allow for doing such useful things at fantastical speed. It’s just a, a sense of we need a corresponding response to that power for making and preventing things like scams, and I find that we haven’t nearly adequately invested in that. And even looking at the AI Incident Database, we’re leading in the world in that particular kind of indexing of very critical information for making the technology safer. But we’re still very much subscale and if anyone wants to support the program, you can go to, it’s DatabaseAI and donate to it. We, every dollar goes to covering these things and making it so people can be better protected and that the safe-, the technology would be safer.

[00:38:14] Bob: So on one side of this ledger we have the, the world’s largest companies making the world’s largest investment ever in data centers, and on this side of the world, we have what sounds like a bake sale to me.

[00:38:26] Sean McGregor: That’s probably a fair description. We’re making some tasty Rice Krispie treats over here, but uh...

[00:38:32] Bob: (chuckles)

[00:38:34] Bob: So what can we do about this future?

[00:38:36] Sean McGregor: One of my favorite sayings often attributed to sci fi author William Gibson, is “The future is already here. It’s just not evenly distributed yet.” There is a great many ills that have been produced as a proof of concept. You can find a, a lot of those in the AI Incident Database, and you can see then what’s coming around the corner, what things that you need to start worrying about. A lot of these stories about deep, deep fake dog, they’re just moving from the perfect concept phase to this is a scam deployed at great scale. And so you can watch what has been proven and what happens, and you can use that to protect yourself, but if you want to be ready for that future that, that’s going to be increasingly distributed, the best way to operate is to just try and figure out what is the unfakeable ground truth. How do you operate in a manner that doesn’t trust the pixels on the screen in front of you or the, increasingly the voice coming out of the speakers. And that’s run things to ground, have a, a physical place, that’s see the person in, person in-person because a lot of these scams are not operating from near you physically because that would make them vulnerable to law enforcement in a way that uh, they’re not then operating internationally.

00:40:04] Bob: Unfakeable ground truth. What does that expression mean?

[00:40:08] Sean McGregor: That is looking at what can’t be produced at the moment by computers, and that’s often not being mediated by a computer in some form. It, it is going physical to an event that can be safe, that can choose its own hazards. But uh, going physical, looking at authoritative, trusted resources, and making sure that you’re communicating via those validated channels, verifying the domain of the email address that you’re receiving the emails from, being skeptical of anything that is not associated with a real identity, with the real world. Ask yourself, could I find this person given a, a few hours and determination? Could I knock on their door? And if you can’t answer yes to that, then be very skeptical, even if you can’t answer yes to that, you’ve got to be careful.