Smashing Security podcast #470: This AI security flaw might be impossible to fix

GRAHAM CLULEY So it's spin doctors, it's lawyers, because that's what you do when you have a serious security hole, isn't it? TANYA JANCA That's what they used to do though. When you used to report a bug, companies would sue you. Unknown Yes, you must be a hacker. Yeah, we're going to send the cops around. Smashing Security, Episode 470. This AI security flaw might be impossible to fix with Graham Cluley and special guest Tanya Janca. Hello, hello, and welcome to Smashing Security episode 470. My name is Graham Cluley. TANYA JANCA And I'm Tanya Janca. GRAHAM CLULEY Tanya, great to have you back on the show. Real delight to have you here. Now, you were on the show a little while ago, but you've got some exciting news. You're going to be signing copies of your new book. Tell us about it. TANYA JANCA Yes, I recently met some of the wonderful people at ESET. We were discussing how I was coming down to Vegas, 'cause I'm going to do a bunch of things at DEF CON. And they said, well, we have a booth at Black Hat. Did you want to show up at our booth and sign some books? So they have bought a ton of books. And so both days at Black Hat, I'm going to hang out at their booth and just give tons of books away and sign books and hang out. And I'm really excited. The folks at ESET are so great. GRAHAM CLULEY Oh, they're a nice bunch. Yeah, I've done some work with them in the past and they're actually sponsoring this episode of the podcast. They've got a really good antivirus product, but it's good to know that they'll also be handing out copies of your book. So this is the latest book from She Hacks Purple, right? TANYA JANCA Yes, it's Alice and Bob Learn Secure Coding. And so if you write code or quite frankly, if you're working with an LLM and it's writing code for you and you need to make sure that code is actually safe, this is the book for you for sure. GRAHAM CLULEY Yeah, make sure you go and visit the ESET booth at Black Hat and you may well bump into Tanya and get her to sign you a free copy of her book. Very nice. Not bad. TANYA JANCA At all, right? GRAHAM CLULEY Now, before we kick off, let's thank this week's wonderful sponsors, CoreView, Vanta, and ESET. We'll be hearing more about them later on in the podcast. JOE This week on Smashing Security, we're not going to be talking about how hackers were able to get Meta's AI to help them hack into Meta Instagram accounts. GRAHAM CLULEY You'll hear no discussion of how Canon has released firmware updates to fix security holes in more than 200 of its enterprise printers. JOE That could allow remote hackers to steal local domain passwords. And we won't even mention how hackers managed to steal the encrypted password vaults of some customers of password manager Dashlane after brute-forcing two-factor authentication. GRAHAM CLULEY So Tanya, what are you going to be talking about this week? TANYA JANCA I want to talk about how prompt injection might be forever. Cornell University wrote a paper and I think it's pretty interesting. GRAHAM CLULEY And I'm going to be asking you to take a deep breath if you've ever uploaded a passport scan to a website. Plus, don't miss our featured interview with Andrea Sivieri of CoreView, where he'll be discussing how hackers can lock your entire organization out of its Microsoft 365 environment without having to trick you into running a single piece of malicious code or handing over a password. All this and much more coming up on this episode of Smashing Security. Smashing Security. Now time for a quick word from our friends at CoreView. Joe, quick question for you. How confident are you in your Microsoft 365 security posture? JOE Graham, I don't even have a Microsoft 365 tenant. GRAHAM CLULEY Oh, for goodness sake, Joe, it's for our sponsor. Just play along with me, right? Picture the scene. It's Monday morning. You've got your coffee, you're wearing your second best hoodie, you're feeling pretty good about your Microsoft 365 setup because you checked Purview, you tightened conditional access, and frankly, you deserve a biscuit. JOE Biscuits? Okay, I'm in. I'll play along with you. Thank goodness for that. So, and then someone forwards you a breach report about a company that did all of that too. So how did they get hacked? Turns out some quiet little permission that crept wider over 3 years, a policy exception that nobody had reviewed, the kind of thing that's invisible until it isn't. GRAHAM CLULEY And this is exactly the stuff that CoreView's free Microsoft 365 Security Posture Check tool is designed to sniff out. It's the drift, the exceptions, the little permissions you stopped looking at because, well, you assumed they were fine. And the spoiler is that they're often not. JOE It's free. It runs locally on your own machine. It does not send your tenant data back to CoreView or anyone else for that matter. And if you'd like a hand setting it up, their team will happily walk you through it. So all you've got to do is visit smashingsecurity.com/coreview to download your free copy of the tool, and even you will be able to answer the question, how secure is your Microsoft 365 tenant? GRAHAM CLULEY And thanks to CoreView for supporting the show. Now, Tanya, you are someone who has been asked to give talks and speak at corporate events around the world in your time. Have you ever visited the UK? TANYA JANCA Yes, I have been to London many times, but I've actually only ever been to London in the UK. I've never seen the rest of it. GRAHAM CLULEY Oh my goodness. There obviously are some amazing other cities in the UK, but chances are that you may have had to get a visa or an ETA, an electronic travel authorisation to come here to do some work. And this is top of my mind at the moment because I realised that my, what was called an ESTA, that's the thing I have to sort out to get in and out of the United States, that's expiring in a few months. Anyway, if you've ever needed to apply for a UK visa or one of these electronic travel authorisations, you'll know it involves handing over some pretty sensitive information. GRAHAM CLULEY Things like a copy of your passport. You might have to take a selfie, some kind of proof of who you are. And my guess is that you would hope that the website you're uploading all of that information to is going to keep it safe and sound, right? TANYA JANCA Yeah, absolutely. GRAHAM CLULEY But what if that site wasn't even an official UK government website? What if your passport, your selfie, and even the precise GPS coordinates of exactly where you were when you took that selfie— I'm looking at your face in horror as I say this— what if all that was left sitting in an open Amazon storage bucket for anyone to stumble across. TANYA JANCA I literally just applied for an American work visa, so I'm literally imagining my data as you're saying this. GRAHAM CLULEY Right. So this was on a UK visa website, is the good news. But of course it could happen in other places as well. So this is all according to a great bit of reporting by Zack Whittaker over at TechCrunch. We reported on some other great research he did last week as well. And this is exactly what happened to customers of a site called UK Visa Portal. So if you needed a UK visa, would you have known to go straight to the official UK government website, which is gov.uk, or might you have ended up somewhere called UK Visa Portal? TANYA JANCA I definitely could have. I found the entire visa application process for the United States quite confusing. And then I actually hired a lawyer and I still found it very confusing. TANYA JANCA There's all of these sites that pretend to be the American site. GRAHAM CLULEY It's very easy to get the wrong one. There are these third-party sites which basically claim, oh, we will do this for you. Sometimes they charge you money when the actual process itself can be free of charge if you go directly to the government for whatever it is, and they're scooping up money. Other times they're just gathering your data and they're just shoving it over to the government website to process it, and they take their commission, don't they? TANYA JANCA Nothing's really free with the government in the United States in my experience. But anyway, that's okay. I'm not a citizen. They're not supposed to serve me anyway. But this is terrifying. Tell me more. GRAHAM CLULEY So in this particular case, the site is called UK Visa Portal. It also operates under a couple of other names like UK Visa and ETA Pass, because apparently just having one misleading name wasn't enough. It's not affiliated with the UK government in any way. It is a third-party commercial service, and the people who use it appear to genuinely believe that they are on the official UK government website. And when they get there, they pay their fee, they upload their passports and selfies, and they leave it to the site to submit the info for the visa or whatever documentation they need, not realizing that their documents are gonna be left sitting on a misconfigured Amazon S3 bucket. TANYA JANCA Okay, so no one thinks that's gonna happen. GRAHAM CLULEY No, that's not on the form. But what was happening was there was a bug on the website's backend, which made it possible to work out the addresses of all of those sensitive files on the web bucket. And according to the person who tipped off Zack Whitaker at TechCrunch, at least 100,000 documents were up there ready for anyone to snaffle up. TANYA JANCA That's so terrifying. Oh my gosh. Yeah, it sounds like IDOR, like insecure direct object reference. GRAHAM CLULEY Chances are it was exactly something like that. It would've been some little part of the URI or URL or some little code, and if you put that here, then you could access the information or you just increase the number each time and go through the entire collection. And this kind of thing just keeps on happening, doesn't it? I mean, it's not a sophisticated attack. It's some developers left t