EXCLUSIVE Several Google Cloud customers say their API keys have been compromised and used by bad actors to run inferencing workloads using the most expensive video and picture models, leaving them with bills for tens of thousands of dollars and weeks of back-and-forth headaches with the Chocolate Factory as they tried to prove they were not responsible for the mess.
The problem is being hashed out on social media, with sites like Reddit collecting stories from Google Cloud users that seem to follow a similar pattern: After months or years paying small monthly bills to Google Cloud for access to tools like Maps, their API keys are discovered, and in minutes they are charged thousands of dollars for API calls to Nano Banana and Veo 3.
Google told The Register this is an industry-wide problem and not a security issue specific to Google. It said the vast majority of these incidents happen due to compromised user credentials such as API keys inadvertently leaked on public code repositories like GitHub, and malicious actors who are actively scraping public repositories.
Google said it encourages all customers to implement robust security practices, including enabling multi-factor authentication, routinely auditing API keys, and ensuring credentials are never committed to public repositories.
But those explanations are complicated by developers and security threat researchers who said there are thousands of accounts which are following Google’s own site configuration rules by placing their APIs in a public client.
Additionally, one user told The Register they had spending caps in place that should have stopped any bill over $250. Yet according to Google those caps can be automatically upgraded to $100,000 – without user input – if the user has spent a total of $1,000 throughout the life of the account, and the account is more than a month old.
‘What the hell’s going on?’
Rod Danan is CEO of Prentus, a company that helps job applicants with interview preparation and tracks job placements for universities. He uses API calls to Google Maps as a part of his platform. For years his bill never topped $50 a month, he told The Register. Then in March he got an email alert from Google saying he was being charged $3,000 and panic took hold.
“It’s just ‘Boom, we just charged you $3,000.’ I’m like, ‘What the hell’s going on?’ And then you go into the application, like, ‘What is triggering this? What is the source?’ So just determining that is honestly not that simple,” he told The Register. “As I’m searching, five minutes go by and another $5,000 get charged. I’m like ‘What the hell is going on? It’s just draining my money.’ ”
Despite the spending caps he said he had in place, by the time he shut down the API minutes later, his credit card had been charged $10,138 almost entirely from Veo 3 video generation and Gemini image output tokens, which are services he has never used and have zero connection to his product.
Google told him it found no evidence of fraud and has thus far refused to issue a refund. But what makes this especially frustrating for Danan is that he said he was following Google’s advice in exposing the API key in the first place.
“You have this Google Maps key, which you know, everyone uses, and the guidance from Google is you’re supposed to load it in your front end. So we did that, and all of a sudden they changed the keys so that the Google Maps key, which is exposed publicly, could be used for Gemini, and then they didn’t disclose that to customers,” he said. “So then, all of a sudden, I just get multiple emails in a row. It’s like $3,000, $5,000, $10,000 charged on your Google account.”
In February, security researchers at Truffle Security Co. published an article warning Google users that their Maps API keys were no longer safe to share publicly. For years, if a coffee shop wanted to place its logo and website on Google Maps, the instructions from Google were to download the widget and upload an API key that linked their site to Google Maps, said Joe Leon, the threat researcher who wrote the warning. He told The Register that about three years ago, Google started allowing those same public API keys to also access Google Gemini models.
“You have all these people that we’re told to like for Maps, ‘Put this key in public.” Now maybe it’s them, maybe it’s someone else in their organization, someone enabled the Gemini API in that same project,” he told The Register. “Now that same key can be used to both access Maps, and also Gemini. That’s the core of what I found.”
He said the first few characters of those API keys followed a particular naming convention: A-I-Z-A. A search of millions of web pages found 3,000 of those Google keys that were first deployed for Maps and are now able to access Gemini, leaving those sites vulnerable to high-dollar credential attacks.
In an email to The Register, Google said it tells users not use the same API key for multiple APIs, and especially through API keys that could be client-facing (browser keys). It recommends to always apply API client restrictions – for example, to restrict the API key to a specific service and apply client application restrictions like “HTTP referrer”, “IP address” , “Android apps.”
Google said it now mandates that users configure API restrictions when they create API keys. Additionally, the company said, it’s no longer possible to create a key that can access both Gemini and Maps.
Leon agrees that Google has taken steps to lock down access since his paper was published.
“The first thing that I’ve seen is they’ve rolled out a new Gemini API key type, which is unrelated, as best I can tell, to the Google API key. So it’s prefixed with capital ‘A,’ capital ‘Q’ ” he said. “Since I published that post, they’ve taken a lot of steps to try to lock this down. The spending caps I saw, they put that in place. I didn’t know that they auto increase it. So that kind of defeats a little bit of the purpose.”
About those spending caps
Developer Isuru Fonseka, based in Sydney, Australia has been building apps in the Google Cloud environment for 10 years. He’s got a side project he has been working on for about two years, but says he’s never exposed the API key that he uses to access his work inside Firebase. Additionally, he set a hard budget cap at $250.
Like Danan, he was alerted to a sudden spending spike with Google on April 29. The attack was so out of character with his purchase history that his credit card company refused the charges.
“I just woke up to a couple of emails where my credit card provider declined a number of transactions,” he said. “So then I logged into GCP to have a look. When I look into transactions, I can see that all these charges are coming through. Some are declined, but previously, there’s like, one for $500, $1,000, or $2,000. These ones went through successfully.”
He reached Google support to flag the spending, ask them what had caused it, and to shut it down, but it takes up to 36 hours for Google support technicians to be able to view a customer’s usage. Google told The Register this is actually faster than industry standard, but for Fonseka, it was still infuriating.
“This was probably the most frustrating part,” he said. “There’s this weird mechanism where they can detect enough to charge your card, but not enough to show you what it is being used on … The damage ended up being in the range of like AUD $17,000 ($12,000) .”
But Fonseka said even if someone were to brute-force his API key, his Google Cloud budget cap was set at Tier 1, which was locked at $250, meaning he should never have been able to spend AUD$17,000 on AI services.
“But when I logged in after the attack, it was set to like Tier 2 or Tier 3, which was like $100,000. I would have never set this,” he said. “I spoke to someone actually in Australia who was also affected by this, and he said that, based on your account standing they automatically upgrade the tier. So if they did, that is just a terrible decision, so they must have automatically upgraded mine.”
Google told The Register it looks like Fonseka might be right.
“What we believe happened in this instance you have shared is the attacker didn’t change the tier; the developer’s usage (driven by the attacker) triggered Google’s automated systems to raise the ceiling, based on meeting Tier 3 qualification of Gemini API, which included at least $1,000 USD in payments to Cloud and 30 days since the first payment,” Google told The Register via email.
In a revamped policy move announced March 16 Google said it would make it easier for users to access higher dollar quotas in GCP by reducing the spending qualifications to reach the next tiers. Additionally, the system “automatically upgrades you to the next tier as your usage grows.”
“You get access to higher rate limits and increased monthly quota as soon as the criteria is met,” Google said on its blog titled “Giving you more transparency and control over your Gemini API costs”
Customers like Fonseka in the first tier would be automatically moved to the next tier – $2,000 – if they spend $100, and then automatically to Tier 3 if they spend $1,000 and have been a customer for 30 days.
Tier 3 has a spending cap between $20,000 and $100,000.
Fonseka said he was tempted to call his credit card company and have them charge back the cost, but he fears that would likely result in the suspension of his project inside Google Cloud, which customers are relying upon.
Danan told The Register that he is in the same boat.
“Even though I had spend caps on it didn’t really matter, like, all you get is alerts,” he said. “I still need Google APIs. I can’t get kicked off because then my app won’t work. We need the Maps API. So there’s sort of a disincentive for you to report this is fraudulent activity to your credit card company.”
Both Danan and Fonseka said they are still negotiating with Google to win a refund. ®
