Infrastructure left open
On 29 January 2025 Wiz Research found two DeepSeek hosts with a ClickHouse database reachable from the internet with no authentication. It allowed full SQL control and held a log table with more than a million entries of plaintext chat history, API secrets and backend details. DeepSeek closed it within hours of being told. The exposure fed directly into regulatory action: Italy’s data protection authority blocked the DeepSeek app the next day. [1][2]
The same pattern repeats at the layer of apps built on top of models. In January 2026 a misconfigured Firebase backend behind the Chat & Ask AI app allowed public reads of roughly 300 million messages from more than 25 million users; it was fixed within hours of the report. In August 2026 Cybernews found an unsecured Azure storage bucket belonging to the aggregator 302.ai holding 17.6 million files and 22.7 terabytes, including archives of conversation data and enterprise customer-service bot transcripts; it was closed in September. In November 2025 OpenAI disclosed that a breach at its analytics vendor Mixpanel exposed names, email addresses and approximate locations of API users, though no prompts, keys or payment data. [3][4][5]
Self-hosted model servers are now a category of their own. A SentinelOne and Censys scan in January 2026 found about 175,000 Ollama hosts answering on the public internet; LeakIX counted 12,269 with no authentication the following month; Intruder found 31% of the Ollama APIs it queried would run a prompt for anyone. In May 2026 Cyera disclosed CVE-2026-7482 in Ollama, estimating around 300,000 vulnerable instances. Anyone can run inference on the owner’s hardware, pull or register models, and in some configurations read memory. Google’s threat intelligence group confirmed in November 2026 that attackers scan for exposed Ollama endpoints and enrol them into resold compute pools. [6][7][8][9]
Share buttons that published private conversations
In July 2025 Fast Company reported that ChatGPT conversations shared with a “make this chat discoverable” option had been indexed by Google. About 4,500 were found by that method; 404 Media later assembled a dataset of around 100,000. The content included mental-health, legal and workplace material. OpenAI removed the option within a day, called it a short-lived experiment and worked with search engines to de-index. [10][11]
Three weeks later Forbes found that Grok’s share button generated public URLs with no warning and that Google had indexed more than 370,000 of them, including names, at least one password, uploaded spreadsheets and instructions for harmful activities. In June 2025 the Meta AI app’s Discover feed had already shown users’ medical questions, relationship problems and home addresses to the public, posted through a share flow many did not understand was public; Meta added a warning. In none of these cases was anything hacked. The product made publishing easy and the consequences invisible. [12][13][14]
Shadow AI: what employees paste, and how often
The origin case is Samsung in April 2023, when engineers pasted internal source code and meeting notes into ChatGPT and the company banned generative AI on its devices. Three years of measurement since then show the behaviour has scaled with adoption rather than fallen away. [15][16]
| Measurement | Figure | Source and date |
|---|---|---|
| Enterprise GenAI use through personal accounts | 72% of use (2025); 47% of users, down from 78% (2026) | Netskope Cloud and Threat Report, 2025 and Jan 2026 [17][18] |
| GenAI data-policy violations | Doubled in 2025; 223 per organisation per month on average, 2,100 in the top quartile | Netskope, Jan 2026 [18][19] |
| Largest violation category | Source code, 46% of violations | Netskope, 2025 [17] |
| Prompts containing sensitive data | 8.5% (Q4 2024); 4.37% of prompts and 22% of uploaded files (Q2 2025); 2.6% of 22.5 million prompts and uploads (full-year 2025), 71% of exposures via ChatGPT | Harmonic Security [20][21][22] |
| Employees who paste data into GenAI; share in unmanaged accounts | 77%; 82% of those pastes in personal accounts; 22% of pastes contain PII or payment data | LayerX, Oct 2025 [23][24] |
| AI interactions involving sensitive data | 34.8% (2025); 39.7% (2026, 222 companies) | Cyberhaven Labs [25][26] |
| Organisations reporting a breach of an AI model or application; share lacking AI access controls | 13%; 97% of those | IBM Cost of a Data Breach, Jul 2025 [27] |
| Extra breach cost where shadow AI was involved | US$670,000 above the US$4.44 million average; shadow AI in 20% of breaches | IBM, Jul 2025 [27][28] |
Two cautions on these numbers. They come from security vendors measuring their own customers, so they describe organisations that already bought monitoring, and they define “sensitive” differently (Harmonic’s falling percentage partly reflects a larger denominator as routine use grew). The direction is consistent across all four vendors and IBM: the volume of corporate data entering models is rising, a large share goes through accounts the company cannot see, and the organisations that get breached are overwhelmingly the ones with no access control on AI use.
LLMjacking: your keys, their usage, your bill
Sysdig coined the term in May 2024 after watching attackers use cloud credentials stolen from a vulnerable web server to call Bedrock, OpenAI, Azure and Anthropic models through a reverse proxy, and modelled the worst-case cost at more than US$46,000 a day. By September 2024 they had observed more than 85,000 Bedrock requests from one campaign, 61,000 of them in a three-hour window, and raised the modelled ceiling to US$100,000 a day with the newest models; one victim reportedly owed about US$30,000 after three hours. A Permiso honeypot key hijacked for an AI chat service generated US$3,500 of charges in two days. [29][30][31][32]
The market matured fast. Within days of DeepSeek’s releases in December 2024 and January 2025 the proxies had added it; Sysdig found more than 55 misused DeepSeek keys and one proxy that ran up nearly US$50,000 in four and a half days, reselling access for US$30 a month. Microsoft’s Digital Crimes Unit sued the group it called Storm-2139 in December 2024 and named four developers in February 2025: they had scraped Azure OpenAI keys that customers leaked publicly, built tooling to strip the content controls and sold access used to generate non-consensual imagery. [33][34][35]
Google’s Threat Intelligence Group published the most complete picture on 2 November 2026. Fraudulent charges exceeded US$100,000 in a single day in individual incidents and reached about US$1 million at some organisations before keys were revoked. One token-relay service pushed nearly US$1 million of token costs onto a single victim and sold “unlimited” access for about US$60 a month. One actor made more than 500,000 API calls in under eight hours. The access vectors were unglamorous: misconfigured service-account keys, hijacked session tokens, stolen GitHub personal access tokens, keys scraped from public repositories and validated with one-token probe requests to stay under volume alerts, and the exposed Ollama endpoints described above. Google’s recommendations were spending caps, per-model key scoping, secrets scanning, phishing-resistant MFA, patching self-hosted gateways and alerting when a model service is enabled in an account that never used one. [9][36]
Where the keys come from
GitGuardian’s annual scan of public GitHub found 23.8 million new hard-coded secrets in 2024 and 28.6 million in 2025, the largest year-on-year jump it has recorded. Secrets for AI services rose 81% to 1.28 million, including 113,000 DeepSeek keys in the provider’s first year. The 2026 report also found 24,008 secrets inside MCP configuration files, 2,117 of them still valid, and that repositories using AI coding assistants were roughly twice as likely to leak a secret. Sixty-four percent of secrets that were valid in 2022 were still usable in January 2026. [37][38][39]
Keys also leak into the training data itself. Truffle Security scanned the December 2024 Common Crawl, 400 terabytes and 2.67 billion pages that feed many models including DeepSeek’s, and found 11,908 live secrets of 219 kinds, from AWS root keys to Slack webhooks. Lasso Security’s earlier finding of 1,681 valid Hugging Face and GitHub tokens, 655 with write access to models used by 723 organisations, shows the same problem on the model-hosting side. [40][41][42]
Runaway spend without an attacker
Not every oversized bill involves a thief. OWASP lists Unbounded Consumption, including “denial of wallet”, as LLM10 in its 2025 Top 10, and its agentic list treats uncapped autonomy as a first-class risk. The mechanics are described in Konsole’s own cost-control webinar: an agent calls the same tool repeatedly, pursues a task with no valid end state, or asks the model to reason about the same problem in a loop; a retry policy hammers an endpoint that is returning an error; a conversation re-sends its whole history on every turn so that turn ten costs many times turn one. [43][44]
The economics make the gap between a controlled and an uncontrolled call enormous. At November 2026 list prices a frontier model costs on the order of US$8 to US$10 per million input tokens and US$40 to US$50 per million output tokens, while a small model costs about ten cents and fifty cents. A key or an agent that can reach the frontier model when the task needed the small one is paying up to a hundred times more per token, which is why Google’s LLMjacking report and every cost-governance guide say the same thing: scope keys to a model, cap spend per key, and alert in real time rather than at invoice time. Well-sourced public post-mortems of accidental loops at large scale are rare, which is itself a finding: companies disclose breaches because they must, and disclose bills because they choose to. [45][46]
What regulators have done so far
- Italy, OpenAI, €15 million, December 2024. The Garante fined OpenAI for training ChatGPT without an adequate legal basis, transparency failures and no age verification. In March 2026 the Court of Rome annulled the decision on procedural grounds (Ireland had become the lead authority), without ruling on the merits. [47][48]
- Italy, Luka (Replika), €5 million, May 2025; Character.AI, €158,000, July 2026. Both for consent and age-verification failures; Character.AI also for a late data-protection impact assessment. [49][50]
- Italy, DeepSeek, January 2025. App blocked the day after the Wiz disclosure; still in force. Berlin’s authority reported the app to Apple and Google in June 2025; South Korea issued a corrective order. [51]
- EU AI Act. Prohibited practices applied from February 2025 and general-purpose model obligations from August 2025. The Digital Omnibus on AI, in force 27 July 2026, moved standalone high-risk obligations from August 2026 to December 2027. [52][53]
Enforcement so far has targeted consent, transparency and children rather than the leakage mechanisms in this report. That will not hold. Chat logs are personal data, exposed chat logs are a breach, and the first regulator to treat an indexed share link or an open log database as a notifiable incident will set the precedent.
The controls that map to each failure
| Failure | Control | Where it lives |
|---|---|---|
| Sensitive data in prompts reaches a third-party model | Detect and mask PII, PHI and secrets before the request leaves your network; restore in the response | Gateway, inline on every call (masking) |
| Employees use personal accounts the company cannot see | Route sanctioned access through one endpoint with per-user tokens; log every call with its source | Gateway and identity |
| Stolen or leaked model keys billed to you | Keys held in a vault, never in code; developers get short-lived tokens; per-key model scope and spend cap; secrets scanning on repositories | Gateway (platform) and CI |
| Agent loops and retry storms | Token budget per run, tool-call ceiling, loop detection, hard stop with a human-held kill switch | Gateway (cost control) |
| Model servers and databases left open | Bind to localhost, authenticate at a gateway, scan your own ranges for listening model ports | Network and platform |
| Share features publish private content | Disable public sharing for workplace accounts; treat shared links as data exports | Policy and tenant settings |