AINovember 22, 2026 · 3 min read

Together AI says a coding agent leaked the key to stored customer prompts

An AI coding agent committed a credential to a public repo, and someone used it to download prompt samples. The intruder tripped Dragos canaries from research that also led OpenRouter to drop three providers.

A laptop on a dark desk at night with one highlighted line of code on screen and a brass key resting on the keyboard, server racks glowing in the background
Illustration: Venture Block

Together AI says an AI coding agent working on one of its public code repositories committed a credential that opened a private store of customer prompts, and an unauthorized party used it to download those prompts before the leak was closed. The inference company disclosed the incident on Nov. 21 in a post signed by CEO and co-founder Vipul Ved Prakash, and it is now reviewing the data to work out which customers need to be told directly.

The disclosure lands in the same week that security firm Dragos published research showing how widely prompts leak as they pass between AI routers, resellers and hosting providers, and Together's account of its own breach leads straight back to that research.

How the key got out

According to Together, the company had collected prompt samples for routine software testing, which it says were authorized for that use, and kept them in a private Hugging Face repository. The samples came from requests sent to three open models it serves, MiniMax M3, GLM-5.2 and Kimi K2.6, mostly on June 24, July 4, July 8 and July 14, 2026.

While working on xorl, a public GitHub repository, an AI coding agent committed a credential with access to that private Hugging Face repository. Together says its automated scanning caught the exposed key and the commit was reverted. In the window before that happened, someone obtained the credential, downloaded the stored samples and then searched them for credentials to other services, testing whether those still worked.

Five of the credentials the intruder tried were planted. They were canaries from a Dragos experiment, fake secrets that exist only so their owner sees when someone uses them. Together says it has worked with an external forensics firm and law enforcement, has found no evidence of prompt leakage beyond this incident, and has not yet finished determining whether personal or other sensitive customer data was in the samples.

Two million tripwires

The experiment behind those canaries is described in a Nov. 19 post by HD Moore, Dragos' chief research officer. From July through November 2026, his team sent roughly two million prompts containing unique URLs and credentials to widely used AI services, then watched for any attempt to use them. Because each secret went into exactly one request to one provider, any use pointed back to the link in the chain that leaked it.

Moore reports three kinds of failure. Attackers had compromised at least three hosting providers, and none of the three knew until Dragos told them. In the largest case, a couple hundred bait AWS keys were stolen, including keys from requests the provider rejected before any model ran, which Moore reads as copying at the provider's front door. The stolen keys were then used for tens of thousands of AWS API calls, as attackers listed storage buckets, searched secrets stores and tried to switch on hosted models in Amazon Bedrock.

Second, some resellers on an inference marketplace were able to view and keep customer prompts with ordinary logging tools, without exploiting any flaw. Third, several providers appeared not to honor privacy settings such as Zero Data Retention (ZDR). Moore describes prompts sent to US providers reaching Chinese AI organizations, and a Chinese lab's systems fetching secret URLs from more than half the prompts sent to its models between July and October, including about two-thirds of requests marked no data collection or ZDR. Ars Technica's report on the research walks through how a single prompt is decrypted and readable at every router and provider it passes through.

OpenRouter pulls providers

The AI router OpenRouter responded on Nov. 22. In a post on its blog, it said the findings showed no breach of OpenRouter itself, but that it had off-boarded three providers, removed 125 endpoints from ZDR eligibility, some as a precaution, and notified all potentially affected customers. It plans to run checks modeled on the Dragos method on an ongoing basis, verify network paths for signs that requests are being forwarded to third parties, and require assurances before any removed endpoint gets its ZDR label back. Its list of provider disclosures so far names Novita, Phala and Together.

What changes

Together says it has moved to scoped or short-lived credentials, and keyless access where supported, checks every pull request for credentials, made ZDR the default for new accounts, and updated the rules governing how its engineers use AI coding agents. It advises customers handling sensitive prompts to turn ZDR on.

The incident is a concrete case of a risk that grows as companies hand more of their code to agents, a market that has drawn deals such as Meta's purchase of Conductor and LangChain's $300 million round. Moore's own warning is that agents will send whatever they can read, credentials included.

His recommendations start with self-hosting for the most sensitive work, which is getting more practical as open models such as Thinking Machines' Inkling 2 Mini fit on a laptop. Short of that, he suggests going directly to a first-tier provider, choosing upstream providers rather than accepting a router's default pool, enabling secret filtering, and seeding your own canary credentials. Dragos says it is now working with about a dozen providers on its findings.

Newsletter

The Block Brief

The stories that matter in startups and venture, in your inbox. Free.

Check your inbox to confirm.

Something went wrong. Try again.

More in AI