Railway tells users to rotate every env variable after a breach

If your team runs anything on Railway, open the dashboard before you keep reading. On 21 October, Railway published a security disclosure saying that an unauthorized third party reached encrypted environment variables stored on the platform. For some variables, the attackers obtained the decrypted contents, including the kinds of secrets most teams keep there: API keys, database credentials and private registry credentials.

Railway's post calls the breach part of "a series of attacks recently targeting platform companies with the aim of compromising AI inference providers". The company notified upstream providers to revoke keys it was told to rotate, and paused all deployments on the platform while the investigation ran.

What Railway is asking users to do

The post is specific. Every user is told to immediately roll every environment variable stored in Railway, including every API key, every database and registry credential, and anything else held there. Then review logs for those services for unusual activity. Railway says it will email customers it believes are directly affected and that status updates will continue to appear at status.railway.com.

Rotating one secret at a time is slow when the exposure is a whole account's worth, so the practical move is to list every service that reads a key out of Railway, generate a new key or credential for each, roll it in and revoke the old one. Private registries are easy to miss; the disclosure names them.

Why an environment variable is a secret the platform can read

The thing a disclosure like this makes clear is that an environment variable held by a hosting provider is not an end-to-end-encrypted secret. It is a secret the hosting provider can decrypt, because it has to pass the value into a running container. Railway says the attacker reached decrypted contents for some variables, which is the plain reading of that trust.

For teams using Railway as the glue in front of several LLM providers, the account's secrets are the provider keys, and the provider keys open paid balances and model access on OpenAI, Anthropic, Google and others. That is why Railway's own description points at AI inference providers as the target of the wider campaign: hitting one platform account exposes several provider accounts at once.

The post does not give numbers, so the share of users whose variables were decrypted is not in the record. It also does not say how the attacker reached the platform. If you want to decide whether to leave Railway over this, both of those will matter and are not yet available; what is available is the instruction to rotate.

Source

Railway: Security disclosure, Wednesday 21 October. Discussion on Hacker News.