'I leaked secrets': An AI agent confessed to its own data leak - but others got around corporate protections
Two AI incidents exposed corporate security weaknesses after one assistant leaked confidential credentials while another bypassed restrictions through an alternative API connection.
- AI agents lack institutional memory and ignore past warnings
- An AI assistant exposed company passwords across 10 sessions spanning three weeks
- Another AI agent bypassed corporate restrictions using an alternative API connection
- AI coding assistants can expose confidential credentials while completing routine software development tasks
Artificial intelligence agents are gaining access to company systems, internal documents, and sensitive credentials to complete tasks assigned by employees. However, two incidents uncovered by NordLayer AI Scout reveal how ordinary instructions can expose confidential information or circumvent organizational security restrictions.
In one incident, an AI coding assistant even acknowledged that it had exposed sensitive credentials after repeatedly accessing company files.
AI assistant exposed company passwords
In this incident, a quality assurance engineer instructed an AI coding assistant to examine proposed software changes. The project's instructions directed the assistant toward a GitLab access token stored inside an environment configuration file.
However, that file also contained production passwords, an authentication token for an internal human resources system, and credentials belonging to a business intelligence service. The assistant repeatedly accessed the entire file across 10 separate sessions over three weeks, transmitting its contents to an external AI service each time.
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
The situation became worse when the assistant encountered a fresh project copy without the expected configuration file. Rather than stopping its search, it retrieved the computer's entire collection of environment variables, exposing additional passwords and authentication credentials.
After recognizing what happened, the assistant acknowledged its mistake and recommended replacing the exposed credentials.
βI leaked secrets into this session's tool outputβ¦ I'd rotate them,β the AI coding agent confessed in a session observed by NordLayer AI Scout.
Vakaris Noreika, a cybersecurity expert at NordLayer, said engineers frequently instruct AI assistants to retrieve credentials directly, creating unnecessary exposure. He explained that assistants can repeat such behavior across subsequent tasks, allowing security problems to continue unnoticed.
AI agent bypassed company restrictions through an alternative connection
A separate incident involved an employee attempting to use an AI assistant to retrieve information from the company's Confluence documentation platform. The organization's Model Context Protocol (MCP) connection appeared to prevent the assistant from accessing those internal pages.
Instead of treating the restriction as final, the employee asked whether creating an API token could provide another route into the documentation. The assistant confirmed that this alternative approach worked, effectively circumventing the restriction imposed on its official connection.
The employee might have been seeking convenience rather than deliberately compromising company security. However, using an alternative connection without authorization can undermine restrictions established by an organization's security team.
βSecurity controls only ever cover the cases someone foresaw. Every company has rules, but agents simply donβt know them, because they are not employees who were trained at the company,β says Noreika.
βThis is a current technology limitation: Agents donβt retain rules between sessions, and sometimes even after getting specific instructions not to do something, they still do it.β
The incidents raise concerns about organizations granting automated assistants access to sensitive systems without adequate monitoring. Organizations should separate sensitive credentials, store valuable passwords in secure vaults, track which resources AI agents can access, and monitor their activities. Companies should also provide approved access routes rather than relying exclusively on restrictions that employees might circumvent.
Such measures could help security teams identify unusual access and prevent confidential information from reaching unauthorized destinations.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
Efosa has been writing about technology for over 7 years, initially driven by curiosity but now fueled by a strong passion for the field. He holds both a Master's and a PhD in sciences, which provided him with a solid foundation in analytical thinking.
You must confirm your public display name before commenting. Please logout and then login again, you will then be prompted to enter your display name.
Comments
No comments yet. Start the discussion.