NEW YORK, Oct 21: Cybersecurity teams are facing a new challenge as artificial intelligence tools designed to help investigate digital attacks sometimes refuse to assist during critical stages of an incident, raising questions about how technology companies should balance security safeguards with the needs of defenders.
The issue has gained attention following a reported compromise involving OpenCode, an open-source AI coding agent. According to an October 21 report by Axios, the incident allowed attackers to obtain API keys belonging to some users of services provided by major AI companies, including OpenAI, Anthropic and Google.
API keys are digital credentials that allow software applications to access online services. If exposed, they can potentially be misused to access resources, run unauthorised requests or generate unexpected charges, depending on the permissions associated with the credentials.
The incident highlighted the growing importance of protecting AI development tools, which may have access to source code, development environments, cloud services and other sensitive resources.
It also exposed a dilemma for security professionals who increasingly rely on advanced AI models to investigate suspicious activity, understand malicious code and accelerate responses to cyber incidents.
According to the Axios report, security investigators initially used proprietary AI models to help examine the incident but later encountered refusals when attempting to continue parts of their investigation.
The problem illustrates how safeguards intended to prevent AI systems from facilitating cybercrime can sometimes interfere with legitimate defensive work. Security investigations may require examining malicious scripts, identifying vulnerabilities and reconstructing the methods used by attackers. Such activities can resemble harmful requests when assessed without sufficient context.
AI developers face the difficult task of distinguishing between legitimate security research and requests that could enable unauthorised access or other forms of digital abuse.
AI tools become part of cybersecurity operations
Artificial intelligence has become increasingly important in cybersecurity as organisations deal with large volumes of alerts, complex software environments and attacks that can spread rapidly across connected systems.
Security teams must examine logs, identify unusual network activity, review code, assess compromised credentials and determine whether attackers have gained access to sensitive information. These tasks can involve enormous quantities of technical data.
AI models can help summarise logs, explain unfamiliar code, compare suspicious behaviour with known attack patterns and organise information gathered during an investigation. Coding assistants can also help researchers understand how software components interact and identify areas that require closer inspection.
These capabilities can reduce the time needed to understand an incident, especially when security professionals are working under pressure.
However, AI systems do not independently establish that a request is legitimate simply because the person submitting it claims to be a defender. The same techniques used to investigate malicious software may also be adapted to develop malware, exploit vulnerabilities or bypass security controls.
Developers therefore introduce restrictions designed to prevent their models from generating instructions or code that could facilitate harmful activities.
Such controls are an important part of responsible deployment, but overly broad restrictions can make tools less useful when legitimate investigators need to examine dangerous code or reproduce an attack within an authorised environment.
Why safeguards can interrupt investigations
Cybersecurity work often requires technical analysis that appears risky when separated from its purpose.
For example, an investigator examining a compromised application may need to understand how a malicious script extracts credentials. A defensive team could also need to reproduce a vulnerability in a controlled test environment to confirm whether a system is exposed.
An AI model may identify elements of these requests as potentially dangerous because they involve credential theft, exploitation techniques or instructions associated with malicious software.
If the model refuses to provide assistance, investigators may need to rewrite their prompts, remove technical details, switch tools or perform the analysis manually.
These interruptions can increase the time required to understand an attack. During an active incident, delays may complicate efforts to determine the scope of a breach, revoke compromised credentials and prevent further unauthorised access.
At the same time, removing restrictions entirely would introduce significant risks. A model capable of producing detailed exploitation instructions for a defender could potentially provide similar assistance to an attacker.
The challenge is therefore not simply to make AI models more permissive. It is to develop systems that can recognise authorised defensive work with greater accuracy while maintaining protections against misuse.
Open-source and proprietary models face different trade-offs
The incident has also renewed discussion about the differences between proprietary AI services and open-weight models.
Proprietary models are typically accessed through services controlled by their developers. Providers can implement usage restrictions, monitor certain forms of abuse and update safety systems as new risks emerge.
These controls can help reduce misuse, but they may also prevent customers from carrying out some technically legitimate tasks. Users generally have limited control over the underlying model’s safety policies.
Open-weight models offer a different approach by allowing eligible users to obtain model parameters and deploy them in environments they control, subject to the applicable licence and technical requirements.
Organisations may be able to adapt such models for internal security workflows, manage access within their own infrastructure and customise their behaviour for particular investigative tasks.
Greater control, however, brings additional responsibilities. Companies deploying their own models must manage infrastructure, protect sensitive information, monitor model behaviour and establish safeguards appropriate to their operations.
Open-weight systems can also be misused if deployed without suitable controls. Their flexibility should not be mistaken for an automatic guarantee of accuracy, reliability or security.
For businesses, the choice between hosted and self-managed AI tools will depend on the sensitivity of their data, the complexity of their investigations, available technical expertise and their ability to manage risks.
Some organisations may choose a combination of systems, using managed services for routine analysis and controlled internal environments for specialised investigations.
Compromised development tools highlight credential risks
The OpenCode incident also underscores the importance of protecting the credentials used by software development tools.
AI coding agents can interact with repositories, execute commands and connect to external services depending on their configuration and permissions. If a tool or its surrounding environment is compromised, the consequences may extend beyond the immediate application.
Exposed API keys can provide access to services under the affected account’s permissions. Depending on the configuration, attackers may be able to make unauthorised API requests, consume computing resources or access data available through the compromised credentials.
Organisations can reduce these risks by limiting the permissions granted to development tools and separating credentials according to their purpose.
Security teams should avoid placing broad, long-lived credentials in environments that do not require them. Short-lived credentials, restricted permissions and regular rotation can help limit the potential damage if a key is exposed.
Companies should also monitor API activity for unusual usage, maintain records of access and establish procedures for quickly revoking compromised credentials.
Development environments need protection alongside production systems. Code repositories, automated workflows, package dependencies and third-party integrations can all become entry points for attackers.
As AI coding assistants gain broader access to business systems, organisations must evaluate their permissions and treat them as part of the overall security architecture rather than as ordinary text-generation applications.
Technology companies face pressure to improve AI security
AI providers are under growing pressure to make their products useful for legitimate cybersecurity work without giving malicious actors an easy route to dangerous capabilities.
One potential approach is to improve how models interpret the context of a request. Rather than relying exclusively on broad keyword-based restrictions, systems could consider the task being performed, the level of technical detail requested and the safeguards surrounding the activity.
However, contextual assessment is difficult. Attackers can misrepresent their intentions, and legitimate researchers may request information that could also be used offensively.
Additional controls could include specialised security-analysis modes, stronger verification for sensitive operations, isolated execution environments and clearer explanations when a model refuses a request.
These measures would not eliminate the risk of misuse, but they could help organisations understand the boundaries of a tool and identify appropriate alternatives when an investigation requires capabilities that a general-purpose model cannot provide.
Security teams also need contingency plans so that critical investigations do not depend on a single AI provider. Maintaining conventional forensic tools, documented procedures and access to alternative analytical systems can reduce the impact of unexpected service restrictions.
AI outputs must still be independently checked. Models can misinterpret logs, generate incorrect explanations or overlook important evidence. Human investigators remain responsible for verifying findings and deciding which actions are justified.
A broader question about trust and reliability
The growing use of AI in cybersecurity creates a new relationship between technology providers and the organisations that depend on their systems.
Businesses increasingly expect AI tools to be available when they need to investigate an attack, but providers must also ensure that their products do not make harmful activities substantially easier.
That tension is likely to become more significant as AI systems move beyond answering questions and begin executing longer sequences of tasks. Agents that can inspect code, use external tools and interact with digital services may offer greater productivity, but they can also introduce additional security and control challenges.
Organisations will need to determine which activities AI agents may perform independently, which require human approval and how their actions should be recorded.
The OpenCode case demonstrates why those questions matter even during defensive operations. Security professionals need tools that are both safe and dependable, particularly when the cost of delay can be significant.
Outlook
The incident points to a need for closer coordination between AI developers, cybersecurity researchers and organisations deploying automated security systems.
Providers must continue improving safeguards, while security teams need robust procedures for credential management, incident response and independent verification of AI-generated analysis.
Businesses should also assess the permissions granted to AI agents, restrict access to sensitive systems and ensure that alternative investigative methods remain available.
The long-term objective is to develop AI systems that can support legitimate security work without providing unrestricted assistance for cybercrime.
Achieving that balance will require more than simply reducing refusals or strengthening restrictions. It will depend on better contextual understanding, transparent controls, secure deployment practices and clear accountability for how AI tools are used.
As cyber threats and AI capabilities continue to evolve, the reliability of defensive tools will become an increasingly important part of organisations’ overall security strategies.