stateofsecurity.com
The Top Three Things Security Teams Need to Understand About AI-Driven Attacks
The easiest mistake security teams can make about AI-driven attacks is to imagine a completely new species of adversary.
That is not what most organizations are facing.
The objectives remain familiar: steal credentials, gain access, persist, move laterally, collect data, commit fraud, extort the victim, or conduct espionage. What AI changes is the economics of the attack. It reduces the time, expertise, and attention required to move from one step to the next.
That distinction matters because it tells defenders where to act.
Google Threat Intelligence Group reported this week that it has observed adversaries moving beyond basic prompting into agentic workflows and AI-enabled automation. In one Q2 2026 case, a threat actor used an AI coding chatbot and agent instructions to help plan, build, and execute a mass credential-harvesting campaign in less than six hours. The framework automated scanning, troubleshooting, and IP rotation with limited human involvement. (Google Threat Intelligence Group, September 8, 2026)
That does not mean every attacker has become an autonomous machine. It means the constraints that used to slow attackers down are weakening.
Security teams need to understand three things.
## 1. AI Changes the Economics of an Attack Before It Changes the Objective
AI is most useful to attackers when it compresses work.
Reconnaissance can be summarized. Public vulnerability information can be translated into testable ideas. Phishing lures can be localized for a target’s language, role, industry, and current concerns. Malware can be debugged. Stolen data can be classified. Infrastructure can be configured. Failed steps can be revised without waiting for a specialist to return.
This is important because many defensive assumptions quietly depend on attacker friction.
We assume a small criminal group cannot personalize messages for thousands of targets.
We assume an inexperienced operator will make obvious technical mistakes.
We assume the time between vulnerability disclosure and reliable exploitation will give us room to patch.
We assume the volume of stolen data will slow an attacker before the most valuable records are identified.
AI weakens each of those assumptions.
Microsoft Threat Intelligence reported in March that most observed malicious AI use still centered on producing text, code, or media. Attackers were using it to draft lures, translate content, summarize stolen data, generate or debug malware, and scaffold scripts or infrastructure. Human operators generally retained control over objectives, targeting, and deployment decisions. (Microsoft Threat Intelligence, March 6, 2026)
OpenAI’s February threat report made a similar point: malicious actors typically combine AI with traditional tools, websites, social accounts, and other platforms rather than conducting an entire operation inside one model. (OpenAI, February 25, 2026)
The first defensive lesson is therefore simple:
> Do not look only for an AI artifact. Look for an attack chain whose cost and cycle time have changed.
An AI-written email may not carry a reliable linguistic signature. AI-assisted malware may still use familiar persistence and command-and-control techniques. AI-summarized stolen data still has to leave the environment. The defender’s best opportunity may be the behavior around the content, not proof of how the content was produced.
Security teams should ask:
* Which attack steps used to require scarce expertise?
* Which controls assume the attacker will be slow, noisy, or inconsistent?
* Which defensive queues already move more slowly than the exposure they manage?
* Where could an attacker use AI to test, fail, adapt, and try again at low cost?
The answer will usually point to a handful of urgent bottlenecks: exposed systems, identity recovery, fraud verification, vulnerability triage, detection engineering, and containment approval.
## 2. Agentic Autonomy Is Real, but It Is Uneven
The industry tends to make one of two errors.
The first is dismissal: AI is only producing better phishing emails.
The second is exaggeration: fully autonomous systems are already discovering unknown vulnerabilities, compromising enterprises, and completing every objective without human direction.
The evidence supports a more disciplined position.
Agentic systems are beginning to perform meaningful portions of real attack workflows. They can observe results, choose the next action, call tools, correct errors, and continue toward an objective. That is more consequential than generating a block of code in a chat window.
At the same time, autonomy varies by actor, task, tooling, access, and model reliability. In its September report, Google described operational agentic credential-harvesting activity, but also said it had not yet observed threat actors deploy fully autonomous zero-day exploitation pipelines against targets in the wild. Its observations instead showed a gradual maturation: adversaries were using AI to accelerate n-day exploit development, refine payloads, and assemble multi-stage chains. (Google Threat Intelligence Group, September 8, 2026)
Anthropic reached a related conclusion after mapping 832 accounts banned for malicious cyber activity between March 2025 and March 2026 to MITRE ATT&CK. Its analysis found AI use moving deeper into post-compromise activity. It also argued that the more durable signal of a dangerous actor is not simply the number of techniques used, but the scaffolding built around the model to chain attack stages together with less human input. (Anthropic, June 3, 2026)
This changes threat modeling.
Security teams should stop treating autonomy as a yes-or-no label. Instead, examine the attacker’s control loop:
1. What can the system observe?
2. Which decisions can it make?
3. Which tools or credentials can it use?
4. How does it determine whether an action succeeded?
5. What causes it to stop, escalate, or change tactics?
The faster that loop runs, the less time defenders have between initial signal and material harm.
That makes defensive latency a security control.
Teams need high-confidence detections for identity changes, token use, privilege escalation, abnormal cloud activity, suspicious repository access, and unusual data movement. They also need preauthorized, bounded containment actions that can slow an attack without waiting for an improvised executive meeting.
The objective is not to automate every response. Some actions can disrupt production, destroy evidence, or help the attacker understand what the defender knows.
The objective is to decide in advance which actions can safely happen at machine speed, which require human judgment, and how quickly that judgment must arrive.
## 3. AI Is Both an Attacker Tool and an Attack Surface
Most discussions about AI-driven attacks focus on what an adversary can do with a model.
Security teams must also consider what an adversary can do to the organization’s AI systems.
Models, prompts, agent instructions, source code, API credentials, cloud compute, vector stores, development tools, model gateways, and connected data sources now have value. They can be stolen, manipulated, poisoned, hijacked, or used as paths into other systems.
Google reported Q2 2026 incidents involving theft of proprietary models, prompts, skills, code, and AI research, along with campaigns that targeted AI credentials and victim cloud resources for unauthorized model workloads. It also described malicious open-source packages and project files designed to influence AI coding assistants or interfere with LLM-based security scanning. (Google Threat Intelligence Group, September 8, 2026)
This creates two connected threat models.
The first is **AI for the attack** : the adversary uses AI to accelerate reconnaissance, social engineering, exploit development, credential harvesting, post-compromise work, or data analysis.
The second is **AI as the attack path** : the adversary targets the AI supply chain, manipulates content consumed by an agent, steals model or service credentials, poisons instructions, or abuses the tools and permissions connected to an AI workflow.
A security program that watches only for AI-generated phishing sees only the first few feet of the problem.
The AI environment should be governed like privileged production infrastructure. At minimum, security teams should know:
* Which models, agents, coding assistants, gateways, and AI services are in use.
* Which identities and API credentials they use.
* Which tools, repositories, data stores, and production systems they can reach.
* Which untrusted content can influence their behavior.
* Where prompts, instructions, skills, memory, and configuration are stored.
* Whether tool calls and administrative changes are logged.
* How an AI service or agent can be isolated, suspended, or reconstituted.
The same principle applies to synthetic media and impersonation. In July, the FBI warned about scammers using AI-generated videos and spoofed sites to impersonate FBI personnel and revictimize people who had already reported fraud. The practical defense was independent verification through known official channels—not confidence that a person could visually identify every fake. (FBI Internet Crime Complaint Center, July 20, 2026)
That is a useful operating rule for enterprises as well:
> When a request can move money, change access, expose sensitive information, or alter production, verify the authority and the transaction through a path the request itself did not supply.
Do not make perfect deepfake detection the control on which the business depends.
## Use the Three-Question Test
For every significant incident, tabletop exercise, threat model, or intelligence review, ask three questions:
### Compression
Which parts of the attack became faster, cheaper, more personalized, or easier to repeat because of AI?
### Autonomy
Which observations, decisions, and actions could be chained together without waiting for a human operator?
### Exposure
Which AI assets, credentials, tools, data sources, and trust relationships could become targets or execution paths?
These questions are more useful than asking whether an attack is “AI-powered.”
That label does not tell the incident commander what to contain. It does not tell the vulnerability team what to patch first. It does not tell the IAM team which recovery path to harden. It does not tell the board whether the organization’s risk has materially changed.
Compression, autonomy, and exposure do.
## What Security Teams Should Do This Week
Start small and make the work concrete.
1. **Update one attack scenario.** Take a familiar credential-theft, ransomware, fraud, or supply-chain scenario and identify where AI could compress the attacker’s work.
2. **Find one defensive delay that matters.** Measure the time from the first credible signal to the first action that meaningfully constrains the attacker.
3. **Inventory the AI control surface.** Identify production agents, coding assistants, model credentials, connected tools, repositories, and sensitive data paths.
4. **Instrument the control loop.** Log identity use, tool calls, configuration changes, unusual compute consumption, repository access, and consequential agent actions.
5. **Rehearse independent verification.** Test a high-consequence request involving a convincing executive, vendor, help-desk, or finance impersonation.
None of this requires abandoning the security fundamentals.
Phishing-resistant authentication still matters. Least privilege still matters. Vulnerability prioritization still matters. Segmentation, logging, egress control, secure software development, tested recovery, and practiced incident response still matter.
AI does not repeal those controls.
It changes the speed and scale at which their weaknesses can be discovered, combined, and exploited.
The question security leaders should ask is no longer:
> Was AI involved in this attack?
The better questions are:
> Where did AI compress the attack chain, what did it allow the attacker to automate, and which AI systems became part of the path?
Teams that can answer those questions will be able to adapt without chasing every dramatic headline.
Teams that cannot may discover that the attacker did not need a fundamentally new technique.
It only needed to perform the old ones faster than the organization could respond.
* * *
## More Information and Assistance
MicroSolved, Inc. can help organizations:
* Assess AI-enabled threat exposure and attack paths.
* Threat-model AI agents, coding assistants, and connected workflows.
* Review identity, API credential, data-access, and tool-use controls.
* Test AI-accelerated fraud, credential theft, supply-chain, and incident-response scenarios.
* Design practical detection, containment, and recovery improvements.
Contact MicroSolved at info@microsolved.com or +1.614.351.1237.
Relax. We’re on watch.
_AI tools were used as a research assistant for this content, but human moderation and writing are also included._
