Use it responsibly · Guide 8 of 10
Use AI to learn and defend, and only test what you have permission to test.
People doing IT admin tasks in a Microsoft lab trial were 34.53% more accurate with Security Copilot. It was a vendor test of its own product.
Try this
Here is a log with private details removed: [paste]. Explain it in plain English, say what looks unusual, and what I should check next. Mark anything you are guessing.
AI can get you to a first rule quickly. You still need to test it against normal activity on sample data.
Try this
Write a Sigma rule to detect [behaviour] in [log source]. List likely false positives and how I could test it on sample data.
Ask how a technique shows up in logs and how to stop it, so you practise thinking like a defender.
Try this
Explain password spraying to a beginner. How does it appear in logs, how would a defender spot it, and what stops it?
A clear report is part of the job, and a good first task to hand to AI.
Try this
Turn these notes into a short incident summary: [notes with private details removed]. Cover what happened, the impact and next steps. List anything I did not confirm.
Only test systems you own or have written permission to test. Use a lab or sample data to practise.
Logs can hold names, addresses and internal hostnames, so strip them before pasting.
Confirm every finding in the data yourself.
A person should approve anything that blocks, deletes or changes a system.
Anthropic reports a group it assesses as state-sponsored used Claude Code against roughly thirty targets. This is a vendor self-report, not independently verified.
The NCSC and OWASP both treat prompt injection as unsolved, so limit what your security tools can do automatically.
Notes, evidence and sources
3 sources, each with its catch.
Bono and Xu, Microsoft · October 2024
181 people doing IT admin tasks in a simulated lab were 34.53% more accurate with Security Copilot and took 29.79% less time overall.
Microsoft staff testing a Microsoft product, in a lab. Multiple-choice questions took longer with it.
In a randomised lab trial (Bono and Xu, Microsoft, October 2024), 181 participants recruited through Upwork completed IT admin tasks in a simulated environment, with half given Security Copilot. "Copilot subjects experienced a 34.53% improvement in overall accuracy and a 29.79% reduction in task completion time." Multiple-choice questions took 25.13% longer with it.
Written by Microsoft staff about a Microsoft product, in a simulated lab rather than live work. The discussion section was not read in full.
Strength: Moderate, vendor-run
Anthropic · 13 November 2025
Anthropic reports that a group it assesses as Chinese state-sponsored used Claude Code against roughly thirty targets and succeeded in a small number of cases.
A vendor self-report, not independently verified.
Anthropic reported (13 November 2025) that a group it assesses "with high confidence" as Chinese state-sponsored used Claude Code against roughly thirty targets and "succeeded in a small number of cases". Anthropic says the AI did "80-90% of the campaign", with humans at about four to six decision points, and that Claude "occasionally hallucinated credentials", "an obstacle to fully autonomous cyberattacks".
Vendor self-report, not independently verified. Words such as "first" and the 80-90% figure are Anthropic's framing, and Anthropic says it only sees its own product's usage. No independent rebuttal was retrieved.
Strength: Weak to moderate
No independent evaluation of AI in security operations centre (SOC) work was found. The sources located were vendor-heavy or vendor-sponsored, so this guide gives no SOC effectiveness figures. The one SOC forecast found is listed on Sources.
1. Bono and Xu, Security Copilot trial · October 2024
2. UK Government AI Playbook · 10 February 2025
3. Anthropic's report of an AI-driven espionage campaign · 13 November 2025
4. Prompt injection as an unsolved problem