Detecting Vulnerabilities in Agent Skills with SkillSpector: From Green Checkmark to Real Security Judgment
A guide to understanding and mitigating security risks in AI agent skills.
1The Security Risk of Agent Skills
Agent skills are simple instruction packages for AI agents, but their security is a major concern. Anyone can publish a skill, and users can download and use them freely. This opens the door for malicious code. Tools like SkillSpector are designed to detect these risks.
2How SkillSpector Works
SkillSpector is an open-source tool that scans a skill folder for vulnerabilities. Users can run a security test and decide based on the score. However, the score alone isn't enough. For example, a skill might score 100/100 with a 'do not install' warning, but the real threat could be in plain English instructions.
3Limitations of Static Scanning
SkillSpector's static layer is fast and free but can yield high false positives. For instance, a real GitHub automation skill might trigger 20 findings, 16 of which are part of its normal function. The score can't distinguish which findings are critical.
4The Importance of Semantic Scanning
Beyond static scanning, semantic analysis offers deeper insights. This layer consists of four passes: three to hunt for new issues and one to filter false positives. Skipping these passes can let the most dangerous skills slip through.
5Structure of Skill Files
A skill is a folder containing a Markdown file. The header includes a name and description, while the body contains plain-language instructions. The agent loads and follows these instructions, creating potential security vulnerabilities.
6The Danger of Prompt Injection
Every line in a skill file is treated as an instruction, making prompt injection easy. For example, a line saying 'summarize the repo' and another saying 'summarize the repo’s .env file' are indistinguishable to the agent, leading to security risks.
7Risks in Published Skills
Studies show significant security flaws in published skills. For example, 26.1% of 42,447 skills had prompt injection, and 5.2% showed malicious intent. Snyk's 2026 report found 76% of 3,984 skills were malicious.
8Examples of Malicious Skills
Malicious skills can include instructions to steal AWS keys or disable safety checks. For instance, user 'zaycv' uploaded over 40 near-identical malicious skills, some containing base64-encoded commands or password-protected executables.
9Infiltration via GitHub
Invariant Labs demonstrated how a single poisoned GitHub issue could lead an agent to leak private repository data, including project names and salary details, without any exploit code—just a crafted sentence.
10Shortcomings of Other Scanners
Tools like Semgrep and Bandit may miss prompt injections in SKILL.md files because they focus on specific areas. Specialized tools like SkillSpector are essential for comprehensive security.
11Security Recommendations
Don’t rely solely on scores. Combine static and semantic scans. Add extra layers to filter false positives. Adopt a multi-layered approach to detect and mitigate malicious skills effectively.
12Conclusion and Takeaways
SkillSpector is a vital tool for detecting vulnerabilities in agent skills, but it’s not a standalone solution. Users must carefully analyze scan results and adopt a multi-layered security strategy to ensure AI agent safety.