Why evaluation matters
Installing a skill is easy, which is exactly why evaluating one matters. A skill becomes part of how your agent thinks and acts. A sloppy skill produces sloppy work; a risky one can do real harm. Spending a few minutes on a clear set of criteria before you install saves far more time later. Since every skill is plain text and files, you can judge it directly by reading it.
Five dimensions cover most of what you need: trigger clarity, focus, documentation, portability, and risk.
Trigger clarity
The skill's description tells the agent when to use it. Read it and ask whether it names a specific situation, input, and goal. A sharp trigger like "use when reviewing a pull request diff before approval" leads to reliable behavior. A vague one like "helps with code" leads to a skill that fires at the wrong time or sits unused. If the trigger is fuzzy, expect inconsistent results.
Focus
The best skills do one job. Open the body and check whether the steps stay on a single workflow. A skill that bundles three unrelated tasks is harder to trust, harder to maintain, and more likely to behave unpredictably. Narrow and deep beats broad and shallow nearly every time.
Documentation
Good documentation is a signal of care. Look for clear, ordered steps, a stated output format, and a note about when not to use the skill. If supporting files exist, they should be explained, not dumped. Well-documented skills are easier to verify, easier to edit, and far more likely to have been tested on real cases.
Portability
A portable skill makes few assumptions about your specific setup. Check whether it hard-codes paths, depends on one obscure tool, or assumes a particular environment without saying so. The more it leans only on capabilities most agents already have, the more cleanly it will drop into your project. Portability also makes a skill easier to share across a team.
Risk
This is the dimension to never skip. Read the skill with a security eye and look for three things:
- Scripts: does it run helper code? If so, can you read and understand that code?
- Network: does it make outbound calls, and to where?
- Credentials: does it ask for or touch secrets, tokens, or keys?
None of these is automatically disqualifying, but each demands scrutiny. A skill that quietly runs an opaque script or reaches out to an unexplained endpoint should not be installed. The deeper treatment lives in are agent skills safe.
Putting it together
Run all five checks quickly. A skill with a clear trigger, tight focus, honest documentation, good portability, and no unexplained risk is one you can install with confidence. If any dimension raises a flag you cannot resolve by reading the source, pass on it.
Directory curation provides a first pass on source and presentation. Browsing the Skills directory or a category narrows the field to candidates worth your own review — it does not replace reading the SKILL.md yourself.
A quick scoring habit
If you evaluate skills often, it helps to make the five checks a fast, repeatable pass rather than a vague gut feeling. Read the trigger and decide in one sentence whether it is specific. Skim the body and decide whether it stays on one job. Note whether the documentation includes steps, an output format, and limits. Scan for hard-coded assumptions that hurt portability. Then do the risk read: scripts, network, credentials. Five quick yes-or-no judgments give you a clearer verdict than a long, unfocused read, and they make it easy to compare two competing skills side by side.
Watch for mismatched value and risk
A useful final question is whether the risk a skill carries is proportionate to what it does. A skill that merely reformats text but insists on network access and credentials is suspicious — the cost does not match the benefit. By contrast, a skill that genuinely needs to call an external service to do its job, and explains exactly why, is easier to accept. Always weigh what you are getting against what you are exposing. When the two are out of balance, walk away.
Test before you trust
Even a skill that passes every paper check deserves a trial run. Give it a small, low-stakes task and watch what it actually does, including any script it runs and any request it makes. Behavior on real input is the ultimate evaluation, and it sometimes reveals gaps that reading alone misses.