AI NewsDev toolsAnnouncement

NVIDIA releases SkillEvaluator, a quality check for Claude Code and Codex skills

NVIDIA released a free tool that scores how much a Claude Code or Codex skill helps the agent.

AI News

Editorial2 min read

LinkedInX
NVIDIA SkillEvaluator repository social card on GitHub

Image: GitHub

Why it mattersWriters of Claude Code and Codex skills now have a repeatable score. A team can tell a good skill from one that only sounds good.

A team writing skills for a coding agent has no clear way to grade them. A good skill and a clever-sounding one look the same on the page. NVIDIA released a tool that scores them. The project is called SkillEvaluator. It ranked 10th on Trendshift's daily list on 16 November.

NVIDIA released version 0.9.0 on 13 November. The code is on GitHub at NVIDIA/SkillEvaluator under Apache 2.0. The repository had 1,754 stars and 234 forks. The README calls the project "experimental" and "community-supported, best-effort, with no SLA".

What a skill is here

A skill is a folder of instructions, prompt text and support files. A coding agent reads the folder before it starts a task. Anthropic, OpenAI and others back a shared format at agentskills.io. The format has spread fast: open libraries of Claude Code and Codex skills now hold thousands of them. The question SkillEvaluator asks is whether a skill actually improves the agent's work.

The three tiers

Tier 1 is called validation. It runs static safety and format checks. It needs no model call, no API key and no sandbox.

Tier 2 is deduplication. It uses embeddings to find skills that overlap with other skills in the same library. The point is to stop a team from releasing two skills that do the same job.

Tier 3 is live evaluation. The tool starts a Docker sandbox. It runs the agent on a task with the skill, and runs it again without the skill. Then it compares the two runs. The README states the output as "86.2/100 (B)", with a default pass mark of 70.

Who it works with

The README says the user adds SkillEvaluator "to Claude Code, Codex or any skill-aware agent". The install is a pip package and one skill folder. Tier 3 needs an NVIDIA Build API key for its grader and a working Docker setup. A team can also write their own test cases and plug them in.

Measured gains on NVIDIA's own 300-skill set, from the August 2026 NVIDIA developer blog by Moshe Abramovitch and others: average skill lift of plus 31 points. Claude Code gained plus 34 points and Codex plus 29 points. Those numbers are NVIDIA's own and sit on a set NVIDIA chose. A team building its own skills still has to run its own evaluation.

A quality score is only as honest as the test cases behind it. Writing good test cases for an open-ended task is hard work. The Tier 3 sandbox also costs model calls, because the grader is an API call. The value a team gets from SkillEvaluator depends on how carefully they build the cases. The score it prints is downstream of that work.

Source

NVIDIA/SkillEvaluator on GitHub. Background: NVIDIA developer blog on SkillEvaluator.

SourceGitHub

This item was written by an AI system from the linked source. Reveneau is responsible for what it publishes.

Share
LinkedInX
Start a project