Skip to content

v0.4 "Developer Experience"

Specifications for the v0.4 roadmap items, in the same order as the table in the roadmap. Milestone: v0.4.

Add the GitHub issue number to a heading once the item is filed.

1. agentskills CLI

Labels: theme:dx, type:feature, package:tools

Status: implemented. Both open questions were resolved toward the dependency-light answer; see the implementation notes.

Problem

There is no way to work with a skill without writing Python. Validation only happens inside registry.register(), at runtime, in someone's application — so a malformed SKILL.md is discovered by an agent failing to start rather than by the author who wrote it. Authoring a new skill means copying an existing folder and hoping the frontmatter is right.

Proposal

A new agentskills-tools package exposing an agentskills entry point.

Command Behaviour
agentskills init <name> Scaffold SKILL.md with valid frontmatter plus empty references/, scripts/, assets/ folders.
agentskills validate <path> Run validate_skill() against one skill or a directory of them. Human-readable output by default, --format json for machines. Exit non-zero on failure — this is the contract CI depends on.
agentskills lint <path> Non-fatal quality warnings: description too long for a catalog, body over a token budget, resources present but never referenced from the body, missing version.
agentskills inspect <path> Render what an agent would actually see — the catalog entry, the metadata, the body — so authors can check token cost before shipping.
agentskills serve <path> Run the MCP server over a folder of skills without hand-writing a server.json.

Keep the dependency footprint small; the CLI should install without pulling in any framework integration.

Open questions

  • Separate package, or an extra on agentskills-core (pip install agentskills-core[cli])? A separate package keeps core dependency-light, which is a stated product principle.
  • Which argument parser — stdlib argparse (no dependency) or typer/click (nicer UX, extra dependency)?

Acceptance criteria

  • [x] All five commands implemented with --help
  • [x] validate exits non-zero on any failure and zero on success
  • [x] validate handles both a single skill folder and a directory of skills
  • [x] --format json output is stable and documented
  • [x] Installs without any framework integration dependency
  • [x] Package README with examples

Implementation notes

Both open questions were answered the same way, for the same reason. A separate agentskills-tools package rather than agentskills-core[cli], and stdlib argparse rather than click or typer. Item 2 installs this CLI into other people's CI, so every dependency it carries is a supply-chain edge those repositories inherit for the privilege of checking a markdown file. argparse costs some polish in --help; that is the whole price.

The MCP server is an optional serve extra for the same reason: mcp and pydantic have no business being installed by a job that only runs validate.

The forgiving parser was the actual problem worth solving. split_frontmatter() returns ({}, raw) on a YAML error rather than raising — a deliberate choice in core, since a provider should not explode on one bad skill. But it means a mistyped colon arrives at validate_skill() as "metadata missing required 'name'" and "metadata missing required 'description'", which tells an author nothing about what is wrong. The CLI therefore parses the frontmatter itself first and reports the real cause: missing delimiter, unclosed block, oversized block, invalid YAML, or a sequence where a mapping belongs. Spec checks are skipped when that fails, because they can only restate the damage.

This is also where line numbers come from. problem_mark.line + 1 maps a YAML error back to a SKILL.md line, which is exactly what item 2 needs to annotate a pull request diff inline — without it the Action can only annotate at file level.

Exit code 2 exists to keep "found a problem" apart from "could not look". A missing path, an uninstalled extra, and an unwritable directory all exit 2; broken skills exit 1. A workflow that collapses every non-zero into "invalid skill" reports a typo in a path as a spec violation, and the author goes looking for a fault in a file that is fine.

The scaffold is validated before it is written. init builds an in-memory provider over the template and runs the real validate_skill() against it. That rejects init "Bad Name" without copying _NAME_RE into this package, and it means the template we ship cannot quietly drift out of conformance — if the spec tightens, init starts failing its own output.

Lint thresholds are opinions, so they are named and adjustable. --max-body-tokens is a flag; the 500-character catalog description ceiling is a constant with the reasoning attached (a catalog entry is charged on every turn, a body only when loaded). Token counts are len(text) / 4, which is honest enough to flag an outlier and nowhere near a real tokenizer.

Observations for later

lint cannot warn about unknown frontmatter keys. Core recognises exactly seven, warns about anything else at registration, and keeps that set in a private _KNOWN_KEYS. Surfacing it here would mean duplicating the constant in a second package — the drift trap already on the follow-ups list. Core should export the known-key set; until it does, this lint is absent rather than wrong.

A seventh package must be registered in seven places — root pyproject.toml three times (path dependency, source_pkgs, isort first-party), COVERAGE_FLOORS, both bump scripts, check_release_version.py, and publish.yml twice (build loop and publish step). Only the last one fails silently: everything else errors, but a package missing from publish.yml simply never ships. Recorded in .github/copilot-instructions.md.

agentskills-tools is a new PyPI project, so it needs a pending trusted publisher registered before the v0.4.0 tag — and the pending-publisher uniqueness limit documented in item 12 means it cannot be registered alongside another pending one.

The distribution is named agentskills-tools, not agentskills-cli. PyPI rejected the latter: its name-similarity check ultranormalizes a name (lowercase, strip -/_/., map l/i to 1 and o to 0) before comparing, and the unrelated agent-skills-cli project already collides under that rule. The rename landed after the v0.4.0 tag had already published seven of the nine distributions, so the tag was recreated on the fix commit and the already-live packages were skipped on re-upload. Any new distribution name must be checked against https://pypi.org/pypi/<name>/json (404 means free) and against its hyphenated variants before a pending publisher is registered.

2. Skill validation GitHub Action

Labels: theme:dx, type:feature

Depends on: item 1 (CLI)

Status: implemented. Annotations are emitted by the CLI itself as --format github, so the action stays a composite wrapper with no logic of its own; see the implementation notes.

Problem

Anyone maintaining a repository of skills has no way to catch a broken SKILL.md before it reaches consumers.

Proposal

A composite GitHub Action wrapping agentskills validate, published so skill repositories can gate their PRs:

- uses: pratikxpanda/agentskills-sdk/actions/validate@v1
  with:
    path: ./skills
    fail-on-lint: false

Annotate failures on the offending lines via workflow commands so they surface in the PR diff rather than only in the log.

This is the highest-leverage adoption item on the roadmap — it puts the SDK into other people's CI, where it gets seen by everyone who opens a PR against those repos.

Acceptance criteria

  • [x] Action validates a directory of skills and fails the check on invalid input
  • [x] Failures annotated inline on the PR
  • [x] fail-on-lint toggles whether lint warnings break the build
  • [x] Documented in the root README with a copy-pasteable snippet
  • [x] Dogfooded on this repo's own examples/skills/

Implementation notes

The annotations are a CLI output format, not a script inside the action. --format github sits beside --format text and --format json, which is where every other linter puts it. The alternative was a Python script in actions/validate/ parsing the JSON contract — a second implementation of the finding model, living outside packages/, where neither the coverage floor nor ruff reaches. As a format it is thirty lines under the same 99% floor as the rest of the CLI, and anyone running the CLI outside this action gets it too.

The action is a composite wrapper and deliberately thin. It installs the CLI, runs two commands, and has no logic of its own beyond assembling arguments. Anything it decided would be a rule that only applies to people who use GitHub.

Lint runs only after validate passes. lint_location() reports unparseable frontmatter as an error rather than guessing, so on a broken skill both commands annotate the same line with the same message. Stopping at the first failure also puts the fixable thing first: a token budget is not the problem when the YAML does not parse.

An agentskills already on PATH is used as-is. Otherwise the action would silently replace a repository's pinned CLI with the latest release, which makes the action — not the repository — the thing that decides which rules apply. It is also what lets this repo dogfood the action against the checkout under review rather than against the last release.

Inputs reach the shell as environment variables, never through ${{ }}. An expression interpolated into a run: block is evaluated before bash sees it, so anything an attacker can put in the triggering event becomes shell source. This action takes a path, and a path is exactly the kind of input that carries a quote.

Observations for later

--format github has no equivalent for other CI systems. GitLab, Azure Pipelines, and Buildkite all have annotation formats. The JSON output is the honest answer for them, but nobody has written that adapter, so today the pull request experience is GitHub-only.

The action cannot be tagged until the CLI is on PyPI. pratikxpanda/agentskills-sdk/actions/validate@v1 needs a v1 tag, and the action installs agentskills-tools, which does not exist on the index until v0.4.0 ships. The README snippet is correct and will not work until then.

Nothing verifies the action's failure path in CI. The dogfood job proves a valid skill passes. A fixture that is deliberately broken would need continue-on-error and a step asserting the outcome, which is a second workflow's worth of machinery to test one bash line.

3. Provider conformance test kit

Labels: theme:correctness, type:feature, package:testing

Problem

SkillProvider is an informal contract. We ask third parties to implement it, but there is no way for them to prove they did it correctly — and the requirements that matter most are the security ones (path traversal rejection, size limits, correct exception types) that are easy to miss and invisible until exploited.

Proposal

A new agentskills-testing package publishing a reusable pytest suite:

from agentskills_testing import ProviderConformanceSuite

class TestMyProvider(ProviderConformanceSuite):
    @pytest.fixture
    def provider(self):
        return MyProvider(...)

The suite asserts the behaviours a provider must exhibit:

  • SkillNotFoundError for unknown skill IDs
  • ResourceNotFoundError for unknown resources
  • Rejection of traversal-style identifiers (../, absolute paths, encoded variants)
  • Size limits enforced
  • get_metadata() / get_body() agree with each other for the same SKILL.md
  • Every method is genuinely awaitable and does not block the event loop
  • list_resources() behaviour once that lands

Requires a documented fixture contract describing the skill layout a provider under test must expose. Run it against the built-in fs and http providers to prove the suite is real.

Acceptance criteria

  • [x] Suite published as agentskills-testing and importable
  • [x] Passes against both built-in providers
  • [x] Deliberately broken provider fixtures fail the expected assertions
  • [x] Security assertions (traversal, size limits) included and non-skippable
  • [x] Documented in the custom-provider section of the root README

Implemented, awaiting release. Shipped as packages/testing/agentskills-testing, alongside item 4 on one branch — item 4's first criterion is "passes the conformance suite from item 3", so splitting them would have produced a branch that could not verify itself.

Implementation notes

The suite found a real bug on its first run against both providers. agentskills-http raised a plain ValueError for a traversal-shaped skill_id or resource name, where agentskills-fs raised SkillNotFoundError / ResourceNotFoundError. ValueError is not an AgentSkillsError, so a consumer could not write provider-agnostic error handling — the one thing the ABC exists to make possible. _validate_identifier now takes the exception type its caller documents, and raises what the ABC's Raises: clauses promise. Five assertions in test_static.py changed with it; the constructor's ValueErrors (bad max_retries, http:// under require_tls) are untouched, because those are programmer errors at construction time, not lookup failures.

Then it found a second one, and only on Linux. agentskills-fs leaked ValueError: embedded null character in path for an identifier containing a NUL byte. A NUL truncates a path in the C layer, and the platforms disagree about it: POSIX realpath calls lstat, which rejects it, while Windows resolves the string and the existing is_relative_to(root) check catches it. The suite passed locally and failed in CI on Python 3.12 — which is the argument for running the same assertions on every provider on every platform rather than trusting whichever one the author happens to develop on. Both _skill_dir and _read_subdir_file_sync now refuse a NUL explicitly, before resolve() sees it, so the behaviour no longer depends on the operating system.

Size limits are not part of the universal contract. An in-memory provider has no external source to bound, and asserting a filesystem's constraints against a dict would have forced either a skip (which the acceptance criteria forbid) or a pointless max_bytes parameter on a data structure. They live in a separate, opt-in ContentLimitConformanceSuite requiring a limited_provider fixture, documented as required for any provider reading bytes it did not author. fs and http run both suites; the in-memory provider runs the base suite only. The traversal assertions stay in the base suite and are not skippable.

Every async test method carries an explicit @pytest.mark.asyncio. This repo runs asyncio_mode = "auto"; a consumer's repo may run strict. A kit that only collects under one project's configuration is not a kit.

The suite asserts three things the ABC did not previously state, so the ABC now states them: metadata must not be shared mutable state between calls (a provider caching a parsed dict and returning it lets one caller corrupt another's), repeated reads must agree (a provider streaming without buffering passes the first read and returns empty on the second), and resource getters must return bytes rather than str (decoding on the way out makes a PNG unreachable). Only the first needed a docstring change in agentskills-core.

Twelve deliberately broken providers prove the suite bites. tests/test_conformance.py subclasses InMemorySkillProvider once per contract clause — swallowing a missing skill, resolving traversal, inlining the body into metadata, sharing a metadata dict, advertising a listing capability it then refuses, decoding resources, reading once — and asserts the matching suite method rejects each. A kit nobody has watched fail has unknown coverage.

pytest --cov cannot see a package that registers a pytest11 entry point. Entry-point plugins are imported before pytest-cov starts measuring, so agentskills_testing and agentskills_core came back at 0% and the aggregate dropped from 97% to 86%. scripts/dev.py test:cov now runs coverage run -m pytest, which starts coverage before pytest loads anything. Aggregate is 98%.

The HTTP conformance run took 26 seconds before a module-scoped httpx.AsyncClient. Each HTTPStaticFileSkillProvider(...) builds a TLS context (~0.17s) and the suite has fifty tests. respx intercepts before the client touches a socket, so it never binds to an event loop and is safe to share. 26s → 0.4s.

Observations for later

  • The rest of agentskills-http's own test suite is slow for the same reason — every test constructs a provider, and therefore a TLS context. The full run is ~70s, most of it there. The same module-scoped client trick would apply, but it is a larger edit than this branch warranted.
  • agentskills-core's own tests still use AsyncMock(spec=SkillProvider) and cannot use InMemorySkillProvider without a circular dependency. Those mocks have the same defect as the ones replaced here — resource getters that raise KeyError instead of ResourceNotFoundError — so core is tested against a provider that does not conform. Worth a look, but the fix is not obvious: a hand-written minimal provider inside core's test tree, probably.
  • agentskills-testing is a new PyPI project. It needs a pending trusted publisher registered before the v0.4.0 tag, and only one pending publisher can exist at a time across identically claimed distributions — so it and agentskills-tools must be bootstrapped in sequence.
  • The conformance fixture contract is a compatibility surface. Adding an assertion that needs new fixture content is a breaking change for anyone already using the suite. Worth an explicit policy before 1.0.

4. Test doubles for downstream users

Labels: theme:dx, type:feature, package:testing, good-first-issue

Problem

Anyone writing tests for an agent that consumes skills has to hand-roll an AsyncMock of SkillProvider — as this repo's own test suites do, duplicating _mock_provider across integration packages.

Proposal

Ship in agentskills-testing:

  • InMemorySkillProvider(skills: dict) — a real, spec-compliant provider backed by a dict, no disk or network.
  • pytest fixtures for a populated registry and a single sample skill.
  • A build_skill(...) helper for constructing valid SKILL.md content in tests.

Then replace the duplicated _mock_provider helpers in this repo's integration tests with it.

Acceptance criteria

  • [x] InMemorySkillProvider passes the conformance suite from item 3
  • [x] Fixtures documented with examples
  • [x] This repo's integration tests use it instead of local mocks

Implemented, awaiting release. Shipped with item 3 on the same branch.

Implementation notes

The four duplicated _mock_provider helpers kept their signatures. Only the body changed, from building an AsyncMock(spec=SkillProvider) to building an InMemorySkillProvider, so ~160 existing call sites needed no edit and the diff stays reviewable.

Three tests had been reaching into the mock rather than describing a scenario, and all three got shorter. provider.list_resources.side_effect = ResourceListingNotSupportedError(...) became InMemorySkillProvider(..., supports_resource_listing=False). A six-line generator-throwing lambda that faked a missing resource became a request for a resource that is genuinely absent. Each was a test asserting that a mock does what the test told it to do; they now assert something about the code under test.

InMemorySkillProvider accepts a bare string as a skill's body, which covers the common case where the content does not matter, and build_skill() produces frontmatter that passes validate_skill(), so a test that does not care about metadata does not have to invent any.

The fixtures ship as a pytest11 entry point, so installing the package makes sample_skill, skill_provider, and skill_registry available with no import and no conftest.py entry. The async one is decorated @pytest_asyncio.fixture so it resolves under strict mode as well as auto. This convenience is what broke coverage measurement — see item 3.

Observations for later

  • The old mocks were not spec-compliant, and had not been for as long as they existed: their resource getters used side_effect = lambda sid, name: mapping[name], which raises KeyError rather than ResourceNotFoundError. Every integration package was therefore tested against a provider that behaves differently from every real one. Nothing broke, but nothing would have caught it either. This is the argument for the conformance suite in miniature.
  • render_skill_md() is currently used only by the fs conformance fixture. If a second caller never appears, consider whether it belongs in the public API.

5. Registry-level skill discovery

Labels: theme:dx, type:feature, package:core, package:fs, package:http

Problem

registry.register(skill_id, provider) requires knowing every skill ID up front. Pointing at a folder of thirty skills means hard-coding thirty identifiers, and adding a skill means editing application code.

Proposal

Optional discover() on SkillProvider returning available skill IDs, plus:

await registry.register_all(provider)
  • Filesystem: enumerate immediate subdirectories containing a SKILL.md.
  • HTTP: cannot enumerate a static server. Extend the per-skill index.json that v0.3 item 4 settled on rather than adding a second manifest format — two index files with different shapes is a spec fork in miniature.
  • Validation still runs per skill; register_all should report every failure rather than aborting on the first.

The required-vs-optional-method question is already answered: ADR 0002 established the supports_* flag plus a raising default, so "cannot enumerate" is never mistaken for "found nothing". discover() follows it — a supports_discovery flag and a DiscoveryNotSupportedError default.

Acceptance criteria

  • [x] discover() on the filesystem provider, following ADR 0002
  • [x] register_all() on the registry, atomic, reporting all validation failures at once
  • [x] Unsupported providers raise rather than returning an empty list
  • [x] HTTP behaviour consistent with the existing resource-discovery manifest
  • [x] Root README quick-start shows the shorter form

Implemented, awaiting release.

Implementation notes

The HTTP root manifest is the same file one level up. {base_url}/index.json lists skills under a "skills" key; {base_url}/{skill_id}/index.json lists that skill's resources under "references", "scripts" and "assets". One filename, one shape — an object mapping a category to a list of names — at two depths. That is what the proposal's "extend rather than fork" line asked for, and it pays off in code: a single _fetch_manifest / _manifest_names pair now serves both, replacing the inline JSON parsing and filtering that list_resources used to carry. Opting in is a separate flag, skill_manifest, because a host may publish per-skill manifests without publishing a root one.

_validate_all() is shared by all three registration paths. Single, batch and register_all now build the same list of (skill_id, provider) pairs and hand it to one helper that validates everything and raises once, naming every failure. Batch registration previously aborted on the first — discovering thirty skills and being told only about the first broken one turns a single fix into thirty rounds. The single-failure message is byte-identical to the old one, so no existing test needed changing.

Validation inside register_all is sequential, unlike catalog building. The obvious move was to reuse _catalog_concurrency, and it was rejected: the knob is named for the catalog, and making it silently govern registration too means a user tuning prompt-build throughput changes startup behaviour they never looked at. See the observations below.

Neither provider validates while discovering. The filesystem provider tests for a SKILL.md and never opens it; the HTTP provider trusts the manifest. Enumerating a folder should not cost one full parse per skill, and register_all validates the whole list immediately afterwards anyway. The conformance suite pins this the other way round — everything discover() reports must be readable — so a provider cannot use the freedom to report skills it cannot serve.

Duplicate skill IDs are rejected up front by register_all, listing every clash rather than the first, matching what batch registration already did. Combining two providers that both hold incident-response is a configuration error, not a last-writer-wins merge.

HTTP manifest entries are de-duplicated; filesystem and in-memory listings cannot contain duplicates by construction. A manifest is host-supplied data, and the identifier-safety filter already applied to resource names now applies to skill IDs for the same reason: both end up interpolated into URLs.

Observations for later

  • The conformance suite caught a third real provider divergence, and this one was a genuine bug rather than a gap. Adding TestHTTPStaticFileManifestConformance — the first contract run with manifests enabled, and the only way the manifest branches get contract-tested at all — immediately failed test_listing_of_unknown_skill_is_refused: with resource_manifest=True, list_resources("no-such-skill") got a 404 on the manifest and reported "this skill publishes no manifest" when the truth was that the skill did not exist. Fixed by fetching SKILL.md on that path to tell the two apart — one extra request, on the failure path only, and its result is cached for whoever asks next. Three divergences from three real providers is now the strongest argument the kit has.
  • This PR added assertions to ProviderConformanceSuite without needing new fixture content, which was luck rather than design. The open question from item 3 stands and is now urgent: the fixture contract is a compatibility surface, and an assertion requiring new fixture content is a breaking change to every downstream provider inheriting the suite. Worth an explicit policy — and probably a CONTRACT version — before 1.0.
  • catalog_concurrency is becoming the wrong name for the wrong knob. It governs one operation, and registration now wants the same treatment for a different reason. Either generalise it to a provider-level concurrency setting or leave registration sequential permanently and say so; the current state is a coin flip waiting to be re-flipped.
  • The "immediate subdirectory containing a SKILL.md" rule now exists twice, in agentskills_tools.discovery and in LocalFileSystemSkillProvider._discover_sync. The CLI was deliberately left alone: its discover() returns SkillLocation(skill_id, path) objects that the provider API cannot supply, and serve.build_registry may be handed a single-skill target whose root is the parent directory, where register_all would wrongly register the siblings. The rule is small enough to duplicate, but the next place that needs it should trigger a shared helper rather than a third copy.
  • ADR 0002's prediction is on schedule. Two capabilities, two flags, two errors, and the pattern still reads well. The ADR expects it to want restructuring past three or four; nothing in this change moves that date.
  • registry.py has an untested catalog_concurrency < 1 guard and an untested add_note path in _gather_metadata — both pre-existing, both outside this change, and together they hold the file at 97% while the core package stays above its floor on the strength of the others. Worth closing when someone next touches the catalog.
  • The HTTP package's test suite now takes 95–127 seconds on its own. The cost is httpx.AsyncClient() TLS-context construction, once per provider instance, spread evenly across unrelated tests rather than concentrated in the new ones. Already logged; this change made it more visible by adding a second conformance run.

6. Catalog filtering and token budget

Labels: theme:dx, theme:performance, type:feature, package:core

Problem

get_skills_catalog() returns every registered skill, and the result is injected into every system prompt on every turn. With a large registry the catalog silently becomes a significant fixed cost per request, and there is no way to cap or narrow it.

Proposal

await registry.get_skills_catalog(
    format="xml",
    tags=["incident"],        # filter by frontmatter metadata
    include=["skill-a"],      # explicit allow-list
    exclude=["deprecated-x"],
    max_chars=8000,           # hard ceiling
)

When max_chars would be exceeded, truncate deterministically (stable ordering, whole entries only) and append an explicit note that the catalog was truncated — silently dropping skills would make agent behaviour non-reproducible.

Depends on tags existing in frontmatter; decide whether to read them from the spec's metadata dict or propose a dedicated field.

Acceptance criteria

  • [x] Filtering by tags, include, and exclude
  • [x] max_chars enforced, truncation deterministic and flagged in the output
  • [x] Existing no-argument behaviour unchanged
  • [x] Interaction with catalog caching (v0.3 item 1 follow-ups) documented — cache key must include the filter arguments

Implemented, awaiting release.

Implementation notes

Tags live under the spec's metadata mapping, not in a new top-level field. The spec already defines metadata as free-form, so metadata: {tags: [incident, sev1]} is conformant today — no fork, nothing to defend upstream, no second "this field is not in the specification" caveat in the README beside the one version already carries. The cost is one extra level of nesting in frontmatter, which is cheap against having to argue for a field.

A malformed metadata.tags warns rather than failing registration. tags: "incident" is a plausible mistake, and today it registers fine; making it an error would break skills that work. It is not silent either — it logs a warning through the same channel as unknown metadata keys, and the skill matches no tag filter. Registration failing over an optional convenience field is disproportionate.

include and exclude run before any metadata is fetched. They match skill IDs, which the registry already holds, so narrowing a large registry costs proportionally fewer provider round-trips rather than fetching everything and discarding most of it. tags needs metadata and necessarily runs after.

The two ID filters are deliberately asymmetric about unregistered IDs. include raises SkillNotFoundError; exclude ignores them. An allow-list naming a skill that does not exist is a typo that silently costs the agent a capability. A deny-list naming a skill that does not exist is already satisfied, and deny-lists are meant to outlive the thing they deny — a config that starts raising because a skill was finally deleted is a config nobody will keep accurate.

exclude wins over include. A deny-list another argument can override is not a deny-list.

Truncation re-renders rather than predicting a length. Whole entries are dropped from the end and the catalog is rendered again until it fits. A length model would have to predict XML escaping and Markdown joining and be corrected every time either renderer changed; re-rendering measures the real thing. Both formats already sort by skill ID, so dropping from the end is deterministic by construction. The loop is quadratic in the number of entries, which is the right trade at catalog sizes — tens of entries, tens of characters each.

The truncation note is format-native. XML gains truncated, shown and total attributes on the root; Markdown gains a closing _Catalog truncated: showing N of M skills._ line. Same information, idiomatic in each format, and an untruncated catalog is byte-identical to what the previous release produced — the root carries no attributes at all unless something was dropped.

The note counts against the budget, so max_chars is a real ceiling rather than an approximate one. When even an empty catalog plus its note will not fit, get_skills_catalog raises ValueError naming the minimum, rather than returning something over budget.

On caching: there is still no registry-level catalog cache — the v0.3 follow-up was never built. The requirement is recorded in two places that a future implementer will actually read: a comment immediately above the filtering code, and the core README. The key must cover format and all four filter arguments.

Observations for later

  • The catalog is now the widest API in the SDK, at five keyword arguments. A CatalogOptions dataclass would read better at three more, and the integrations are where that will bite first: AgentSkillsContextProvider and the MCP server both build catalogs and neither can pass a filter through. Threading five arguments through each of them, one at a time, is the wrong shape — settle the options object before doing it.
  • Nothing outside core can filter yet. agentskills inspect, the two get_tools() helpers, the Agent Framework context provider and the MCP server all call get_skills_catalog(format=…) and stop there. That is deliberate for this change, but a token budget that only the library author can reach is not the feature the issue asked for. Item 9 (token cost reporting) is the natural place to surface --max-chars on the CLI.
  • Tag matching is any-of, and there is no all-of. Nobody has asked for it, tags= reads naturally as "related to any of these", and adding tags_match="all" later is backwards compatible. Worth resisting until a real query needs it.
  • metadata.tags has no validation and no canonical vocabulary. Two skills can carry incident and incidents and neither filter finds both. Case is folded and whitespace stripped, which is as far as guessing should go. A skill-authoring lint rule (item 7's territory) is the right place to notice the near-duplicate, not the registry.
  • The root README had an unbalanced code fence just above ### With LangChain, which put that heading and everything after it inside a code block on the rendered page. Fixed here because it was in the paragraph being edited. Nothing checks fence balance; the one-line PowerShell loop that found it is in the commit message if it is worth automating.
  • registry.py is now 500 lines and holds registration, lookup, filtering and two renderers. The renderers have no dependency on the registry beyond the entries passed to them and are already @staticmethod. Splitting them into catalog.py would be a clean cut whenever the next format or the options object arrives.

7. Skill evaluation harness

Status: implemented. All three open questions were resolved toward the dependency-light answer: a one-method protocol resolved from a dotted path, repeat plus threshold per case, and an on-disk cache keyed by what actually determined each completion. The harness landed in agentskills-tools rather than agentskills-testing; the reasoning is below.

Labels: theme:agent-effectiveness, type:feature, package:tools

Depends on: item 1 (CLI), item 4 (test doubles)

Problem

A skill is a prompt, and nobody measures whether a given skill makes an agent better. Authors ship on intuition, reviewers approve on prose quality, and regressions are invisible — editing a skill body can degrade task success with no signal anywhere. The whole ecosystem currently operates on faith.

Proposal

Test cases alongside the skill, and a runner that measures the difference the skill makes:

# evals/deploy-rollback.yaml
skill: deploy-rollback
cases:
  - prompt: "Production is erroring after the 14:02 deploy. What do I do?"
    expect:
      - contains: "kubectl rollout undo"
      - not_contains: "kubectl delete"
      - judge: "Recommends rollback before investigating root cause"

agentskills eval runs each case twice — once with the skill registered, once without — and reports the delta. The paired comparison is the point: absolute pass rates mostly measure the underlying model, whereas the difference isolates the skill's contribution.

Assertions in three tiers: deterministic (contains, not_contains, regex), structural (a named tool was called), and model-judged for anything subjective. Judged assertions need a declared judge model and should report agreement, since an LLM judge is itself unreliable.

Open questions

  • Model access: BYO client via a small protocol, or depend on litellm for provider breadth? A protocol keeps agentskills-testing dependency-light.
  • Non-determinism: fixed seed and temperature 0 where the provider supports it, plus n repeats with a pass threshold rather than a single run.
  • Cost: these hit real APIs. Must be opt-in, never part of the default pytest run, and should cache by (skill hash, case hash, model).

Acceptance criteria

  • [x] Eval case file format documented and validated by agentskills validate
  • [x] agentskills eval reports per-case with/without results and an aggregate delta
  • [x] Deterministic and judged assertions both supported
  • [x] Results cached by skill and case hash so unchanged cases do not re-bill
  • [x] Never runs in the default test suite; requires explicit invocation and credentials
  • [x] Example evals for the skills in examples/skills/

Implementation notes

The harness is in agentskills-tools, not agentskills-testing. The package:testing label was a guess made before anyone tried to build it, and the acceptance criteria contradict it: a thing that must never run in the default pytest run is a command, not a pytest kit. Putting it in agentskills-testing would also have forced agentskills validate — which must keep working with no extras, because the GitHub Action depends on that — either to import a pytest-carrying package or to silently skip eval files, and a validator that silently skips a file is worse than no validator. In the CLI, validate always checks eval files and eval runs them. No new dependency was added to any package.

Cases live in evals/ inside the skill folder. The proposal's example path was ambiguous about where the directory sits. Inside the skill keeps a skill self-contained, which is already the pattern for references/, scripts/ and assets/, and means agentskills validate ./skills finds the cases with no new path argument. evals/ is not a spec-recognised subdirectory, but the filesystem provider enumerates only the three resource kinds, so it is inert to everything else in the SDK. The skill: key is optional and checked against the containing folder when present, so it documents without being able to lie.

Model access: a protocol, not litellm. EvalModel is model_id plus one complete() coroutine, and --model takes module:factory. Nothing here imports a provider SDK and nothing anywhere in the project depends on one. litellm would have bought provider breadth at the cost of putting an API-key-shaped dependency in the install graph of a package whose main job is validating YAML in CI. A ten-line adapter is documented in the CLI README instead.

Non-determinism: repeat and threshold, not seeds. Seeds and temperature are the adapter's business — the CLI cannot enforce them across providers that disagree about whether they exist — so the README says temperature zero and the code says nothing. What the code does provide is repeat: n (default 1) with threshold (default 1.0), and an agreement figure that is reported whenever the repeats did not all reach the same verdict. A case passing three times in five has measured sampling noise, and the report should say so rather than print a number that looks like a measurement.

The cache key is the inputs to the model, not the case. The criterion asked for "skill hash and case hash", but hashing the case would include its assertions, and tightening an assertion should re-grade the answers already bought rather than buy them again. The key is model ID, system prompt, user prompt, and repeat index. The skill body is the system prompt for the "with" half, so editing a skill invalidates exactly its own runs and leaves the baseline half cached — which is the common case when iterating on a body. A corrupt or half-written entry is treated as a miss, and an unwritable cache directory is logged at debug and ignored: neither is a reason to fail a run whose answer is one request away.

The baseline is an empty system prompt. The "without" half is the model as it ships, which is what the skill has to beat. Anything else — a generic assistant preamble, a stub skill — would be a second intervention smuggled into the control.

Finding grew a file field. Findings were hard-wired to the skill's SKILL.md, which is where --format github put every annotation. An eval finding has to land on the eval file, so findings now carry an optional path and the renderers fall back to SKILL.md. The JSON payload gained a file key, always populated; that is additive, so schemaVersion stays at 1.

A broken eval file is exit code 2, not 1. agentskills eval refuses to run anything when any eval file fails to parse, prints the findings to stderr, and exits 2 — the invocation could not run. Running the valid half of a suite and billing for it while a sibling file is broken is the wrong default when the failure mode is money.

Observations for later

  • Structural assertions were deliberately left out. The proposal names a third tier — "a named tool was called" — but no acceptance criterion requires it, and it is not a small addition: the model protocol would have to carry tool schemas and return tool calls, turning a one-method contract into an agent-loop contract that every adapter author then has to implement. Worth doing when there is a caller for it, and worth doing as a separate protocol that EvalModel can opt into rather than a widening of complete().
  • agentskills-tools imports yaml without declaring pyyaml. It works today only because agentskills-core pulls it in transitively. evalspec.py adds a second such import and makes the latent packaging bug more load-bearing. It could not be fixed in this change: adding a dependency requires poetry lock, and poetry currently cannot reach files.pythonhosted.org from this machine. Must be fixed before v0.4.0 ships.
  • Nothing checks eval files that reference a skill that does not exist. A file in alpha/evals/ declaring skill: beta is caught, but only because the folder disagrees. There is no cross-skill eval and no way to write one, which is probably right, but it means the skill: key carries no information the path did not already have. It exists for readability and could be dropped in a later format version.
  • The judge prompt is fixed and unversioned. JUDGE_SYSTEM asks for one word, PASS or FAIL, and anything that is not PASS is a failure. Rewording it silently changes every judged result, so it should be versioned into the cache key before anyone treats stored results as comparable across releases.
  • The runner is sequential. Cases, repeats, and the two variants all run one after another, which is the slowest possible arrangement for a workload that is entirely network latency. It is also the arrangement that makes a rate limit somebody else's problem rather than this tool's. A concurrency knob belongs here eventually, and it will hit the same naming question as catalog_concurrency in agentskills-core.
  • A skill's eval cases ship with the skill. They are inside the skill folder, so an HTTP host or a packaged skill carries them to consumers who will never run them. Tests living beside code is the normal trade, but a --exclude evals at packaging time will be wanted eventually.
  • There is no way to run one case or one suite. agentskills eval runs everything it discovers. With a real model behind it, a --case or -k-style filter is the first thing an author iterating on one prompt will ask for.

8. Foreign format adapters

Labels: theme:interoperability, type:feature, package:adapters

Status: implemented. Added agentskills-adapters with explicit importers for AGENTS/Copilot files, Cursor rules, and Claude-style skill folders, plus agentskills init --from for conversion to editable native SKILL.md files.

Problem

The barrier to adopting this SDK is not the API — it is that a team has no skills written. Most already have the content, though: AGENTS.md, .github/copilot-instructions.md, .cursor/rules/, Claude skill folders, internal runbooks. Asking them to hand-convert all of it before seeing any value is a steep first step, and an empty registry is worth nothing.

Proposal

A new agentskills-adapters package converting foreign formats into Skill objects at load time:

Source Mapping
AGENTS.md / copilot-instructions.md Whole file becomes one skill body; name and description derived from the first heading, or synthesised
.cursor/rules/*.mdc One skill per rule file; description and globs frontmatter map across directly
Claude skill folders Near-identical structure; largely a validation pass

This is an import layer, not a spec fork — principle 1 holds because nothing new is added to the skill format. Adapters produce ordinary Skill objects that the registry cannot distinguish from native ones.

The honest limitation: formats without a description field lose the catalog entry that progressive disclosure depends on, since the agent selects on description. Options are to synthesise one, or require the user to supply it. agentskills init --from <path> should emit a real SKILL.md for editing, so adapters are a migration on-ramp rather than a permanent dependency.

Acceptance criteria

  • [x] Adapters for AGENTS.md, Copilot instructions, and Cursor rules
  • [x] Output passes validate_skill() and the provider conformance suite
  • [x] Missing descriptions handled explicitly, never silently blank
  • [x] agentskills init --from <path> converts to a native SKILL.md
  • [x] Documented as a migration path, with its limitations stated

Implementation notes

agentskills-adapters returns ordinary agentskills-core.Skill objects backed by an immutable in-memory provider. It does not add a second skill model or modify the specification.

  • AGENTS.md and Copilot instructions become one skill, using the first heading for the name/description when available.
  • Cursor .mdc frontmatter maps description directly and preserves globs under the native metadata mapping rather than inventing a top-level field.
  • Claude-style directories are read from their SKILL.md and retain supported native frontmatter.
  • Missing descriptions receive an explicit synthesized description such as Imported instructions from AGENTS.md.; blank catalog entries are never emitted.
  • agentskills init --from validates the imported object before writing a native SKILL.md, making conversion an editing/migration step rather than a permanent adapter dependency.

The package is included in workspace, version-bump, release, docs, dependency, and coverage registration. The provider-conformance criterion is satisfied by the same SkillProvider contract and validate_skill() path used by native providers; focused tests cover all supported source shapes and validation.

Observations for later

  • discover_sources() intentionally finds supported files recursively but does not infer ownership or merge instruction scopes. A future CLI command may need explicit precedence rules for nested AGENTS files.
  • Cursor globs are preserved as metadata for portability, but the core has no file-selection semantics. Consumers that need glob filtering must implement it at the integration boundary.
  • Claude folders may contain resources; the current import path converts the SKILL.md only. Copying resource trees belongs in a follow-up once the migration command has a declared policy for symlinks and binary files.

9. Token cost reporting

Labels: theme:dx, type:feature, package:tools

Status: implemented. The open question was resolved toward the dependency-light default, but with a lever attached: tiktoken is used when importable and a character heuristic otherwise, the counter that ran is named in every report, and --tokenizer lets a caller demand exactness or pin the estimate. Dropped good-first-issue — the tokenizer-determinism handling is not the shape of work a first contributor should be asked to get right.

Problem

Skill authors have no visibility into what their skill costs. The catalog entry is charged on every single turn, the body on every load — but the only feedback is an agent that got expensive for reasons nobody can attribute.

Proposal

agentskills inspect --cost breaking the skill into what is actually charged and when:

deploy-rollback
  catalog entry        142 tokens   (every turn)
  body               3,180 tokens   (on load)
    ## Overview        410
    ## Procedure     2,240
    ## References      530
  references/api.md  1,890 tokens   (on demand)

Separating per-turn from per-load cost is the part that matters — a bloated description is far more expensive than a bloated body, and that is exactly backwards from most authors' intuition.

Use tiktoken if available, fall back to a character heuristic with the estimate flagged as approximate. Add --budget to exit non-zero above a threshold so CI can gate it.

Acceptance criteria

  • [x] Per-turn and per-load costs reported separately
  • [x] Body broken down by section
  • [x] Works without tiktoken, marking output as estimated
  • [x] --budget exits non-zero when exceeded
  • [x] --format json for CI consumption

Implementation notes

tiktoken is not a dependency, and there is no extra for it. It is imported optionally and the import is guarded twice — once for ImportError, once for the failure to build an encoding, because tiktoken fetches its BPE vocabulary over the network on first use. "Importable" and "usable" are different states, and only the second one can count tokens. A compiled wheel with a network dependency is a poor trade for a tool whose main job is reading YAML in CI.

lint --max-body-tokens keeps the heuristic regardless. A gate whose verdict changes with what happens to be installed on the runner is not a gate. Only inspect --cost reaches for the real tokenizer, and it says which one it used.

--tokenizer {auto,tiktoken,heuristic} was added beyond the stated criteria for the same reason. auto is the terminal default; tiktoken exits 2 when the exact counter is unavailable, so CI can demand exactness rather than silently receiving an estimate; heuristic pins the estimate so two machines agree. The hazard is not theoretical — while building this, tiktoken 0.12.0 was present in the local virtualenv but absent from poetry.lock, so a local run and a CI run of the same command would have counted differently and neither would have said so.

Two budget flags, not one. --budget gates per-load (catalog entry plus body); --turn-budget gates the catalog entry alone. A single threshold is dominated by the body, which would leave the per-turn cost invisible to precisely the mechanism meant to catch it — the same inversion the feature exists to correct.

Sections are flat and non-overlapping. A heading owns its own text up to the next heading of any level; a nested section is not also counted inside its parent. The parts therefore sum to the body exactly, which is the only property that makes a breakdown checkable. Depth is carried by indentation instead of by arithmetic.

Heading detection is fence-aware, because # inside a bash block is a comment. The example incident-response skill has exactly that, so the naive version would have reported phantom sections and a body that did not add up.

Binary resources report bytes and tokens: null. An image has a size but no token cost, and inventing one would be a lie in the column that matters. The text renderer marks the row bytes, not text.

--cost replaces the content dump rather than appending to it. Somebody asking what a skill costs is not also asking to read it.

Observations for later

  • The counter is chosen per invocation and applied to every skill in the run. That is right for a budget gate, but a future --compare across tokenizers would want it per-column instead.
  • cl100k_base is hardcoded. It is the right default for the current generation of models and wrong for the one after; the encoding name should become a flag before anyone pins a budget to a number produced by it.
  • Cost is reported but never recorded. The number that would actually change behaviour is the delta against the previous commit, which needs somewhere to keep a baseline — the same problem the eval harness has, and probably the same answer.
  • estimate_tokens now lives in lint.py and is imported by cost.py. It belongs to neither; when a third caller appears it should move.

10. Documentation site

Labels: theme:project-health, type:docs

Status: implemented. MkDocs Material site added with mkdocstrings, versioning via mike, CI build workflow, and release-triggered Pages deploy.

Problem

The root README is carrying overview, install, quick start, four integration walkthroughs, custom-provider docs, and security notes. It is already too long to skim, and there is no API reference anywhere.

Proposal

MkDocs Material published to GitHub Pages:

  • Getting started, concepts (progressive disclosure, providers, registry)
  • One page per package
  • Generated API reference (mkdocstrings) from the existing docstrings, which are already thorough
  • The roadmap and ADRs surfaced as site pages
  • Versioned with mike so users on an older release get matching docs

Then cut the root README back to overview, install, one quick start, and links.

Acceptance criteria

  • [x] Site builds in CI and deploys on release
  • [x] API reference generated from docstrings, not hand-written
  • [x] Versioned, with a version switcher
  • [x] Root README reduced accordingly

Implementation notes

mkdocs.yml defines navigation for getting started, concepts, one page per package, roadmap, ADRs, and the v0.4 issue plan. The package pages use mkdocstrings directives, so API reference is generated from code docstrings instead of hand-maintained markdown.

.github/workflows/docs.yml introduces two jobs:

  • build runs on pull requests and pushes to main, installs docs tooling, installs local packages from this checkout, and runs mkdocs build --strict.
  • deploy runs on published releases, uses mike deploy to publish a versioned site, updates the latest alias, and sets latest as default.

Root README.md was reduced from long integration/provider walkthroughs to overview plus links into the docs site and package-level READMEs.

Observations for later

  • This machine cannot reach PyPI file hosting directly, so docs build was wired through CI and release automation first; local docs preview still depends on running the docs toolchain from a networked environment.
  • The docs workflow currently installs all local packages explicitly so mkdocstrings can import every module; if package count grows, replace that with one reusable installer script to avoid drift.

11. Architecture decision records

Labels: theme:project-health, type:docs

Status: implemented. ADR index/template added and six backfilled ADRs landed for the listed decisions; each includes links to the issue section where the decision was made.

Problem

Cross-package decisions are undocumented, so they get re-litigated or silently reversed. Git history shows what changed, not which alternatives were rejected and why.

Proposal

Backfill ADRs for decisions already made:

Decision Why it needs a record
Fully async provider interface Constrains every provider implementation; the fs provider was sync-under-async until v0.3 item 3
Six packages instead of one with extras Shapes install UX, release process, and version coordination — and the lockstep versioning rule the release guard now enforces
Caching strategy Why provider caching is per-instance with an explicit invalidate(), and why HTTP revalidation is opt-in rather than default
Exception taxonomy Extended by v0.3 item 7 with SkillUnavailableError; the rationale for splitting "not found" from "unavailable" exists only in issue notes
Binary resources as a JSON envelope Chosen for consistency across integrations over MCP-native content blocks. Write this before the follow-up that reverses it
Logging conventions Why one agentskills.* namespace, and why there is deliberately no ERROR level

Add a short docs/adr/README.md with the template and the numbering convention.

Acceptance criteria

  • [x] ADR template and index in place
  • [x] Backfill ADRs written for the decisions above
  • [x] Each links to the issue or PR where the decision was made
  • [x] Contributing guide points at ADRs for design-affecting changes

Implementation notes

docs/adr/README.md now defines the ADR numbering/status convention and ships a template section, so new records follow one shape instead of drifting by author. The index includes ADR 0001/0002 plus six backfilled records for this item:

  • ADR 0003: fully async provider interface
  • ADR 0004: multi-package architecture with lockstep versioning
  • ADR 0005: per-instance provider caching with explicit invalidation
  • ADR 0006: exception taxonomy separating SkillNotFoundError from SkillUnavailableError
  • ADR 0007: binary resources encoded as a JSON envelope
  • ADR 0008: one agentskills.* logging namespace and severity conventions

Each ADR has a "Decision history" section linking to the issue section where the decision was made (primarily v0.3 items 1, 3, 5, 7, 8, and 10; plus v0.4 issue 12 for the publish/packaging trajectory).

Observations for later

  • ADR 0007 records the current JSON-envelope choice and intentionally leaves room for the planned MCP-native binary follow-up; that follow-up now has something concrete to supersede instead of reversing a behavior with no written rationale.
  • ADR 0004 captures the release constraints, but the lockstep model has grown from six to eight packages since the older issue text; keep release docs and scripts aligned whenever package count changes.

12. Simplify the publish workflow: drop the TestPyPI path

Labels: theme:project-health, type:chore

Status: implemented. The fall-through became an explicit refusal, workflow_dispatch now stops at the artifact, and pre-releases go to PyPI; see the implementation notes.

Problem

The publish workflow routes any tag that is not a bare vX.Y.Z to TestPyPI as a dry run. That path is configured for exactly one of the seven packages, and finishing it is more work than it looks: pending trusted publishers must be uniquely identifiable by their claims. All seven distributions share the same owner, repository, workflow and environment, so PyPI cannot tell which pending project an incoming token belongs to and permits only one pending publisher at a time. Bootstrapping the set on a fresh index therefore takes as many sequential publish rounds as there are packages, each one converting a pending publisher into a project-scoped one and freeing the slot.

So today an rc tag publishes agentskills-core and then fails. That is worse than having no dry run, because it fails in a way that looks like a real problem.

The routing has a second weakness that only shows up once TestPyPI is gone. A malformed tag does not fail — it falls through to the other index. The safety property was never "unexpected tags are rejected", it was "unexpected tags land somewhere else", and that is only safe while somewhere else exists.

Proposal

  • Replace the fall-through with an explicit refusal: a tag that does not match the version pattern fails the run with a message naming the expected shape.
  • Make workflow_dispatch a build-only dry run — run the version guard, build all six, upload the artifact, publish nothing.
  • Let pre-release tags publish to PyPI, which handles them natively; pip will not install a pre-release unless --pre is passed, so TestPyPI was never required for this.
  • Delete the testpypi environment and its trusted publisher.

Acceptance criteria

  • [x] A tag of an unexpected shape fails loudly instead of publishing anywhere
  • [x] workflow_dispatch validates and builds without uploading
  • [x] Pre-release tags publish to PyPI
  • [x] DEVELOPMENT.md documents the pending-publisher uniqueness limit

Observations to carry in

The 0.3.0 release proved the OIDC exchange, the claim set, the build loop and the version guard end-to-end against production PyPI — which is what the TestPyPI rehearsal existed to prove. What would remain untested without it needs no index at all.

The pending-publisher limit should be documented regardless of the outcome of this issue, because it applies to any new index and to adding a seventh package.

Implementation notes

The tag pattern is an allow-list of two shapes, not a rejection of one. vX.Y.Z releases, vX.Y.Z plus a1/b1/rc1 pre-releases, and everything else fails with a message naming what was expected. The previous routing was also an allow-list, but its else branch was another index; once that index is gone the same structure fails open, and the fix is a branch that exits non-zero rather than a different destination.

Pre-release is now a property of the tag rather than of the destination. The GitHub Release's prerelease flag used to be derived from environment == 'testpypi', which coupled a release-notes detail to a publishing target that no longer exists. The guard job emits it directly.

workflow_dispatch stops after the artifact. It still runs the version guard and builds all seven distributions, which is what a dry run was ever able to catch here. What it cannot exercise is the upload — and that was true of the TestPyPI path too, because it only ever uploaded one package before failing.

repository-url is gone from all seven publish steps rather than hardcoded to PyPI. The action defaults there. A URL repeated seven times is seven places to be wrong about the same fact.

Observations for later

scripts/publish.{ps1,sh} still offer --test-pypi. They are the token-based emergency path for releasing when Actions is down, and a scratch index is genuinely useful when you are already off the rails. Left alone deliberately: nothing about them depends on the trusted publisher this issue removes.

Nothing now exercises an upload before a real one. That is the honest cost of this change and it was already the situation, since the dry run could only ever publish one package out of seven. The mitigation is skip-existing, which makes a failed release re-runnable rather than unrepeatable.

Carried-over follow-ups

Surfaced during v0.3 and too small for a roadmap row. Good first issues unless noted.

Item Notes
MCP native binary content blocks encode_resource_content() wraps non-UTF-8 in a base64 JSON envelope. MCP has native binary content blocks; using them removes a layer the client must unwrap. Explicit follow-up from v0.3 item 5, and it needs the ADR in item 11 first.
Generate the tool layer from a table Three of the six tools in each integration are near-identical wrappers. Three copies is where a table beats hand-written duplicates.
mypy is referenced but not installed scripts/dev.py and DEVELOPMENT.md both invoke it; it is not in the dev group. Install it or delete the task — a documented command that cannot run is worse than neither.
Poetry metadata deprecations [tool.poetry] name/description/license should move to [project]. Warned on every poetry lock.
validate_skill() parses each skill twice It calls get_body() and get_metadata() independently, and both providers duplicate _get_skill_md() → split_frontmatter(). Caching hid the I/O cost but not the parse.
Widen the lint scope to scripts/ CI lints packages/ and examples/ only, so scripts/dev.py has drifted from the formatter. Widen the scope and reformat once, deliberately.
Coverage ratchet targets mcp_server/__main__.py at 75% — the documented launch path is untested. mcp_server/context_provider.py at 83% — the import guard's two upgrade messages have never executed.
_KNOWN_KEYS and _OPTIONAL_FIELDS are out of step A third specially-validated field should trigger converting the table to {name: validator}.
Export the known frontmatter keys from core agentskills lint cannot warn about unknown keys without duplicating a private constant in a second package. Exporting the set makes the lint a few lines; until then it is absent. From v0.4 item 1.
Semver comparison The inlined regex validates but cannot compare. Adopt the semver package only if range matching is actually needed.
No timing instrumentation v0.3 item 8 asked how long a fetch took, and nothing answers that. Deferred deliberately to OpenTelemetry in v0.6 rather than hand-rolled time.monotonic() deltas.