v0.5 "Agent Effectiveness"¶

Specifications for the v0.5 roadmap items, in the same order as the table in the roadmap. Milestone: v0.5.

Add the GitHub issue number to a heading once the item is filed.

Items 1 and 2 change contracts the rest of the milestone builds on — the frontmatter schema and the shape of a body fetch. They are ordered first so the others are specified against a settled surface rather than a guessed one. Items 3, 5 and 6 are leaf work that can proceed in parallel once those land.

1. Selection metadata: `when_to_use` and `when_not_to_use`¶

Labels: theme:correctness, type:feature, package:core

Status: shipped in v0.5.0. All four open questions were resolved; see the implementation notes. The upstream proposal was evaluated and deferred — see the acceptance criteria.

Problem¶

description is the only signal an agent has when choosing a skill, and it carries positive evidence only. There is no way to say where a skill stops applying — that incident-response is for a production outage and not for a failing test on a laptop.

False activation is not a lesser failure than non-activation. It is worse. Non-activation costs the tokens of a catalog entry the agent ignored; false activation costs a full body load and puts instructions written for a different situation in front of the model, which then follows them. The SDK currently has no vocabulary for expressing the boundary.

Authors work around this by folding negative conditions into description. That is the wrong place for three reasons: description is capped at 1024 characters by validate_skill(), so the negative signal competes for room with the positive one; it is charged on every turn for every registered skill, whether or not that skill is ever used; and a description that reads "use for X, but not Y, unless Z" is worse at its primary job of being discriminative.

This item is first in the milestone because item 3 ranks skills over a text corpus. Deciding what fields exist after the ranking is built means specifying the corpus twice.

Proposal¶

Two optional frontmatter fields, each a list of short strings:

---
name: incident-response
description: Triage and mitigate production incidents.
when_to_use:
  - A production service is degraded or down
  - An alert has fired and needs triage
when_not_to_use:
  - Debugging a failing test locally
  - Planning capacity for a future launch
---

Implementation is deliberately small. Add both to _OPTIONAL_FIELDS in agentskills_core.validation with type list, which extends _KNOWN_KEYS for free, and add an element-level check that every entry is a non-empty string. Nothing else in core needs to know they exist.

They surface in the catalog next to the description, and only when set — the same rule version already follows, so an unversioned skill without selection metadata renders byte-identical to today:

<skill>
  <name>incident-response</name>
  <description>Triage and mitigate production incidents.</description>
  <when_to_use>
    <case>A production service is degraded or down</case>
  </when_to_use>
  <when_not_to_use>
    <case>Debugging a failing test locally</case>
  </when_not_to_use>
</skill>

These fields are charged on every turn, so they must be bounded and they must be droppable. Cap each entry at 200 characters and each list at five entries — a sixth condition is a sign the skill should be split. Add get_skills_catalog(..., selection_hints=False) for callers who want the cheap catalog back. Without both of those this feature is a way to quietly triple the per-turn cost of a registry.

Tooling follows: agentskills lint warns when a skill has neither field and a description over some length (the author is probably already smuggling conditions into it), agentskills init scaffolds both as commented examples, and agentskills inspect --turn-budget must count them — a per-turn budget that ignores per-turn fields is not a budget.

Per product principle 1 these stay optional and backward-compatible, and the design should be taken upstream to the Agent Skills specification rather than kept as a local extension.

Open questions¶

list[str] or a single free-text string? A list renders and ranks cleanly and bounds naturally; a string is nicer to write and does not push authors into telegraphic fragments.
Top-level, or inside the free-form metadata mapping? tags went into metadata precisely to avoid needing anything defended upstream. The same argument applies here and cuts against a top-level field — but selection metadata is far more central than tags, and burying it hurts discoverability.
Should when_not_to_use be injected at all, or only used for ranking? Negative conditions may be more valuable to a retrieval scorer than to the model, and keeping them out of the prompt would halve the added per-turn cost.
Does an empty list mean "no restrictions" or "author has not thought about it"? Lint needs to tell those apart to say anything useful.

Acceptance criteria¶

[x] when_to_use / when_not_to_use validated as lists of non-empty strings, both optional
[x] Per-entry and per-list limits enforced with a clear error naming the limit
[x] Both rendered in the XML and markdown catalogs, omitted entirely when absent
[x] selection_hints=False restores the current catalog output byte-for-byte
[x] A skill written before this change validates and renders unchanged
[x] inspect --turn-budget counts them; lint warns when both are missing
[x] Fields documented in the concepts page
[x] Upstream proposal evaluated — deferred, and deliberately not filed. The specification already treats description as the mechanism for activation boundaries, and documents an optimization loop for tuning it against near-miss queries. Proposing structured fields therefore needs trigger-rate data showing they beat a tuned description, and we do not have that data. Retained as an optional local extension; revisit with measurements rather than argument.

Implementation notes¶

list[str], top-level, both injected. The first two open questions were answered by the cost argument rather than the ergonomic one. A list bounds naturally — five entries of 200 characters is a ceiling an author can see, where a free-text string only has a length — and a renderer that emits one <case> per entry gives a ranker in item 3 a corpus it can score without re-parsing prose. Top-level rather than inside metadata for the reason the spec itself puts description at the top level: this is selection-critical, and burying it under a free-form mapping tells authors it is decoration.

when_not_to_use is injected, not ranking-only. Keeping it out of the prompt would halve the added per-turn cost, but it would also mean the model never sees the one signal that prevents false activation — which is the failure this item exists to fix. selection_hints=False is the escape hatch for callers who want the tokens back, and it turns off both fields together, because a catalog that shows where a skill applies but not where it stops is worse than one that shows neither.

An empty list means "the author considered it and found nothing." validate_skill() accepts when_to_use: [], and lint distinguishes an empty list from an absent key with in metadata rather than a truthiness check. missing-selection-metadata therefore fires only when neither key is present and the description is over 200 characters — short descriptions are not smuggling anything.

Nothing downstream needed changing. Every integration builds its catalog through SkillRegistry.get_skills_catalog(), and inspect --turn-budget costs the string that call returns, so both the new fields and the selection_hints switch propagate for free.

The renderer drops malformed values instead of raising. Registration validates these fields, so a bad value in the renderer means the catalog is being built over metadata that never passed through register(). _cases_of() warns and omits, on the principle that a malformed hint should cost a skill its hint, not its place in the catalog — the same shape as _tags_of().

Observations for later¶

The 200-character entry cap is a guess. It is enforced with a clear message and easy to raise, but nothing has measured what length actually helps a model discriminate.
Item 3 should rank over when_to_use entries directly. They are already normalised, bounded, and one condition per string, which is a better corpus than the description.
missing-selection-metadata overlaps description-too-long-for-catalog above 500 characters, where both fire. That is deliberate — they recommend different fixes — but if the pair proves noisy, the longer rule should suppress the shorter one.

2. Section-level disclosure¶

Labels: theme:agent-effectiveness, type:feature, package:core, package:langchain, package:agentframework, package:mcp-server

Status: shipped in v0.5.0. All four open questions were resolved; see the implementation notes. One documented deviation from the proposal: get_outline() returns a SkillOutline, not a bare list[SectionRef].

Problem¶

Progressive disclosure currently has two levels and then a cliff. An agent sees a catalog entry (tens of tokens), and the only way to learn anything more is get_skill_body(), which returns the entire markdown body. A thorough 4,000-token skill charges all 4,000 to answer a question covered by one 300-token section.

This penalises exactly the skills that are worth writing. An author who documents six scenarios properly produces a skill that is more expensive to consult than a shallow one, so the format quietly rewards thin content. That inverts the point of the product.

The building block already exists in the wrong package: split_sections() in agentskills_tools.cost splits a body at its headings, respects fenced code blocks so a # shell comment is not read as a heading, and guarantees the parts sum to the whole. agentskills inspect --cost already shows authors a per-section breakdown — the SDK can measure what a section costs but cannot serve one.

Proposal¶

Promote sectioning into agentskills-core and expose it one level below the body.

Move Section and split_sections() into agentskills_core, and have agentskills-tools import them rather than keep its own copy. The v0.4 notes already flagged cross-package duplication of parsing constants as a drift trap; this is the same trap with an algorithm.
Skill.get_outline() -> list[SectionRef] and SkillRegistry.get_skill_outline(skill_id), returning title, level, an addressable key, and an estimated size per section.
Skill.get_section(key) -> str and SkillRegistry.get_skill_section(skill_id, key), raising a SectionNotFoundError in the existing LookupError family.
Two new tools — get_skill_outline, get_skill_section — in all three integrations.

This must not become a provider capability. It is computed on top of get_body(), which providers already cache, so no third-party provider has to grow a markdown parser and the ADR 0002 capability-flag machinery stays out of it. The outline of a skill is a property of its content, not of where the content is stored.

A section fetch is not automatically cheaper than a body fetch. Each one costs a tool call, a model turn, and the tokens of the outline that preceded it. For a 600-token body the round trip is pure loss. The outline tool should therefore report the body's total size and say plainly when fetching the whole thing is cheaper, using the same token counter as inspect --cost so the two never disagree. Shipping this without that guidance makes the common case worse in order to improve the rare one.

Open questions¶

Addressing. Headings are not unique — ## Setup can legitimately appear twice. Slug-plus-ordinal is simple and matches the flat, non-overlapping model split_sections() already uses. A hierarchical path (installation/setup) reads better but implies a tree the splitter does not build.
The outline reports level but addressing is flat. Showing nesting the address space does not model is at best confusing. Either the splitter grows real nesting or the outline stops advertising depth.
Where does the size threshold live — a core constant, a registry argument, or purely advisory text in the tool description?
Should get_skill_body gain a sections=[...] argument instead of a separate tool, so an agent can fetch three sections in one call rather than three?

Acceptance criteria¶

[x] Section / split_sections() live in core; tools re-exports rather than reimplements
[x] get_skill_outline and get_skill_section on Skill and SkillRegistry
[x] Unknown section key raises a LookupError subclass, consistent with the taxonomy
[x] No change to the SkillProvider ABC and no new capability flag
[x] Both tools exposed in the LangChain, Agent Framework, and MCP integrations
[x] Outline reports total body size and identifies when a whole-body fetch is cheaper
[x] A body with no headings yields a single section and still works end to end
[x] Conformance suite unchanged — this is not a provider concern

Implementation notes¶

Addressing is a flat slug with an ordinal on collision. ## Rolling Back a Deploy becomes rolling-back-a-deploy; a second ## Setup becomes setup-2. A hierarchical path was rejected because it would advertise a tree the splitter does not build — and because depth does not disambiguate anyway: ## Setup followed by ### Setup still collides, which is exactly what the flat model says should happen.

The outline shows depth as indentation and states plainly that sections do not nest. The alternative — growing real nesting in the splitter — would break the property that the parts sum to the whole, which agentskills inspect --cost depends on to report a body's cost as a breakdown rather than an overlapping double-count. So the rendered text carries the caveat in words: "Sections do not nest — a section covers only its own text, up to the next heading of any level, so fetching a parent does not include what is indented under it."

The size threshold is a core constant, WHOLE_BODY_CHEAPER_TOKENS = 1000, surfaced as SkillOutline.whole_body_is_cheaper and stated in the rendered text. Not a registry argument: nobody would tune it, and an untuned knob is just a bigger API. Not advisory prose in three tool descriptions: that is precisely the kind of duplicated constant the v0.4 notes flagged as a drift trap.

get_skill_body did not gain a sections=[...] argument. A separate single-key tool keeps each tool's contract one sentence long, and the outline's "past about three sections, call get_skill_body instead" guidance already answers the N-round-trips problem — at three sections of a body you are usually better off buying the whole body.

Deviation: get_outline() returns SkillOutline, not list[SectionRef]. The "which is cheaper" judgement needs the whole-body figure next to the per-section figures, and it needs to be rendered identically everywhere. A bare list would have pushed both into the three integrations, which is how they would come to quote three different costs for the same skill. SkillOutline.render() is the single agent-facing rendering; .sections, .total_tokens and .whole_body_is_cheaper remain available to callers who want to build their own.

estimate_tokens moved from agentskills_tools.lint into core for the same reason: the outline's numbers and inspect --cost's heuristic counter must not be able to disagree. tiktoken stays a tools-only refinement layered on top, since the runtime path cannot take that dependency.

Observations for later¶

The 1,000-token threshold is a considered guess, not a measurement. It should be revisited against real traces of agents choosing between an outline and a body.
Batch section fetch (get_skill_sections(skill_id, keys)) was deliberately deferred. If traces show agents routinely pulling two or three sections, it becomes worth the second tool.
Indentation-versus-flat-addressing remains a live tension. The outline shows a shape it does not let you address. If agents are observed fetching a parent expecting its children, the answer is probably to stop indenting, not to start nesting.

3. Semantic skill selection¶

Labels: theme:agent-effectiveness, type:feature, package:retrieval

Depends on: item 1 (selection metadata supplies the corpus)

Status: shipped in v0.5.0. All five open questions were resolved; see the implementation notes. One documented deviation from the proposal: this did require a change to agentskills-core — a single optional total= keyword on get_skills_catalog() — because two of the acceptance criteria could not otherwise both hold. The reasoning is in the notes.

Problem¶

The catalog is injected on every turn, so per-turn prompt cost is linear in the number of registered skills. get_skills_catalog() already offers include, exclude, tags and max_chars, but every one of those requires the caller to know the answer in advance. Nothing narrows the catalog based on what the user actually asked.

Two costs compound. The obvious one is tokens: at fifty skills the catalog alone is thousands of tokens on every turn of every conversation. The less obvious one is accuracy — selection quality degrades as the candidate list grows, so a large registry is worse at picking the right skill and charges more for the privilege. max_chars is not a substitute; it drops entries from the end, which is arbitrary with respect to relevance.

Proposal¶

A new agentskills-retrieval package. A SkillSelector protocol with one method returning scored skill IDs, and two implementations:

LexicalSelector — zero-dependency BM25 over description, when_to_use, when_not_to_use and tags. The default, so the package is useful with nothing else installed.
EmbeddingSelector — embeds each skill's text once and ranks by cosine similarity. The embedder is a one-method protocol resolved from a dotted path, exactly as the v0.4 eval harness resolves models, so no embedding SDK becomes a dependency of anything.

The integration point already exists. A selector returns skill IDs; those IDs feed the existing include= filter, which runs before any metadata is fetched — so narrowing to five of fifty skills also avoids forty-five provider round trips. Core needs no new code and gains no knowledge of retrieval. If a design for this requires changing agentskills-core, it is the wrong design.

Embeddings are computed once per skill and cached by content hash, so re-registering an unchanged skill is free and editing one invalidates only itself. In-memory by default with a hook for persistence; a hosted registry should not re-embed its corpus on every process start.

Selection must be visible. This trades a deterministic prompt for a better one, and a skill that was scored out is indistinguishable, from inside the agent, from a skill that was never registered. So: log every selection with its scores, and report it in the catalog the way truncation already is — selected="5" total="50" on the XML root, a closing note in markdown.

Open questions¶

How is this measured? The v0.4 eval harness measures whether a skill helps once the model has it; that is a different question from whether the right skill was chosen. Selection needs a labelled fixture set — queries paired with the skill that should win — and a recall@k number that has to clear a bar before the feature is recommended. The roadmap already warns that catalog pruning must be evaluated before it becomes a default; shipping ranking without a measurement is that mistake one item earlier.
What is the query? The last user message is the obvious choice and is wrong for conversations where the topic was established several turns ago.
Does the selector see negative conditions? when_not_to_use is a demotion signal, not a match signal, and BM25 has no natural way to express that.
Separate package or a core extra? Precedent from v0.4 item 1 says separate package. Worth restating explicitly rather than inheriting by accident.
A floor as well as a top-k? Returning the five best of fifty irrelevant skills is worse than returning nothing and letting the agent answer directly.

Notes¶

agentskills-retrieval is a new distribution, so before the v0.5.0 tag it needs a tenth entry in every place a package must be registered (root pyproject.toml three times, COVERAGE_FLOORS, both bump scripts, check_release_version.py, check_declared_dependencies.py, publish.yml twice, docs.yml twice, mkdocs.yml nav, and a docs/packages/ page) plus a pending trusted publisher on PyPI — and only one pending publisher can exist at a time, so it must be registered alone. The name and its ultranormalized neighbours were free on PyPI as of 2026-08-17.

Acceptance criteria¶

[x] agentskills-retrieval published, with LexicalSelector usable with zero extra dependencies
[x] EmbeddingSelector with a dotted-path embedder protocol and no vendor SDK dependency
[x] Selector output composes with include= and requires no change to agentskills-core — composes with include=, but see the deviation below on total=
[x] Embeddings cached by content hash, with a documented persistence hook
[x] Every selection logged with scores, and reported in both catalog formats
[x] A labelled fixture set and a published recall@k for both selectors
[x] Opt-in; the default path is byte-identical to today

Implementation notes¶

Deviation: one keyword was added to agentskills-core. Two criteria were in direct conflict. "Composes with include= and requires no change to core" and "reported in both catalog formats" cannot both hold as written, because core computed the total in truncated="true" shown="N" total="M" after filtering. An include=-narrowed catalog therefore had shown == total and silently claimed to be the whole registry — the exact failure the visibility criterion exists to prevent.

The resolution is get_skills_catalog(total=...): an optional denominator, meaning "the size of the set these entries were chosen from". Core accepts a number and learns nothing about ranking, scoring, queries or selectors, which is what "no change to core" was actually protecting. The default is the post-filter count, so a deliberate static tags= filter does not pay for a "truncated" note on every turn, and a total below the shown count is ignored rather than rendering "showing 2 of 1". The alternative — teaching every integration to prepend its own note — would have put the same sentence in three packages, which is the drift trap the v0.4 notes already flagged twice.

Measurement is a labelled fixture set with asserted floors. tests/conftest.py holds twelve skills with deliberately overlapping vocabulary and thirteen queries phrased as a user would phrase them, each paired with the skill that should win. tests/test_recall.py asserts recall@1 ≥ 0.85 and recall@3 = 1.0, so a change that makes ranking worse fails CI rather than being noticed in production. The numbers are published in the package README with the caveat that a small synthetic corpus is a regression guard, not a benchmark. The EmbeddingSelector is measured against a deterministic stub embedder, which tests the plumbing only — a recall number produced by a stub would be a claim about nothing, and the README says so rather than quoting one.

The caller supplies the query; the package never derives one. Both obvious defaults are wrong in opposite directions: the last user message alone fails whenever the topic was set several turns ago ("try that again" ranks against nothing), and concatenating the history drags in every topic the conversation has touched. Guessing on the caller's behalf would turn an obvious integration decision into a silent accuracy regression, so select() takes a string.

Negative conditions demote, they do not match. when_not_to_use is indexed as a separate field and its score subtracted at NEGATIVE_WEIGHT = 0.5. One bag of words would make a skill match the very query its author wrote it to disclaim — "not for local test failures" would make the skill more likely to win on a local test failure. The weight is below 1.0 because a disclaimer is weaker evidence than a description: authors write far fewer of them and phrase them loosely, and at 1.0 a single shared word could cancel a genuine description match. It is a constructor argument, and 0.0 restores the naive behaviour.

Separate package, restated deliberately. Following v0.4 item 1's precedent rather than inheriting it by accident. A core extra would put a ranking algorithm behind an import guard in the one package everything else depends on, and the whole design here is that core stays ignorant of retrieval.

Both a floor and a top-k, with different defaults for a reason. DEFAULT_EMBEDDING_MIN_SCORE = 0.25 is meaningful because cosine is bounded and comparable across corpora — though still model-specific, hence a constructor argument. DEFAULT_LEXICAL_MIN_SCORE = 0.0 is as high as BM25 honestly allows: the scores are unbounded and corpus-relative, so zero catches "shares no term with any skill" but cannot catch "matched a common word and is still irrelevant". That limit is stated in the README rather than papered over with an invented constant, and it is why recall is measured.

An empty selection falls back to the full catalog. "Selection has no opinion" is not "the agent should have no skills". A wrong prune silently removes a capability, which is a worse and far harder-to-diagnose failure than a few wasted tokens. The fallback is logged, and Selection.rejected carries the near-misses so that "scored just under the floor" and "never registered" are distinguishable.

No stemming. It would need a dependency, and the package's claim is that it is useful with none. The cost is real — "deploys" will not match "deploy" — and buying that back is the main thing EmbeddingSelector is for.

Observations for later¶

The recall floors are over a synthetic corpus that the same person wrote both halves of. The honest next step is a fixture set drawn from real registries and real queries.
BM25's inability to express a meaningful non-zero floor is the weakest part of the design. If traces show irrelevant skills clearing zero on common words, the fix is probably score normalisation against the corpus, not a hand-picked constant.
total= currently reuses the truncated="true" vocabulary, which is not quite right: a selection is not a truncation. If agents are observed treating a narrowed catalog as evidence that skills were dropped arbitrarily, the attribute deserves its own name.
The stub-embedder recall figure is a placeholder. A published number for at least one real embedding model would make EmbeddingSelector recommendable rather than merely available.

4. Session-aware progressive disclosure in the context provider¶

Labels: theme:agent-effectiveness, type:feature, package:agentframework

Status: shipped in v0.5.0 — sub-items (a), (b), (f) and the load-driven half of (d). (c), (e), (g) and (h) are deferred with reasons; see the implementation notes.

Problem¶

AgentSkillsContextProvider.before_run() re-injects the full catalog, the full usage instructions, and all five tools on every agent.run() call, regardless of what the agent already loaded. In a long session that is repeated token spend on information the model already has, plus repeated provider I/O.

Agent Framework's BaseContextProvider gives us before_run / after_run hooks, a provider-scoped state dict that persists across runs within a session, a shared session.state, and a per-invocation SessionContext (input_messages, instructions, tools, response, metadata, options).

Proposal¶

Ordered by value; each could ship independently.

a. Cache the built prompt in state. Store the assembled catalog and instructions after the first before_run(); skip provider I/O on subsequent runs. Optional cache_ttl. Biggest win for HTTP-backed registries and the cheapest to implement.

b. Track loaded skills. In after_run(), inspect context.response for skill tool calls and record IDs in state["loaded_skills"]. Skip re-injecting content the agent already pulled.

c. Conditional tool injection. First turn: inject only get_skill_metadata and get_skill_body. Once a skill body has been loaded, add get_skill_reference / get_skill_script / get_skill_asset for that skill. Framework-level progressive disclosure instead of a prompt instruction, and a smaller tool surface for the model to reason about.

d. Catalog pruning. Inspect context.input_messages for topic signal and advertise only relevant skills, falling back to the full catalog when the topic is unclear. Highest risk — a wrong prune hides a skill the agent needed. Should be opt-in and evaluated before it becomes a default.

e. Usage telemetry via after_run. Count skill and resource usage into state, and expose a callback so hosts can export it. Answers "which skills are actually used" and feeds adoption analytics.

f. Cross-provider signalling. Publish metadata["agentskills_loaded_skills"] on context.metadata so other context providers can adapt; optionally read another provider's topic classification to inform (d).

g. Context messages instead of instructions. Use context.extend_messages() so injected skill content carries source attribution and can be filtered by other providers via context.get_messages(sources={"agentskills"}).

h. Agent-aware formatting. Use the agent handle to pick catalog format by model family (XML for Claude, markdown elsewhere) rather than a fixed constructor argument.

Notes¶

(a) and (b) are the core of the issue; (d) needs an evaluation harness before it can be trusted.
(d) is item 3 solved a second time, worse — with conversation state but without a measurement. Prefer running item 3's selector here over inventing a second ranker in an integration package.
(c) is far more valuable once item 2 lands: the tool surface then grows per section as well as per skill.
Anything here that generalises beyond Agent Framework should land in agentskills-core so the LangChain and MCP paths benefit too.
API reference: _sessions.py

Acceptance criteria¶

[x] Prompt/catalog cached across runs within a session, with opt-out
[x] Loaded skills tracked in provider-scoped state
[x] Repeated before_run() calls remain idempotent
[x] Token count for turn N+1 measurably lower than today for a multi-turn session

Implementation notes¶

Caching alone cannot satisfy the fourth criterion. Caching the assembled prompt saves provider I/O; it does not save a single token, because turn N+1 sends exactly the same text. The token reduction has to come from pruning. Once a skill's full body is in the conversation, its catalog entry is being paid for twice. So (a) and (b) shipped together, and (b) is what the fourth criterion actually rests on.

Pruning is driven by full-body loads only. BODY_TOOLS = {"get_skill_body"}. get_skill_section and get_skill_outline deliberately do not mark a skill loaded: a fragment is not the skill, and pruning the entry after one section read would hide most of it while the agent still needs to know it exists. get_skill_metadata is likewise not a load — it restates the catalog entry rather than replacing it.

Pruned entries are replaced by a compact reminder, so the IDs never disappear from the prompt. An agent that finds no trace of a skill will reasonably conclude it does not have one.

Pruning is declined when it would not pay. The reminder is a fixed cost and a catalog entry is not, so for very terse skills the pruned prompt can be larger. The provider builds both, keeps the shorter, and logs at debug level when it declines. This is the one regression the feature exists to prevent, and asserting it away in a test would have been cheaper than honest. The extra render happens once per cache invalidation, not once per turn.

Never prune to nothing. If every registered skill has been loaded, the full catalog is emitted. "You have no skills" is a worse lie than repeating entries.

Item 3's total= keyword is reused so a pruned catalog reports the narrowing honestly (shown="2" total="3") instead of quietly presenting itself as complete.

No cache_ttl, despite (a) offering one. Within a session the invalidation signal is the registry's skill set and the loaded set, not the clock — both are in the cache key. A TTL would be an untuned knob whose only correct value is the one the key already computes.

Idempotency lives in context.metadata, not state. context is fresh per run and state persists for the session, so a state marker would suppress injection on turn 2 entirely. The marker is agentskills_injected:{source_id}, which also lets two providers over two registries both contribute to one context.

after_run reads context.response rather than wrapping the tools. A caller who passed get_tools() output straight to the agent is then tracked identically to one who let the provider inject. Malformed tool arguments are skipped, never raised on: this is bookkeeping running after a successful turn, and it must not be the thing that fails it.

State values are JSON-compatible — a sorted list, not a set — because a host is entitled to serialise and restore the session.

(d) catalog pruning by topic is not implemented, per this issue's own note: it is item 3 solved a second time, worse. What shipped is pruning by what the agent read, which needs no ranker and cannot hide a skill the agent has not seen. Hosts wanting topic narrowing should compose agentskills-retrieval in front of the registry.

(c), (e), (g), (h) deferred. (c) conditional tool injection is worth much more after item 5's fast path settles what a minimal tool surface looks like. (e) telemetry wants a designed export surface, not a counter dict. (g) extend_messages changes where skill content appears in the transcript and deserves its own evaluation. (h) agent-aware formatting means sniffing model families from a handle whose shape is not stable.

Observations for later¶

Pruning assumes the loaded body is still in the conversation. A host that compacts or truncates history can drop it, leaving the agent with a reminder pointing at text that is no longer there. prune_loaded_skills=False is the escape hatch; a real fix needs a signal from the host that history was compacted, which the framework does not currently expose.
The loaded set is per provider state, so it does not survive a process restart unless the host persists session state. That is the host's call, and the values are JSON-safe for it.
body_loads_in() matches on tool name, so a host that renames the tools loses tracking silently. Worth revisiting if renaming ever becomes supported.

5. Single-skill fast path¶

Labels: theme:performance, type:feature, package:langchain, package:agentframework, package:mcp-server

Depends on: item 3 (the selected set, not just the registered set)

Status: shipped in v0.5.0. All four open questions were resolved; see the implementation notes. One documented deviation: the opt-in is a resolved FastPath value rather than a boolean flag, because resolution is async and two of the three entry points are sync. The reasoning is in the notes.

Problem¶

An agent with exactly one registered skill pays the full discovery apparatus to reach content there was never a choice about. Today that means a catalog listing one skill, five tool definitions, a block of usage instructions describing a multi-step workflow, and then a complete model round trip while the agent calls get_skill_body and waits.

Every part of that is waste in the single-skill case. A catalog exists to let a model choose; there is nothing to choose. Usage instructions describe a workflow with one branch. And the round trip adds latency plus a second billed model turn to deliver text that could have been in the first prompt.

This is also the shape of most first contact with the SDK — one skill, one agent, one job — so the configuration people evaluate the library on is the one it handles least efficiently.

Proposal¶

When the effective skill set is exactly one, inject that skill's body directly into the instructions and skip the catalog, the usage instructions, and the body-fetch tool.

"Effective" means after filtering and selection, not merely len(registry.list_skills()) == 1. A registry of fifty narrowed to one by item 3 is the same situation and should take the same path — which is why this is specified after retrieval.

Resource tools stay. References, scripts and assets are still genuinely progressive, and a skill that carries a 2 MB dataset must not have it inlined because the skill count happened to be one.

Body size decides whether this is a win, so it has to be a named threshold rather than an assumption. Inlining a 20,000-token body on every turn of a long conversation is strictly worse than one tool call that happens once. Gate on a token ceiling using the same counter as item 2, default it conservatively, and log when the fast path was declined and why — silently taking a different code path based on content size is the kind of behaviour that makes token bills impossible to explain.

Ship it opt-in. Promote it to the default only once the numbers are in.

Open questions¶

What is the ceiling, and is it tokens or turns? A body worth inlining for a three-turn conversation may not be for a thirty-turn one, and the integration knows the former but not the latter.
Does the agent still need the usage instructions? They explain resource tools too, so dropping them wholesale may leave the model unaware it can read references.
Interaction with item 4(a). Once the prompt is cached per session, the repeated-injection cost this item avoids is already partly paid down; the two need measuring together rather than separately.
Does the skill's identity still need to be visible so the agent can attribute its behaviour, or does the body simply become part of the system prompt?

Acceptance criteria¶

[x] Opt-in flag on all three integrations with consistent naming
[x] Triggers on the effective set of one, including a set narrowed by selection
[x] Resource tools remain available; only the catalog, instructions and body tool are dropped
[x] Token ceiling enforced, configurable, and logged when the fast path is declined
[x] Measured saving published for a single-skill agent over a representative conversation
[x] Default path unchanged when the flag is off

Implementation notes¶

The decision lives in agentskills-core, not in each integration. New agentskills_core.fastpath provides resolve_fast_path(), FastPath, DEFAULT_FAST_PATH_MAX_TOKENS, FAST_PATH_DROPPED_TOOLS and FAST_PATH_RESOURCE_INSTRUCTIONS. "How many skills, how big is the body" is identical everywhere, so three copies could only ever disagree — and the ceiling is the part that has to be tuned. One knob is tunable; three that must be kept in step are not. This also satisfies "consistent naming" structurally rather than by discipline.

The opt-in is a value, not a boolean — a deviation from the spec. Resolution is async (it reads the body), but get_tools() and create_mcp_server() are sync. A fast_path: bool would have forced either an async variant of both or a second resolution path per integration. So the caller resolves once and passes the result:

fast_path = await resolve_fast_path(registry)
tools    = get_tools(registry, fast_path=fast_path)                  # langchain, agentframework
server   = create_mcp_server(registry, name=..., fast_path=fast_path)
provider = AgentSkillsContextProvider(registry, fast_path=fast_path)

One keyword, one type, all three. None is the normal path, so the call is safe unconditionally, and the prompt an integration injects cannot disagree with the tools it drops about which skill was chosen.

The ceiling default is derived, not guessed — this is the answer to "tokens or turns". The normal path pays the catalog and usage instructions every turn (~506 tokens measured together) and the body once. The fast path pays a ~200-token wrapper and the body every turn. Over T turns the fast path wins while body × (T − 1) < 306 × T:

Conversation length	Fast path wins while body is
1 turn	any size
2 turns	< 612 tokens
3 turns	< 459 tokens
10 turns	< 340 tokens
100 turns	< 309 tokens

The integration knows the body size but not the turn count, so DEFAULT_FAST_PATH_MAX_TOKENS is 300 — the value that needs no assumption about the latter. The first draft used 2000, which the measurement showed loses badly past a handful of turns. 300 is deliberately small: a skill elaborate enough to exceed it is one where re-sending the body every turn costs more than the single tool call the fast path removes, which is exactly when it should not fire. A core test pins the arithmetic and a LangChain test pins the ~506 it rests on, so neither can rot silently.

Measured saving, for a four-section incident-response skill (119-token body) at the default ceiling: 49% at one turn, 42% at three, 38% at ten. The round trip the fast path also removes — a whole extra billed model turn — is not in those numbers.

Four tools are dropped, not one — a second deviation. The criterion says "the body tool", which predates item 2 adding get_skill_outline and get_skill_section. All three index into a body the agent is already holding. get_skill_metadata goes too: it answers "should I use this skill", which is the question the fast path exists because nobody is asking. Leaving any of them registered offers the model three ways to re-fetch text it already has.

Usage instructions are trimmed, not dropped — the answer to the third open question. Dropping them wholesale would leave the model unaware it can read references at all; keeping them would point it at a catalog that is no longer there. FAST_PATH_RESOURCE_INSTRUCTIONS is the resource half without the selection workflow, and it is embedded in FastPath.prompt.

The skill keeps its identity — the answer to the fourth. The body is wrapped in <skill id="..."> rather than becoming an anonymous slab of system prompt, so the agent can still attribute its behaviour to a named skill.

MCP omits tools at construction rather than declining at call time. MCP has no way to hide a registered tool later, and a tool that exists but always refuses is worse than one that was never advertised. Both skills://catalog/* resources serve FastPath.prompt, so an existing client that already injects the catalog needs no change.

Interaction with item 4(a) — the second open question. They do not overlap. Item 4's cache saves provider I/O on repeat turns; the fast path removes the catalog and a round trip altogether. When fast_path is set the provider skips caching and pruning entirely: the body is in the prompt, so there is nothing left to cache against and nothing to prune.

Observations for later¶

resolve_fast_path is resolved once by the caller, so a registry that changes afterwards leaves a stale decision. The Agent Framework provider could re-resolve in before_run since that hook is async; it does not today, and the READMEs say to resolve again instead. Worth revisiting if mid-session registration turns out to be common.
The ~506 figure is measured against the integrations' own usage-instruction text. If that text is rewritten, the derived default should be recomputed rather than left alone.
Nothing wires include= to agentskills-retrieval automatically; the READMEs show the two-line composition. An integration-level selector= argument would remove the step but would also put retrieval in the dependency graph of all three integrations.

6. Vision-native assets¶

Labels: theme:agent-effectiveness, type:feature, package:core, package:langchain, package:agentframework, package:mcp-server

Depends on: the binary-safe resource envelope shipped in v0.3

Status: shipped in v0.5.0. All four open questions were resolved; see the implementation notes. Native delivery is opt-in via vision=True, so the default behaviour is byte-for-byte what it was and the return-type widening is not a breaking change. Recorded as ADR 0009.

Problem¶

Every integration returns resource content as str, so encode_resource_content() wraps anything that is not valid UTF-8 in a JSON envelope with base64 in a content field. For a PNG that means the model receives a description of an image instead of an image.

The token economics are the worst available. Base64 inflates the payload by about a third, the result tokenizes as unstructured noise at roughly one token per few characters, and the model cannot see any of it — so a diagram costs thousands of tokens to convey exactly nothing. The 64 KiB inline cap makes it worse in a different way: most real screenshots exceed it, so they come back as a stub saying the file was too large to include. Either way an author cannot ship a skill that shows the agent an architecture diagram or a UI screenshot.

v0.3 made binary resources safe — they are no longer silently corrupted by decode("utf-8", errors="replace") — and explicitly deferred making them useful. This is that follow-up.

Proposal¶

Stop stringifying resources the host can render natively, and hand them to the model as content blocks:

Integration	Native representation
MCP	`ImageContent` — already in the protocol, and the specific follow-up v0.3 named
LangChain	multimodal content blocks on the message
Agent Framework	the framework's binary/data content type

Core gains one small classifier — given a filename and bytes, report the media type and whether it is a candidate for native rendering — so the three integrations cannot drift on what counts as an image. encode_resource_content() stays exactly as it is and remains the fallback for everything else; this adds a branch above it rather than replacing it.

The 64 KiB inline cap should become per-modality. It was chosen for base64 in a text field, where size maps directly to tokens. A natively-attached image is charged by tile count, so the old number is both too small for images and irrelevant to how they are billed.

Open questions¶

This changes a tool's return type from str to a union, in all three integrations. That is a breaking change to the tool contract and needs either a version gate or an opt-in flag, and the decision should be recorded as an ADR alongside ADR 0007 rather than quietly amending it.
What happens on a text-only model? Handing an image block to a model that cannot accept one is an API error, not a degradation. The integration may not know the model's modality, in which case this has to be caller-declared.
Which formats qualify? PNG, JPEG, GIF and WebP are uncontroversial. PDF is supported natively by some models and not others; SVG is text, so it already arrives readable and arguably should not be converted to a raster block.
Does list_skill_resources need to advertise renderability, so the agent knows an asset is worth requesting before it spends a tool call finding out?

Acceptance criteria¶

[x] Media-type and renderability classifier in core, used by all three integrations
[x] Images returned as native content blocks in MCP, LangChain, and Agent Framework
[x] Non-renderable resources keep the existing JSON envelope, unchanged
[x] Per-modality size limits, documented, replacing the single inline cap
[x] Behaviour on a text-only model is defined and tested, not an exception from the provider API
[x] ADR recorded for the tool return-type change
[x] An example skill carrying an image, with a test proving the model receives it natively

Implementation notes¶

One classifier, in core. classify_resource(name, data, *, max_inline_image_bytes) returns a ResourceMedia — media type, size, whether it is text, whether it is worth rendering. All three integrations call it and differ only in how they wrap the result. encode_resource_content() is untouched and remains the fallback, so this is a branch above v0.3 rather than a replacement.

Detection is by magic bytes, then by name. A name is a claim; bytes are evidence. A .png holding a ZIP is not renderable, and a real PNG called .dat is. The failure mode of the reverse order is promising an image and delivering bytes the provider rejects.

The first version asked "does it decode as UTF-8?" before "what is it?", and a test caught the consequence: a short GIF or WebP header padded with NULs decodes cleanly, so real images were being classified as text. Sniffing now comes first.

The return type widens for everyone; the runtime type only changes when asked. vision defaults to False on get_tools() and create_mcp_server(), so no existing caller sees anything new. A test in each integration pins that the tool set and the argument schema are identical either way — the widening is in the annotation, not the wire.

Scripts were left alone. get_skill_script still returns str. A script is code; there is nothing to render, and widening it would buy nothing.

Resolving the open questions¶

Version gate or opt-in flag? Opt-in flag. A version gate forces every caller to change on a schedule we set, to get a feature many do not want. vision=False costs one keyword for those who do. Recorded as ADR 0009 alongside ADR 0007, which it extends rather than amends.

What happens on a text-only model? The caller declares the capability, via the same flag, and the library never guesses. There is no reliable way to ask a model whether it can see, and the integrations are handed a registry rather than a model. Handing an image block to a text-only deployment is an API error from the provider, which is exactly why this cannot be inferred. Tested as the default path in all three integrations.

Which formats qualify? PNG, JPEG, GIF, WebP. PDF is excluded: some models read it and others reject it, so including it would make the correctness of a tool call depend on which model is behind it. SVG is excluded because it is text the model can already reason about — rasterising it would trade something it can think about for something it can only look at.

Should list_skill_resources advertise renderability? No. Answering honestly means fetching and classifying every resource's bytes to serve a listing call; answering from the extension means sometimes promising a renderable image that is not one. It saves a single tool call and costs a read of everything.

Per-modality limits. max_inline_image_bytes defaults to 5 MiB against 64 KiB for opaque binaries. The binary cap tracks tokens, because base64 in a text field is billed per byte; a native image is billed by tile count, so the same ceiling would have turned nearly every real screenshot into a stub saying it was too large. 5 MiB is the lowest per-image limit among the major vision APIs, so it is the one that is safe everywhere.

Observations for later¶

The example skill now ships two diagrams of the same thing, escalation-flowchart.mermaid and severity-ladder.png, and only one of them is worth rendering. That contrast is the point, but it is also a hint that skill authors will want guidance on when to bundle a picture at all.
Oversized images fall back silently to a JSON envelope with "encoding": "none". The model is told the file was too large, which is honest but not actionable. Downscaling to fit the ceiling would be more useful and needs an image library the SDK does not currently depend on.
vision=True is per-get_tools() call, so an application talking to two models with different modalities builds two tool lists. That is correct but slightly awkward, and worth revisiting if per-call capability negotiation ever becomes available.

v0.5 "Agent Effectiveness"¶

1. Selection metadata: when_to_use and when_not_to_use¶

Problem¶

Proposal¶

Open questions¶

Acceptance criteria¶

Implementation notes¶

Observations for later¶

2. Section-level disclosure¶

Problem¶

Proposal¶

Open questions¶

Acceptance criteria¶

Implementation notes¶

Observations for later¶

3. Semantic skill selection¶

Problem¶

Proposal¶

Open questions¶

Notes¶

Acceptance criteria¶

Implementation notes¶

Observations for later¶

4. Session-aware progressive disclosure in the context provider¶

Problem¶

Proposal¶

Notes¶

Acceptance criteria¶

Implementation notes¶

Observations for later¶

5. Single-skill fast path¶

Problem¶

Proposal¶

Open questions¶

Acceptance criteria¶

Implementation notes¶

Observations for later¶

6. Vision-native assets¶

Problem¶

Proposal¶

Open questions¶

Acceptance criteria¶

Implementation notes¶

Resolving the open questions¶

Observations for later¶

1. Selection metadata: `when_to_use` and `when_not_to_use`¶