Testing Toolkit (agentskills-testing)

Provider conformance suite and in-memory test provider.

pip install agentskills-testing

agentskills_testing

Conformance suite and test doubles for Agent Skills providers.

Two things live here:

• :class:ProviderConformanceSuite — subclass it, supply a provider fixture, and pytest runs the whole provider contract against your implementation.
• :class:InMemorySkillProvider — a real, spec-compliant provider backed by a dict, for tests that need a skill but not a disk.

See https://github.com/pratikxpanda/agentskills-sdk for the guide.

ContentLimitConformanceSuite

Assertions for providers that read bytes they did not author.

A size limit is not part of the universal contract — an in-memory provider has nothing to bound, and demanding one would be asserting a filesystem's constraints against a dict. It is required of any provider reading from disk, a network, or anything else a caller can grow without asking.

Supply a limited_provider fixture whose limit is small enough that the fixture's own content exceeds it::

class TestMyProviderLimits(ContentLimitConformanceSuite):
    @pytest.fixture
    def limited_provider(self):
        return MyProvider(..., max_bytes=8)

limited_provider

limited_provider() -> SkillProvider

Return a provider whose size limit the contract skill exceeds.

InMemorySkill dataclass

One skill's content, held in memory.

Attributes:

Name	Type	Description
metadata	dict[str, Any]	Parsed frontmatter. Must carry name and description to satisfy the specification.
body	str	Markdown instructions.
references	dict[str, bytes]	Reference documents by filename.
scripts	dict[str, bytes]	Scripts by filename.
assets	dict[str, bytes]	Assets by filename.

resources

resources(kind: str) -> dict[str, bytes]

Return the resource mapping for kind.

InMemorySkillProvider

Bases: SkillProvider

A spec-compliant :class:~agentskills_core.SkillProvider backed by a dict.

Parameters:

Name	Type	Description	Default
skills	Mapping[str, InMemorySkill | str] | None	Skills by ID. A plain string value is taken as the body of a default skill, so the common case stays short.	None
supports_resource_listing	bool	Set False to emulate a backend that cannot enumerate, which is what the HTTP provider does without a manifest.	True
supports_discovery	bool	Set False to emulate a backend that cannot list the skills it holds.	True

Example::

provider = InMemorySkillProvider({"incident-response": build_skill()})
registry = SkillRegistry()
await registry.register("incident-response", provider)

add

add(skill_id: str, skill: InMemorySkill | str | None = None) -> InMemorySkill

Add or replace a skill.

Parameters:

Name	Type	Description	Default
skill_id	str	ID to register the skill under.	required
skill	InMemorySkill | str | None	The skill, or a string taken as its body. Omit it for a default skill named after skill_id.	None

Returns:

Type	Description
InMemorySkill	The stored skill.

discover async

discover() -> list[str]

Return the stored skill IDs, sorted.

Raises:

Type	Description
DiscoveryNotSupportedError	If the provider was built with supports_discovery=False.

get_asset async

get_asset(skill_id: str, name: str) -> bytes

Return an asset.

get_body async

get_body(skill_id: str) -> str

Return the skill's markdown body.

get_metadata async

get_metadata(skill_id: str) -> dict[str, Any]

Return a copy of the skill's frontmatter.

get_reference async

get_reference(skill_id: str, name: str) -> bytes

Return a reference document.

get_script async

get_script(skill_id: str, name: str) -> bytes

Return a script.

list_resources async

list_resources(skill_id: str) -> dict[str, list[str]]

Return the skill's resource names, grouped by kind and sorted.

Raises:

Type	Description
ResourceListingNotSupportedError	If the provider was built with supports_resource_listing=False.
SkillNotFoundError	If the skill is unknown.

skill_ids

skill_ids() -> list[str]

Return the registered skill IDs, sorted.

ProviderConformanceSuite

Assertions every :class:~agentskills_core.SkillProvider must satisfy.

Subclass it and provide a provider fixture::

class TestMyProvider(ProviderConformanceSuite):
    @pytest.fixture
    def provider(self):
        return MyProvider(...)

The fixture must be populated per :data:CONTRACT. Nothing here is opt-out: a provider that cannot pass these is not a provider, it is a class with the right method names.

provider

provider() -> SkillProvider

Return the provider under test, populated per :data:CONTRACT.

build_skill

build_skill(skill_id: str = DEFAULT_SKILL_ID, *, description: str = DEFAULT_DESCRIPTION, body: str = DEFAULT_BODY, version: str | None = None, metadata: Mapping[str, Any] | None = None, references: Mapping[str, bytes] | None = None, scripts: Mapping[str, bytes] | None = None, assets: Mapping[str, bytes] | None = None) -> InMemorySkill

Build a valid :class:InMemorySkill.

The defaults produce a skill that passes validate_skill(), so a test that does not care about content does not have to invent any.

Parameters:

Name	Type	Description	Default
skill_id	str	Becomes the name in the frontmatter.	DEFAULT_SKILL_ID
description	str	Frontmatter description.	DEFAULT_DESCRIPTION
body	str	Markdown instructions.	DEFAULT_BODY
version	str | None	Optional semver string.	None
metadata	Mapping[str, Any] | None	Extra frontmatter, merged over the generated fields.	None
references	Mapping[str, bytes] | None	Reference documents by filename.	None
scripts	Mapping[str, bytes] | None	Scripts by filename.	None
assets	Mapping[str, bytes] | None	Assets by filename.	None

Returns:

Type	Description
InMemorySkill	A skill ready to hand to :class:InMemorySkillProvider.

render_skill_md

render_skill_md(skill: InMemorySkill) -> str

Render skill as the SKILL.md text a file-backed provider would hold.

Useful for populating a temporary directory when a test needs the filesystem provider rather than this one.