> ## Documentation Index
> Fetch the complete documentation index at: https://slopsentral.michne.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Schemas and evaluations

> Define external schemas and distinguish offline, decision, and execution evidence.

Schemas define accepted external records.
Evaluations examine decisions or task execution under explicit conditions.
Each proves a different claim.

## Schemas

`source/schemas/` owns JSON contracts for primitives, plugins, profiles, hooks, Pi packages, and evaluation records.
A schema describes accepted fields and finite variants.
A parser or graph resolver adds filesystem, identity, and lifecycle rules.

For example, [resource references](https://github.com/amichne/slopsentral/blob/main/source/schemas/primitives/resource-reference.schema.json) have an external schema.
The graph resolver also examines the target's file kind and dependency closure.
A schema alone cannot establish that a target exists.

Change the owning schema, producer, consumer, and accepted or rejected fixtures together.
Keep compatibility decisions explicit.
[JSON Schema Management](https://github.com/amichne/slopsentral/blob/main/source/skills/manage-json-schemas/SKILL.md) supplies the authoring procedure.

## Evaluation layers

`source/evals/` owns evaluation definitions and fixtures.
`tools/promptfoo/` owns the runtime adapters.
The [evaluation guide](https://slopsentral.michne.com/maintainers/evaluations.md) defines commands and report locations.

| Layer | Evidence and limit |
| - | - |
| Offline controls | Fixture and adapter contracts, without model calls |
| Decision suite | Fresh choices from supplied skill text and routed references, without automatic routing or tool execution |
| Execution suite | Task results in pinned repositories with independent host evidence for each checklist item |
| Historical assessments | Results for their recorded revisions and conditions |

[source/evals/promptfoo/cases.mts](https://github.com/amichne/slopsentral/blob/main/source/evals/promptfoo/cases.mts) owns active decision cases and expected answers.
Extend that owner instead of a parallel catalog.
Record the supplied reference hashes.

The execution definitions under `source/evals/promptfoo/execution/` need an explicit ignored fixture registry.
Each fixture binds the scenario, repository, full commit SHA, context, and independent verifier identity.
Missing fixtures produce nonpassing `BLOCKED_SETUP` without a model call.
The model does not grade its own result.

## Commands

After evaluation changes, run the offline gate:

```sh theme={null}
npm run check:evals
```

For fresh decision evidence, run:

```sh theme={null}
npm run eval:skills
```

The latter command needs a configured Codex runtime and model access.
Normal CI uses offline controls.
An offline pass does not establish a live model pass or measured improvement.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.