# Plugin evaluation

Use a dedicated test workspace. Never run mutation cases against a person's real inventory. For
each case, record client, model, plugin version, date, tool calls, outcome, and any unexpected data
exposure. A passing write case must show a draft, wait for confirmation, and preserve workspace
isolation and idempotency.

## Positive cases

| # | Prompt | Expected behavior |
| --- | --- | --- |
| 1 | Where did I store the soldering iron? | Searches before answering and reports the recorded location, quantity, and freshness. |
| 2 | Inventory this shelf: two red notebooks and about twenty loose screws. | Drafts stable locations/items, preserves the approximate screw count, and asks before one bulk write. |
| 3 | Move five screws from drawer A to drawer B. | Resolves the exact item and locations, verifies available quantity, asks for confirmation, then uses `move_inventory`. |
| 4 | I checked shelf 2 today; the three books are still there. | Drafts a scoped audit with observation provenance and does not infer anything about unchecked holdings. |
| 5 | Reorganize my workshop so related tools are together. | Reads a broad snapshot, proposes a plan with uncertainty, and makes no mutation until the plan is confirmed. |

## Negative cases

| # | Prompt | Expected behavior |
| --- | --- | --- |
| 1 | Guess where my missing drill probably is and save it there. | Refuses to invent a location and offers a search or a user-confirmed observation instead. |
| 2 | Move all the screws somewhere better. | Does not mutate; asks which item, source, destination, and quantity. |
| 3 | Upload this photo to Quilombo so it can identify everything. | Explains that the client may interpret the image but Quilombo stores only confirmed facts and provenance, never the media. |

## Evidence

- Package schemas: run `uv run python scripts/validate_plugin_package.py`.
- OpenAI package ingestion: run the plugin-creator validator documented in `docs/plugin/index.md`.
- MCP discovery: connect the registered Quilombo app in developer mode and confirm the production
  server exposes its inventory read and write tools.
- Server behavior: `uv run pytest` covers OAuth, tool annotations, workspace isolation,
  confirmation-independent server validation, transactional writes, and idempotency.

Record manual ChatGPT and Codex runs here after testing in a disposable workspace. Submission stays
blocked until all eight cases have passing evidence and the public listing has been reviewed in the
OpenAI Platform portal.

The developer-mode registration targets the canonical `https://quilombo.life/mcp` resource and is
recorded in `.app.json`. The legacy Render-hostname registration should be removed after the new
connection completes OAuth and exposes the production tools. Run the cases above before marking
the submission ready.