Quest: Make Effection documentation reliable for coding agents

[taras]

Story

A coding agent starting from Effection's published website should be able to find the relevant documentation and implement a non-trivial application correctly without using remembered APIs or reading implementation source first.

The download-all exercise is the test case. Its implementation is evidence about the documentation, packages, and evaluation setup; it is not the product of this quest.

Goal 1: Agents can find the applicable documentation

Signals

• An agent starts with llms.txt and follows the intended guides, API reference, and EffectionX package documentation.
• It does not need search results, production URLs, rendered-HTML extraction, or implementation source to complete its first implementation.
• Local EffectionX packages and the application use the same Effection installation.

Metrics

• All required documentation links remain on localhost during a local exercise.
• Source, tests, history, branches, and pull requests are inspected zero times before the first implementation.
• Every linked @effectionx/* package resolves inside the configured local checkout.
• The agent can identify the relevant APIs and packages from the published indexes.

Current status: Achieved against the local documentation stack in #8 and #10. The corresponding website changes remain open in Effection #1241 and Effection #1242.

Goal 2: The documentation leads agents to correct lifetime behavior

Signals

• Failure and cancellation stop active and queued work.
• Native Promise work and required cleanup settle before the operation finishes.
• A Promise that acquires a resource cannot outlive the cleanup responsible for that resource.

Metrics

• At most five downloads run concurrently.
• Zero queued downloads start after shutdown begins.
• Failure propagates with zero active or queued operations left behind.
• Zero partial files remain after failure or cancellation.
• Zero network requests, filesystem operations, or required cleanup remain when the operation reports completion.
• Deterministic tests pass for normal resource acquisition, halt during pending acquisition, acquisition failure, and cleanup failure.

Current status: The local TaskBuffer fix prevented queued admission during teardown in runs #6 through #9, without an application guard. Promise-based resource acquisition remains unresolved in Effection #1243.

Goal 3: Each exercise produces reviewable, attributable evidence

Signals

• A person can see what the agent was asked, what it produced, and how it assessed the experience.
• Results are attributed to the model responsible for each phase.
• A documentation change is evaluated with a fresh run rather than inferred from the changed text.

Metrics

• transcript.md contains the task prompt, implementation response, assessment prompt, and assessment response—and no internal event log.
• The draft pull request description contains only the assessment response.
• The implementation, assessment, and packaging model are recorded when they differ.
• Every conclusion links to the run that supports it.

Current status: The exercise harness implements the intended transcript and pull request contract. Run #9 demonstrated the need for phase-level model attribution. The manual export in #10 demonstrated why an internal event log is not a useful transcript.

Todo

Documentation delivery

• Add the canonical useAbortSignal() → Promise API → until() example (Effection #1238).
• Validate local Markdown routes for guides, EffectionX packages, the API index, and API symbols in fresh exercises.
• Merge the orderly Promise teardown guidance (Effection #1240).
• Merge the same-site llms.txt, AGENTS.md, guide, and package-documentation routing (Effection #1241).
• Merge support for the local EffectionX checkout (Effection #1242).
• Replace the run() JSDoc fetch example with a small operation that returns a value (Effection #1244).
• Merge the corrected run() example (Effection #1244).

Runtime behavior

• Identify and fix TaskBuffer admission during teardown (EffectionX #260).
• Validate the local TaskBuffer fix without an application shutdown guard.
• Merge the TaskBuffer fix and documentation (EffectionX #260).
• Record the unresolved Promise-based resource-acquisition question (Effection #1243).
• Add a deterministic test that holds acquisition pending, begins halt, and then releases acquisition.
• Test normal completion, pending-acquisition halt, acquisition failure, cleanup failure, and exactly-once cleanup.
• Decide whether the validated answer is existing guidance or requires an Effection API change.
• Implement and document the accepted answer.

Exercise evidence

• Automate isolated worktrees, implementation, assessment, transcript, commit, push, and draft pull request creation.
• Keep the transcript to the four user-visible prompt and response sections.
• Keep the pull request description to the assessment response.
• Record the model responsible for implementation, assessment, and packaging whenever they differ.
• After #1243 is resolved and its answer is published, run download-all again as a regression check.

Progress log

• #1 and #2 established the baseline. They exposed stale discovery paths, unclear TaskBuffer behavior, and the pending resource-acquisition window.
• #3 tested the first Promise and API-documentation improvements and exposed conflicting cleanup guidance.
• #4 and #5 reproduced the old TaskBuffer shutdown defect. The dependency changed during run 01a0bf77-codex: download-all #5, so it is not evidence about the fixed package.
• #6 and #7 validated the local TaskBuffer fix but still encountered documentation-delivery friction and repeated the pending resource-acquisition defect.
• #8 completed the intended local documentation route without source inspection and used the fixed TaskBuffer without an application guard.
• #9 validated the same route as a multi-model workflow, but its assessment missed the pending-acquisition defect.
• #10 validated the Markdown API index and symbol routes. Its implementation still did not prove that native Promise work settled before teardown completed.

Next checkpoint

Resolve Effection #1243 with deterministic tests before running the downloader exercise again. More repetitions against the current guidance are unlikely to distinguish a documentation problem from the still-open Promise-resource question.