Recorded per-test traces, phase 1 (record only) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Record a verifying trace for every executed test: which TestPrune symbols (at which content hash) it executed, which repository files it read, and which fixture, pool and static-init scopes it inherited. Store those traces durably and measure them against the phase-1 done-bar. Test selection does not change in this phase.

Architecture: A post-compile Mono.Cecil weaver writes a woven copy of a test project's build output into bin/Traced/<tfm>/. That copy hardlinks every unchanged file and references a tiny recorder assembly. The recorder attributes each probe hit to the current xUnit v3 test through TestContext.Current, which is AsyncLocal-based, or through an explicit scope a host sets. At process exit it writes one NDJSON dump per process. An ingester joins the dump's probe ids, through the weave manifest, to TestPrune symbols and their content hashes. It writes the result to a separate SQLite trace database that never shares a file with the symbol index. FsHotWatch gains an opt-in tests.traces mode that launches the woven copy and ingests the dumps. The mode first applies to full runs (confirm/nightly), then to every run.

Tech Stack: F# (net10.0 for tooling, net8.0 for the recorder); Mono.Cecil 0.11.6; System.Reflection.Metadata (in the box); Microsoft.Data.Sqlite (already a Core dependency); xUnit v3 on Microsoft Testing Platform v2; Unquote; FsHotWatch plugin framework.

Global Constraints

• Open-source repositories: no private tracker ids, no consumer project names, no session links in code, comments, commits, changelogs or docs. Commit trailer: Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>.
• VCS is jj. Always jj commit -m "…". For a message with backticks, $ or quotes, use jj describe --stdin <<'MSG' … MSG, then jj new.
• F# only, formatted with Fantomas, FSharpLint clean, TreatWarningsAsErrors=true, GenerateDocumentationFile=true on packable projects (repository convention).
• TestPrune.Trace.Recorder targets net8.0. Its only package dependency is FSharp.Core pinned to 8.0.403 (DisableImplicitFSharpCoreReference plus an explicit VersionOverride). Loaded into a consumer's test process, it must never raise that process's FSharp.Core floor.
• TestPrune.Trace targets net10.0 and references Mono.Cecil 0.11.6.
• The weaver refuses optimized assemblies (DebuggableAttribute without DisableOptimizations/0x100). Release builds inline small functions across assemblies, so their entry probes never fire.
• Every woven PDB must decode with System.Reflection.Metadata, and every touched non-generic method must JIT-prepare in the test app's own runtime. Any failure refuses the project, which then runs untraced exactly as today.
• No module-value ldsfld probe. In Debug builds, F# reads module values through get_x(), so the method-entry probe on the getter records them.
• [<Literal>] and inline symbols cannot be probed. Phase 1 records nothing for them; phase 2 handles them with a one-hop static rule.
• Phase 1 never changes a verdict or a selection. A recording failure is logged and stored as a refused or incomplete trace, never raised.
• A trace is either complete or explicitly marked incomplete with reasons. Nothing is silently partial.
• The trace database is a separate file with its own TraceSchemaVersion and forward-only migrations. A TestPrune.Core SchemaVersion bump never touches it.
• Overhead is measured as process CPU time (user+sys), never wall time.
• Everything is behind a flag. With no tests.traces key in .fshw.json, FsHotWatch behaves byte-for-byte as it does today.
• Verification:
  • Inner loop: dotnet build, then dotnet run --project tests/<Project> --no-build -- --filter-class <Class>.
  • Gate: each repository's mise run ci. Read the fshw verdict file as well as the exit code.
  • Never pipe a gating command into tail, head or grep. Redirect the output to a file, then read the file and the exit status separately.

Background: what the spikes established

This section is here so an implementer does not re-derive it. Every number was measured with the prototypes this plan productizes.

• Mechanism:
  • A post-compile IL weaver plus an AsyncLocal-attributed recorder.
  • MS CodeCoverage and Coverlet keep process-global counters, which are unusable per test when tests run in parallel.
  • EventPipe has no per-call method-enter event.
  • Profiler ELT hooks cannot read AsyncLocal attribution.
• Probe set (final):
  • method entry in product assemblies;
  • union-case probes: the callee-side get_Tag return; the union's tag-field ldfld outside the tag getter; isinst on a case class, recorded only on success; castclass/ldfld on a case class; _unique_X singleton reads;
  • type probes: isinst recorded on success, castclass, unbox(.any), the FSharp.Core UnboxGeneric/UnboxFast/TypeTestGeneric/TypeTestFast intrinsics, and ldfld(a) of a product type's field outside the declaring type;
  • a .cctor try/finally depth counter that routes hits to a S:static-init scope.
• Test assemblies are woven sites-only: no method-entry probes, but their matches and type tests on product types are recorded. Full test weaving is optional and costs about 15 % CPU.
• Two weaver bugs were caught only by JIT verification:
  • a union case literally named Tag has a static get_Tag returning the case singleton;

  • a case field named tag gets a backing field _tag.

Fixes: - the tag getter is the instance get_Tag returning int32; - the tag field is the field that getter returns, never a field chosen by name. - PDB bug: F# emits a hidden sequence point at IL offset == code size. Cecil cannot bind it to an instruction and keeps the raw offset, so after inserting a probe it writes a negative delta into the portable PDB. SRM, and therefore MS CodeCoverage, then drops the whole method. - Fix: before inserting, collect every unbound sequence point; afterwards, re-create each one at offset + inserted bytes. - With the fix, coverage is identical with and without weaving, so traces record in the same run as coverage. - Attribution: - Isolated vs parallel on 80 tests showed 0 extra-in-parallel ids. - The only ids missing in parallel runs are once-per-process code: .cctors, closure-singleton constructors, and a handful of memoised lazies. - Unattributed (A:ambient) hits: 0. - Overhead: under CPU-time noise for method-entry plus site probes. Wall time is bimodal and is set by a suite's fixed-timeout tests, not by probes; that is why the done-bar uses CPU. - Join: - Match by name within the same file before matching by line. Line matching mis-attributed a record field to its neighbour when the index and binary trees differed. - Union-generated members map to cases or are dropped. - _X and X@DebugTypeProxy classes map to case X. - Hit-level mapped rate on a ~9k-test suite: 98.1 % of method ids, 99.4 % of case ids and 99.0 % of type ids. - Storage: - Pairs layout: about 15 MiB per 8k unit traces; selecting 200 changed versions takes 0.6 ms. - Unmapped executed methods become file-level entries, which is sound.

Decisions (with reasons)

D1. Packaging

How the recorder gets into the test app: the shadow-bin step rewrites the shadow copy's <App>.deps.json:

• It adds a project-type library TestPrune.Trace.Recorder/<version> whose runtime asset is TestPrune.Trace.Recorder.dll.
• It lists that library as a dependency of the root target library.
• It copies the DLL into the shadow directory.
• The original bin/Debug/<tfm>/<App>.deps.json is never written: the shadow file is a fresh file, not a hardlink.
• If a consumer's deps.json already lists the recorder (it referenced the package directly), the step checks that the version equals the weaver's. On a mismatch it refuses with RecorderVersionSkew.
• The recorder is copied without its PDB. MS CodeCoverage skips modules without symbols, so the recorder never appears in a consumer's coverage report. [<ExcludeFromCodeCoverage>] is deliberately not used: it would also hide the recorder from this repository's own coverage ratchet. Task D1 verifies the claim on a real coverage run. If it fails, the README's fallback is a <ModulePath> exclusion in the consumer's coverage settings. Without this, recorder sources outside the consumer's repository would be CoveragePathRejected rows, which widen selection.

How attribution hooks xUnit v3 / MTP:

• The recorder reflects once over Xunit.TestContext, xunit.v3.core. It binds TestContext.Current to a delegate and the ITestContext.Test/TestClass/TestMethod/TestCollection properties to PropertyInfos. It takes no compile-time xUnit dependency: the same recorder serves any xUnit v3 version whose public ITestContext shape is unchanged.
• Scope precedence per hit is:
  1. static-init depth > 0 → S:static-init;
  2. an explicit scope (Scopes.Enter) → that key;
  3. an xUnit test → T:<ITest.UniqueID>;
  4. a class (fixture construction) → C:<class>;
  5. a collection → L:<collection>;
  6. an assembly → A:assembly;
  7. otherwise → A:ambient.
• No per-test flush hook in phase 1. The recorder dumps once, at ProcessExit.
  • xUnit v3 publishes TestState only on a new context object created after the test finished, and MTP registers in-process extensions only through a build-time generated entry point. A per-test flush would need a compile-time package reference in every consumer test project, and the scope would still be flushed before class disposal ran.
  • An exit-time dump attributes every late continuation of a test to that test. That over-attributes, which is sound, so there are no "late hits" to mark.
  • A process killed before exit writes no dump. Every test of that process is then recorded as having no trace (reason recorder-no-output on the run), which is also sound.
  • Memory is O(tests × ids/64) words: about 113 MB for a 9k-test project at 100k ids. The done-bar measures it. Two-level sparse bitsets are the documented fallback if it bites; that is recorded as an ADR in Task 13.
• Test outcome (passed/failed/skipped) is not read from the recorder. It comes from the CTRF report fshw already reads as the verdict source. It is joined on the test's display name, then on (class, method).

D2. Shadow bin

• Location: <projectDir>/bin/Traced/<tfm>/, a sibling of bin/Debug/<tfm>/, at the same depth. Tests that probe upward for a repository marker (*.sln, mise.toml, .fshw.json) or navigate relative to AppContext.BaseDirectory behave identically. A scratch directory outside the repository made at least 15 repo-probing TestPrune tests fail.
• Build flow (ShadowBin.prepare), run once per traced project launch:
  1. Mirror bin/Debug/<tfm>/ into bin/Traced/<tfm>/ with hardlinks (link(2), or CreateHardLinkW on Windows), falling back to a copy on EXDEV. Mirroring re-links every run. A rebuilt bin/Debug file is a new inode, and a stale hardlink would otherwise keep the old bytes. Files present in the shadow but not in the source are deleted.
  2. Select the weave set: every *.dll in the mirror whose portable PDB lists at least one document under the repository root. This auto-detects "built from this repo" with no configuration. The test project's own assembly is woven SitesOnly by default (weaveTests: "full" opts in to method probes); every other repo assembly is woven Full.
  3. Refuse the project if any weave-set assembly is optimized.
  4. Compute the weave key: SHA-256 over the weaver version, the recorder version, the weave mode per assembly, and each weave-set input's .dll + .pdb bytes. On a cache hit under <projectDir>/obj/traced/<key>/, reuse the woven files. On a miss, weave into that directory.
  5. Replace, never write through, a hardlink. Woven assemblies, their PDBs, the patched deps.json and the stamp are written to a temp file and moved over the mirror entry with File.Move(tmp, dst, true). A rename replaces the directory entry. Writing into the hardlinked file would corrupt bin/Debug. A test pins this.
  6. SRM-decode every woven PDB. On a new weave key, JIT-verify by launching the shadow apphost with DOTNET_STARTUP_HOOKS pointing at the recorder and TESTPRUNE_TRACE_VERIFY=<report.json>. The recorder's startup hook PrepareMethods every touched method in the test app's own runtime, including shared frameworks such as ASP.NET Core, writes the report and exits before Main. The verdict is cached in the weave-key directory.
  7. Write bin/Traced/<tfm>/.testprune-trace.json (the stamp): weave key, manifest path, id count, and verify verdict.
• Invalidation: the weave key is content-addressed, so a rebuild with new bytes is a miss and anything else is a hit. obj/traced/ holds at most the 3 most recent keys; older ones are deleted at the end of prepare. bin/Traced and obj/traced live under bin/ and obj/, which the fshw watcher, .gitignore and build-output freshness checks already ignore.

D3. Storage that survives TestPrune schema bumps

• A separate SQLite file. fshw uses .fshw/test-traces.db and the CLI uses .test-prune-traces.db. Core deletes the index file on a SchemaVersion bump, and Ports.PluginStore tables live inside that file. That is why the plugin-store seam cannot hold data that costs a full recording run to rebuild.
• Forward-only migrations. TraceSchemaVersion is stored in PRAGMA user_version. Opening an older file applies each missing migration in one transaction. A newer file is refused with TraceSchemaNewerThanConsumer and left untouched. The file is never deleted by code.
• Keys survive re-index. Symbols are referenced by full_name (and symbol_versions interns (full_name, content_hash)), never by symbols.id, which a re-index reassigns.
• Hash scheme is part of the fingerprint E. If Core changes what a content hash means (for example, the split Type hash in Task 0), every trace recorded under the old scheme becomes a fingerprint mismatch rather than a spurious "verifies". Phase 1 uses Core's Database.SchemaVersion as the hash-scheme number, because any change to hash semantics must bump it (stored hashes are recomputed).
• Tables (from the design, normalised on scope; see Task 3 for the DDL):
  • trace_runs: one per (run, test project);
  • trace_scopes: every recorded scope of a run (T/C/L/A/P/S). Fixture and pool scopes are stored once;
  • symbol_versions: interned;
  • trace_entries (symbol_version_id, scope_id);
  • trace_inputs (scope_id, kind, key, hash): files, existence probes, directory listings and file-level entries;
  • trace_tests: the current trace per (test_key, env_fingerprint), with status, complete and incomplete_reasons;
  • trace_test_scopes (test_id, scope_id): the test's own T scopes (theory rows are unioned) plus every inherited scope.
• Deviations from the design's §4 sketch, with reasons:
  • Entries are keyed by scope, not by test. The same class/pool scope content is otherwise duplicated into every test of the class.
  • trace_runs is per project, because E is per test project (deps.json).
  • There is no late_hits column, because the exit-time dump has no late hits (D1).
• Fingerprint E: SHA-256 over canonical JSON of:
  • the runtime description (reported by the recorder);
  • the SHA-256 of the original <App>.deps.json;
  • OS and architecture;
  • the recorder and weaver versions;
  • the hash scheme;
  • SHA-256 of each configured fingerprintInputs file (sorted);
  • the value of each configured fingerprintEnv variable (sorted; a hash of the value is stored, never the value).

D4. FsHotWatch integration

Config (.fshw.json):

{
  "tests": {
    "traces": {
      "record": "full-runs",
      "db": ".fshw/test-traces.db",
      "weaveTests": "sites",
      "fingerprintInputs": ["global.json"],
      "fingerprintEnv": ["ASPNETCORE_ENVIRONMENT"]
    },
    "projects": [
      { "project": "Some.Tests", "traces": false }
    ]
  }
}

• record: "off" (the default when the key is absent), "full-runs" or "every-run".
• A per-project "traces": false opts that project out.

When it records:

• The question is TestMode.recordsTraces policy mode, and it lives in TestMode.fs, the only file allowed to branch on the mode.
• full-runs records under PassThrough (confirm, nightly).
• every-run records under both modes: partial runs replace only the traces of tests they executed.
• Phase 1a ships full-runs. Phase 1b flips a consumer to every-run only after the done-bar holds on full runs (Task F5).

Launch:

• For a traced project, fshw calls TraceSession.prepareProject, which runs D2.
• It then launches the shadow apphost directly (bin/Traced/<tfm>/<AssemblyName>), not dotnet run. dotnet run --no-build resolves the original output path. The shadow apphost carries the shadow deps.json and runtimeconfig.
• App arguments are derived from the configured dotnet run … args. run, --project/-p X, --no-build, --no-restore, -c/--configuration X and -f/--framework X are dropped, as is every standalone -- token. Filter, CTRF and coverage arguments are appended unchanged.
• Any other token before the first -- is a dotnet run option fshw does not understand. The project then falls back to the normal untraced launch, with reason unrecognized-run-option:<token>.
• DOTNET_ROOT is set from the environment or from the resolved dotnet muxer's directory. dotnet run sets it for its child, and an apphost launched directly needs it.
• Environment added:
  • TESTPRUNE_TRACE_OUT=<runDir>/traces/<project>/;
  • TESTPRUNE_TRACE_IDS=<id count>;
  • TESTPRUNE_TRACE_REPO_ROOT=<repoRoot>.

Ingestion:

• It runs after the parallel project runs finish, serially, next to coverage ingest.
• For each traced project, fshw calls TraceIngest.ingest with:
  • the dump directory;
  • the manifest;
  • the project's CTRF outcomes;
  • the launch's input tree hash and the tree hash at ingestion;
  • Ports.toSymbolStore db.
• If the tree moved during the run, the run is stored status = tree-moved and every test of it is complete = 0, reason tree-moved: the index no longer describes the binary.

Failure handling:

• Every trace step is inside a try that turns an exception into a refused or failed trace_runs row and a warning line. A refused prepare falls back to the untraced launch, so the project still runs and the verdict is exactly today's.
• Incomplete-trace reasons are a closed union, IncompleteReason, each with one producer:
  • NoOutcome (the test is not in the CTRF report);
  • NotPassed of status (failed, skipped or other);
  • SourceDrift of file (the PDB document hash differs from the file on disk);
  • NotIndexed of file (the executed code's file is absent from the index);
  • ChildProcessUntraced of fileName (the test started a process with no dump);
  • RecorderOverflow (a probe id ≥ TESTPRUNE_TRACE_IDS);
  • TreeMoved;
  • DumpRejected of reason;
  • UnmappedCode of detail (executed code the joiner mapped to neither a symbol nor a file).

D5. Consumer side (kept out of this repository)

A consumer with pooled in-process web servers needs to do four things; the concrete tasks are in a separate consumer plan:

• send the current scope key on test HTTP and browser clients (X-Test-Trace-Scope);
• install a first-in-pipeline Test-only middleware that calls Scopes.Enter with it;
• run pool construction under an explicit P: scope;
• link each fixture that acquires a pooled server to that server's pool scope.

This repository provides:

• the stable reflection contract (TestPrune.Trace.Recorder.Scopes: Enter, Exit, CurrentKey, LinkCurrentTo), documented in the README. A consumer can bind it by reflection, with no package reference, and it is inert when the recorder is absent;
• child-process inheritance, implemented in the recorder's process-start shims (Task 6), so consumers write nothing for it.

D6. Phase-1 done-bar: every item is a command with a threshold

File structure

TestPrune (this repository)

FsHotWatch

Cross-task contracts

These are fixed in Task 1, so tasks that run in parallel agree on names.

Recorder public surface (namespace TestPrune.Trace.Recorder; every type [<AbstractClass; Sealed>] with static members, so each is a stable CLR static class):

type Probes =
    static member Hit: id: int -> unit
    static member HitIfNotNull: value: obj * id: int -> unit      // weaver emits: dup; ldc id; call
    static member HitIfTrue: value: bool * id: int -> unit
    static member HitTag: tag: int * baseId: int -> unit           // records baseId + tag
    static member EnterStatic: unit -> unit
    static member ExitStatic: unit -> unit

/// The consumer contract. Bind by reflection or by package reference. Inert when inactive.
type Scopes =
    static member Enter: key: string -> unit        // AsyncLocal override for the current flow
    static member Exit: unit -> unit
    static member CurrentKey: unit -> string        // "T:<uid>" | "C:…" | … | null when inactive
    static member LinkCurrentTo: scopeKey: string -> unit   // current scope inherits scopeKey

Dump format (testprune-trace/1, one file per process, written as trace-<pid>.ndjson.tmp then renamed to trace-<pid>.ndjson):

{"format":"testprune-trace/1","pid":123,"parentScope":null,"runtime":".NET 10.0.0","os":"OSX","arch":"Arm64","ids":102651,"cpuMs":43120,
 "counters":{"test":0,"class":0,"collection":0,"assembly":0,"override":0,"staticInit":0,"ambient":0,"overflow":0}}
{"key":"T:abc","test":{"class":"Ns.M+C","method":"m","display":"Ns.M+C.m"},"parents":["C:Ns.M+C","L:Test collection for Ns.M+C","A:assembly"],
 "links":[],"ids":[5,17,19],"inputs":[{"kind":"read","path":"/repo/x.json"}],"children":[{"pid":456,"file":"dotnet","env":true}]}
{"end":true}

A dump without the {"end":true} line is rejected (DumpRejected "truncated").

Manifest (manifest.tsv, one row per probe id, tab-separated, no header): id kind assembly type member document firstLine lastLine, where kind is one of user, gen, cctor, case or type. documents.tsv has path sha256 (the PDB's recorded source hash, lowercase hex).

Shared model (src/TestPrune.Trace/Model.fs):

module TestPrune.Trace.Model

type ProbeKind =
    | UserMethod
    | GeneratedMethod
    | StaticCtor
    | UnionCase
    | TypeUse

type ManifestRow =
    { Id: int
      Kind: ProbeKind
      Assembly: string
      TypeName: string   // CLR full name, nested types with '+'
      Member: string     // method name, or case name for UnionCase, "" for TypeUse
      Document: string option   // absolute path from the PDB
      FirstLine: int
      LastLine: int }

type WeaveMode =
    | Full
    | SitesOnly

type Manifest =
    { Rows: ManifestRow[]
      Documents: Map<string, string>  // path -> sha256
      IdCount: int }

/// Named so no case shadows a core module (`List`) or a record label (`Exists`).
type InputKind =
    | FileRead
    | ExistenceProbe
    | DirectoryListing

type RecordedInput = { Kind: InputKind; Path: string }
type ChildNote = { Pid: int; FileName: string; EnvInjected: bool }

type TestIdentity = { Class: string; Method: string; Display: string }

type RecordedScope =
    { Key: string
      Test: TestIdentity option
      Parents: string list
      Links: string list
      Ids: int[]
      Inputs: RecordedInput list
      Children: ChildNote list }

type HitCounters =
    { Test: int64; Class: int64; Collection: int64; Assembly: int64
      Override: int64; StaticInit: int64; Ambient: int64; Overflow: int64 }

type ProcessDump =
    { Pid: int
      ParentScope: string option
      Runtime: string
      Os: string
      Arch: string
      IdCount: int
      CpuMs: int64
      Counters: HitCounters
      Scopes: RecordedScope list }

type OutcomeKind =
    | Passed
    | Failed
    | Skipped
    | OtherOutcome

type TestOutcome = { Name: string; Outcome: OutcomeKind }   // CTRF "name" verbatim

type IncompleteReason =
    | NoOutcome
    | NotPassed of OutcomeKind
    | SourceDrift of file: string
    | NotIndexed of file: string
    | ChildProcessUntraced of fileName: string
    | RecorderOverflow
    | TreeMoved
    | DumpRejected of reason: string
    /// Executed code the joiner could map to neither a symbol nor a source file.
    | UnmappedCode of detail: string

Task graph and parallel groups

G0:  T0 (external, in flight)  ∥  T1
G1:  T2  ∥  T3  ∥  T4  ∥  F1  ∥  F2  ∥  F3        (and consumer C1–C3, see the consumer plan)
G2:  T5 (T4)  ∥  T6 (T2,T4)  ∥  T7 (T2,T4)  ∥  T8 (T2,T4)
G3:  T9 (T3,T7)
G4:  T10 (T5,T6,T8,T9)  ∥  T11 (T9)
G5:  T12 (T10)
G6:  T13 release (T10–T12)
G7:  F4 (T13, F1–F3)            → FsHotWatch release
G8:  D1 dogfood done-bar on TestPrune (F4 released + pinned)   ∥  consumer C4–C6
G9:  F5 every-run (D1 bar met on full runs)

Each task ships on its own, behind a flag:

• New packages are inert until a host calls them.
• F1–F3 are dormant code paths until F4 wires them.
• F4 is gated on tests.traces.record ≠ off.
• Consumer changes are no-ops while the recorder is absent.

Task 0: Split the Type content hash (in flight elsewhere; interface only)

This task is being done separately. Phase 1 does not block on it. This section fixes the interface this plan relies on.

Interface consumed by this plan:

• TestPrune.Database.SchemaVersion is bumped (15 → 16) by the change, because stored content_hash values change meaning. Task 9 folds SchemaVersion into the fingerprint E as the hash scheme. So every trace recorded before Task 0 lands becomes an R5 fingerprint mismatch, not a spurious verification.
• SymbolInfo.ContentHash for Kind = Type covers the type header only: name, attributes, type parameters, base and interface list. Union cases (Kind = DuCase), record fields and members each carry their own range's hash. The joiner (Task 7) maps a type probe to the Type symbol and a case probe to the DuCase symbol either way.
• If Task 0 also introduces per-field symbols (for example Kind = Property for record fields), Task 7's TypeUse rule stays "map to the Type". A follow-up (not in phase 1) may split a field read into its field symbol. That follow-up allocates per-field ids, so it is a re-weave. The weave key changes with the weaver version, so the re-weave happens automatically.

No steps here.

Task 1: Scaffold the two packages, the test project and the fixtures

Files:

• Create: src/TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj
• Create: src/TestPrune.Trace.Recorder/Contract.fs
• Create: src/TestPrune.Trace.Recorder/Probes.fs (stub bodies)
• Create: src/TestPrune.Trace.Recorder/CHANGELOG.md
• Create: src/TestPrune.Trace/TestPrune.Trace.fsproj
• Create: src/TestPrune.Trace/Model.fs (the contract in "Cross-task contracts", verbatim)
• Create: src/TestPrune.Trace/CHANGELOG.md
• Create: tests/TestPrune.Trace.Tests/TestPrune.Trace.Tests.fsproj
• Create: tests/TestPrune.Trace.Tests/ScaffoldTests.fs
• Create: tests/TestPrune.Trace.Tests/Fixtures.fs (paths to the built fixtures)
• Create: tests/TraceFixtures/src/FxLib/{FxLib.fsproj,Types.fs,Values.fs,Logic.fs}
• Create: tests/TraceFixtures/src/FxDriver/{FxDriver.fsproj,Program.fs}
• Create: tests/TraceFixtures/tests/FxTests/{FxTests.fsproj,AttributionTests.fs}

The fixtures are laid out as a miniature repository (src/, tests/) so TestPrune's own indexer can index tests/TraceFixtures as a repository root; Task 7 relies on that. - Modify: TestPrune.slnx, Directory.Packages.props, semantic-tagger.json, .fshw.json

Interfaces:

• Produces: TestPrune.Trace.Recorder.Contract literals (below). Tasks 2, 4, 5, 6 and 8 use these names and never spell a probe name inline.

• Produces: Probes and Scopes with their final signatures and no-op bodies (Task 2 fills them in).

• Produces: TestPrune.Trace.Model (the contract above).

• Produces: Fixtures.fxLibDir, Fixtures.fxDriverDir, Fixtures.fxTestsDir, Fixtures.repoRoot. Each is an absolute bin/Debug/net10.0 directory of a fixture built by the normal solution build.

• [ ] Step 1: Write the failing test

tests/TestPrune.Trace.Tests/ScaffoldTests.fs:

module TestPrune.Trace.Tests.ScaffoldTests

open System.IO
open System.Reflection
open Xunit
open Swensen.Unquote
open TestPrune.Trace.Recorder
open TestPrune.Trace.Tests

[<Fact>]
let ``recorder exposes every probe the contract names, with the contract signature`` () =
    let t = typeof<Probes>

    let has name (args: System.Type[]) =
        t.GetMethod(name, BindingFlags.Public ||| BindingFlags.Static, null, args, null) |> isNull |> not

    test <@ has Contract.Hit [| typeof<int> |] @>
    test <@ has Contract.HitIfNotNull [| typeof<obj>; typeof<int> |] @>
    test <@ has Contract.HitIfTrue [| typeof<bool>; typeof<int> |] @>
    test <@ has Contract.HitTag [| typeof<int>; typeof<int> |] @>
    test <@ has Contract.EnterStatic [||] @>
    test <@ has Contract.ExitStatic [||] @>

[<Fact>]
let ``recorder targets net8 and carries no package dependency but a low FSharp.Core`` () =
    let asm = typeof<Probes>.Assembly
    let refs = asm.GetReferencedAssemblies()
    let isFramework (n: string) = n = "netstandard" || n.StartsWith "System." || n = "System"
    // Anything that is neither the framework nor FSharp.Core would be a package the
    // consumer's test process has to resolve.
    let foreign = refs |> Array.filter (fun a -> a.Name <> "FSharp.Core" && not (isFramework a.Name))
    let fsCore = refs |> Array.find (fun a -> a.Name = "FSharp.Core")
    test <@ Array.isEmpty foreign @>
    test <@ fsCore.Version.Major = 8 @>

[<Fact>]
let ``the fixtures are built next to the tests`` () =
    test <@ File.Exists(Path.Combine(Fixtures.fxLibDir, "FxLib.dll")) @>
    test <@ File.Exists(Path.Combine(Fixtures.fxDriverDir, "FxDriver.dll")) @>
    test <@ File.Exists(Path.Combine(Fixtures.fxTestsDir, "FxTests.dll")) @>

• [ ] Step 2: Run the test to verify it fails

Run: dotnet build Expected: FAIL. The build breaks because tests/TestPrune.Trace.Tests does not exist yet, so nothing named TestPrune.Trace.Recorder compiles.

• [ ] Step 3: Create the projects

src/TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net8.0</TargetFramework>
        <PackageId>TestPrune.Trace.Recorder</PackageId>
        <Version>0.1.0</Version>
        <Authors>Michael Glass</Authors>
        <Description>Runtime recorder loaded into a woven test process: attributes probe hits to the current xUnit v3 test.</Description>
        <PackageLicenseExpression>MIT</PackageLicenseExpression>
        <RepositoryUrl>https://github.com/michaelglass/TestPrune</RepositoryUrl>
        <Optimize>true</Optimize>
        <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
        <GenerateDocumentationFile>true</GenerateDocumentationFile>
        <!-- Loaded INTO a consumer's test process: never raise its FSharp.Core floor. -->
        <DisableImplicitFSharpCoreReference>true</DisableImplicitFSharpCoreReference>
        <NoWarn>$(NoWarn);NU1605;NU1608</NoWarn>
    </PropertyGroup>
    <ItemGroup>
        <Compile Include="Contract.fs" />
        <Compile Include="Probes.fs" />
    </ItemGroup>
    <ItemGroup>
        <PackageReference Include="FSharp.Core" VersionOverride="8.0.403" />
    </ItemGroup>
</Project>

src/TestPrune.Trace.Recorder/Contract.fs:

/// Names the weaver emits calls to. The weaver resolves each one by name on the recorder
/// assembly it ships with, so a rename here is a compile error in the weaver, never a
/// runtime MissingMethodException in a consumer's test process.
module TestPrune.Trace.Recorder.Contract

[<Literal>]
let ProbesType = "TestPrune.Trace.Recorder.Probes"

[<Literal>]
let IoType = "TestPrune.Trace.Recorder.Io"

[<Literal>]
let ProcessShimsType = "TestPrune.Trace.Recorder.ProcessShims"

[<Literal>]
let Hit = "Hit"

[<Literal>]
let HitIfNotNull = "HitIfNotNull"

[<Literal>]
let HitIfTrue = "HitIfTrue"

[<Literal>]
let HitTag = "HitTag"

[<Literal>]
let EnterStatic = "EnterStatic"

[<Literal>]
let ExitStatic = "ExitStatic"

/// Environment the host sets on a traced test process.
[<Literal>]
let OutEnv = "TESTPRUNE_TRACE_OUT"

[<Literal>]
let IdsEnv = "TESTPRUNE_TRACE_IDS"

[<Literal>]
let RepoRootEnv = "TESTPRUNE_TRACE_REPO_ROOT"

[<Literal>]
let ParentScopeEnv = "TESTPRUNE_TRACE_PARENT_SCOPE"

[<Literal>]
let VerifyEnv = "TESTPRUNE_TRACE_VERIFY"

[<Literal>]
let VerifyAssembliesEnv = "TESTPRUNE_TRACE_VERIFY_ASSEMBLIES"

[<Literal>]
let DumpFormat = "testprune-trace/1"

src/TestPrune.Trace.Recorder/Probes.fs (stub; Task 2 replaces the bodies):

namespace TestPrune.Trace.Recorder

/// Static entry points the weaver calls. Stub until the recorder core lands.
[<AbstractClass; Sealed>]
type Probes =
    static member Hit(id: int) : unit = ignore id
    static member HitIfNotNull(value: obj, id: int) : unit = ignore (value, id)
    static member HitIfTrue(value: bool, id: int) : unit = ignore (value, id)
    static member HitTag(tag: int, baseId: int) : unit = ignore (tag, baseId)
    static member EnterStatic() : unit = ()
    static member ExitStatic() : unit = ()

/// The consumer-facing scope contract (see README). Inert when the recorder is inactive.
[<AbstractClass; Sealed>]
type Scopes =
    static member Enter(key: string) : unit = ignore key
    static member Exit() : unit = ()
    static member CurrentKey() : string = null
    static member LinkCurrentTo(scopeKey: string) : unit = ignore scopeKey

src/TestPrune.Trace/TestPrune.Trace.fsproj:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net10.0</TargetFramework>
        <NoWarn>$(NoWarn);NU1605;NU1608;MSB3277;NETSDK1188</NoWarn>
        <PackageId>TestPrune.Trace</PackageId>
        <Version>0.1.0</Version>
        <Authors>Michael Glass</Authors>
        <Description>Recorded per-test dependencies for TestPrune: IL weaver, shadow bin, trace store and symbol joiner.</Description>
        <PackageLicenseExpression>MIT</PackageLicenseExpression>
        <RepositoryUrl>https://github.com/michaelglass/TestPrune</RepositoryUrl>
        <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
        <GenerateDocumentationFile>true</GenerateDocumentationFile>
    </PropertyGroup>
    <ItemGroup>
        <AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
            <_Parameter1>TestPrune.Trace.Tests</_Parameter1>
        </AssemblyAttribute>
    </ItemGroup>
    <ItemGroup>
        <Compile Include="Model.fs" />
    </ItemGroup>
    <ItemGroup>
        <ProjectReference Include="../TestPrune.Core/TestPrune.Core.fsproj" />
        <ProjectReference Include="../TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj" />
    </ItemGroup>
    <ItemGroup>
        <PackageReference Include="Mono.Cecil" />
        <PackageReference Include="Microsoft.Data.Sqlite" />
    </ItemGroup>
</Project>

Directory.Packages.props: add <PackageVersion Include="Mono.Cecil" Version="0.11.6" /> to the ItemGroup.

tests/TestPrune.Trace.Tests/TestPrune.Trace.Tests.fsproj: copy tests/TestPrune.Tests/TestPrune.Tests.fsproj's first PropertyGroup verbatim (Exe, MTP runner, IsPackable=false, the same NoWarn), then:

    <ItemGroup>
        <Compile Include="Fixtures.fs" />
        <Compile Include="ScaffoldTests.fs" />
    </ItemGroup>
    <ItemGroup>
        <ProjectReference Include="../../src/TestPrune.Trace/TestPrune.Trace.fsproj" />
        <ProjectReference Include="../../src/TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj" />
        <!-- Build order only: the fixtures must be built before these tests read them. -->
        <ProjectReference Include="../TraceFixtures/src/FxDriver/FxDriver.fsproj" ReferenceOutputAssembly="false" />
        <ProjectReference Include="../TraceFixtures/tests/FxTests/FxTests.fsproj" ReferenceOutputAssembly="false" />
    </ItemGroup>
    <ItemGroup>
        <PackageReference Include="xunit.v3.mtp-v2" />
        <PackageReference Include="Unquote" />
        <PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" />
    </ItemGroup>

tests/TestPrune.Trace.Tests/Fixtures.fs:

module TestPrune.Trace.Tests.Fixtures

open System
open System.IO

/// The repository root: the first ancestor of the test binary holding TestPrune.slnx.
let repoRoot =
    let rec up (d: DirectoryInfo) =
        if isNull d then failwith "TestPrune.slnx not found above the test binary"
        elif File.Exists(Path.Combine(d.FullName, "TestPrune.slnx")) then d.FullName
        else up d.Parent

    up (DirectoryInfo AppContext.BaseDirectory)

/// The fixtures' miniature repository root (it has its own src/ and tests/).
let fixtureRoot = Path.Combine(repoRoot, "tests", "TraceFixtures")

let private built (sub: string) (project: string) =
    Path.Combine(fixtureRoot, sub, project, "bin", "Debug", "net10.0")

let fxLibDir = built "src" "FxLib"
let fxDriverDir = built "src" "FxDriver"
let fxTestsDir = built "tests" "FxTests"

Fixture projects:

• FxLib: the three F# files exactly as below. They cover the probe shapes: a 2-case union, a 5-case union, an all-nullary union, a struct union, a record, a class hierarchy, an interface, module values, a literal and an active pattern.

  tests/TraceFixtures/src/FxLib/FxLib.fsproj:

  <Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <TargetFramework>net10.0</TargetFramework> <IsPackable>false</IsPackable> <NoWarn>$(NoWarn);NU1605;NU1608</NoWarn> </PropertyGroup> <ItemGroup> <Compile Include="Types.fs" /> <Compile Include="Values.fs" /> <Compile Include="Logic.fs" /> </ItemGroup> </Project>

  Types.fs:

  namespace FxLib
  
  type Shape =
      | Circle of radius: float
      | Square of side: float
  
  type Color5 =
      | Red
      | Green of int
      | Blue of int
      | Cyan of string
      | Magenta
  
  type Dir =
      | North
      | East
      | South
      | West
  
  [<Struct>]
  type SResult =
      | SOk of okValue: int
      | SErr of errValue: string
  
  /// A case literally named `Tag` and a case field named `tag`: the two shapes that
  /// produced invalid IL in the prototype.
  type Tricky =
      | Tag
      | Other of tag: string
  
  type Point = { X: int; Y: int }
  
  type Animal() =
      abstract Speak: unit -> string
      default _.Speak() = "..."
  
  type Dog() =
      inherit Animal()
      override _.Speak() = "woof"
  
  type Cat() =
      inherit Animal()
      override _.Speak() = "meow"
  
  type IGreeter =
      abstract Greet: string -> string
  
  type Greeter() =
      interface IGreeter with
          member _.Greet n = "hi " + n
  

  Values.fs:

  module FxLib.Values
  
  let computeThreshold () = 40 + 2
  let threshold = computeThreshold ()
  let constValue = 7
  
  [<Literal>]
  let LiteralValue = 99
  
  let defaultShape = Square 3.0
  let names = [ "a"; "b" ]
  let unusedValue = computeThreshold () * 2
  let thresholdPlus x = threshold + x
  let prebuiltBlue = Blue 3
  let constPlus x = constValue + x
  

  Logic.fs:

  module FxLib.Logic
  
  open FxLib
  
  let area shape =
      match shape with
      | Circle r -> 3.0 * r * r
      | Square s -> s * s
  
  let isCircleOnly shape =
      match shape with
      | Circle _ -> true
      | _ -> false
  
  let colorCode c =
      match c with
      | Red -> 1
      | Green n -> 10 + n
      | Blue n -> 20 + n
      | Cyan s -> s.Length
      | Magenta -> 5
  
  let turn d =
      match d with
      | North -> East
      | East -> South
      | South -> West
      | West -> North
  
  let sval r =
      match r with
      | SOk v -> v
      | SErr e -> e.Length
  
  let tricky t =
      match t with
      | Tag -> 0
      | Other s -> s.Length
  
  let (|Big|Small|) (p: Point) = if p.X + p.Y > 10 then Big else Small
  
  let classify p =
      match p with
      | Big -> "big"
      | Small -> "small"
  
  let sumPoint (p: Point) = p.X + p.Y
  
  let describe (o: obj) =
      match o with
      | :? Dog as d -> "dog " + d.Speak()
      | :? Cat -> "cat"
      | _ -> "other"
  
  let castDog (o: obj) = (o :?> Dog).Speak()
  let aboveThreshold x = x > Values.threshold
  let addConst x = x + Values.constValue
  let addLiteral x = x + Values.LiteralValue
  let defaultArea () = area Values.defaultShape
  let isCircleProp (s: Shape) = s.IsCircle
  let samePoint (a: Point) (b: Point) = a = b
  let greet (g: IGreeter) = g.Greet "x"
  let readRepoFile (path: string) = System.IO.File.ReadAllText path
  

• FxDriver: a console app (OutputType=Exe, IsPackable=false) with ProjectReferences to ../FxLib/FxLib.fsproj and ../../../../src/TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj. Its Program.fs runs each scenario inside Scopes.Enter name / Scopes.Exit (), with pre-built values created in a "warm" scope, the same design as the prototype driver.

  module FxDriver.Program
  
  open FxLib
  open TestPrune.Trace.Recorder
  
  let scope name (f: unit -> unit) =
      Scopes.Enter("T:" + name)
  
      try
          f ()
      finally
          Scopes.Exit()
  
  [<EntryPoint>]
  let main _ =
      let circle = ref Unchecked.defaultof<Shape>
      let circle2 = ref Unchecked.defaultof<Shape>
      let square = ref Unchecked.defaultof<Shape>
      let blue = ref Unchecked.defaultof<Color5>
      let east = ref Unchecked.defaultof<Dir>
      let err = ref Unchecked.defaultof<SResult>
      let dog = ref Unchecked.defaultof<Dog>
      let cat = ref Unchecked.defaultof<Cat>
      let pt = ref Unchecked.defaultof<Point>
      let tagCase = ref Unchecked.defaultof<Tricky>
  
      scope "warm" (fun () ->
          circle.Value <- Circle 2.0
          circle2.Value <- Circle 2.0
          square.Value <- Square 1.0
          blue.Value <- Blue 3
          east.Value <- East
          err.Value <- SErr "e"
          dog.Value <- Dog()
          cat.Value <- Cat()
          pt.Value <- { X = 1; Y = 2 }
          tagCase.Value <- Tag
          Values.threshold |> ignore)
  
      let sink = System.Collections.Generic.List<obj>()
      let run name (f: unit -> obj) = scope name (fun () -> sink.Add(f ()))
      run "caseA_area" (fun () -> box (Logic.area circle.Value))
      run "caseB_isCircleOnly" (fun () -> box (Logic.isCircleOnly square.Value))
      run "static_case_match" (fun () -> box (Logic.defaultArea ()))
      run "tag5" (fun () -> box (Logic.colorCode blue.Value))
      run "tag5_static" (fun () -> box (Logic.colorCode Values.prebuiltBlue))
      run "nullary" (fun () -> box (Logic.turn east.Value))
      run "struct" (fun () -> box (Logic.sval err.Value))
      run "tricky_tag" (fun () -> box (Logic.tricky tagCase.Value))
      run "modval" (fun () -> box (Logic.aboveThreshold 50))
      run "sameFileValue" (fun () -> box (Values.thresholdPlus 1))
      run "typetest_dog" (fun () -> box (Logic.describe (box dog.Value)))
      run "typetest_cat" (fun () -> box (Logic.describe (box cat.Value)))
      run "unboxgeneric" (fun () -> box (Logic.castDog (box dog.Value)))
      run "record" (fun () -> box (Logic.sumPoint pt.Value))
      run "testcode_match" (fun () -> box (match circle.Value with Circle r -> r | Square s -> -s))
      run "testcode_typetest" (fun () -> box (match box cat.Value with :? Dog -> 1 | _ -> 2))
      run "constValue" (fun () -> box (Logic.addConst 1))
      run "literal" (fun () -> box (Logic.addLiteral 1))
      run "isCaseProp" (fun () -> box (Logic.isCircleProp square.Value))
      run "activePattern" (fun () -> box (Logic.classify pt.Value))
      run "recordEquality" (fun () -> box (Logic.samePoint pt.Value pt.Value))
      run "unionEquality" (fun () -> box (circle.Value = circle2.Value))
      run "fileRead" (fun () -> box (Logic.readRepoFile (System.IO.Path.Combine(System.Environment.GetEnvironmentVariable "TESTPRUNE_TRACE_REPO_ROOT", "global.json"))))
      0
  

• FxTests: an xUnit v3 project (xunit.v3.mtp-v2, UseMicrosoftTestingPlatformRunner, IsPackable=false) referencing ../../src/FxLib/FxLib.fsproj only (not the recorder). AttributionTests.fs:

  module FxTests.AttributionTests
  
  open System
  open System.Diagnostics
  open System.Threading.Tasks
  open Xunit
  open FxLib
  
  type SharedFixture() =
      member val Seed = Logic.sumPoint { X = 1; Y = 1 }
  
  type ClassA(fixture: SharedFixture) =
      interface IClassFixture<SharedFixture>
  
      [<Fact>]
      member _.``a sync area`` () = Assert.Equal(12.0, Logic.area (Circle 2.0))
  
      [<Fact>]
      member _.``a async task turn`` () =
          task {
              do! Task.Yield()
              Assert.Equal(South, Logic.turn East)
          }
  
      [<Fact>]
      member _.``a Task.Run describe`` () =
          task {
              let! s = Task.Run(fun () -> Logic.describe (box (Dog())))
              Assert.Equal("dog woof", s)
          }
  
  type ClassB() =
      [<Fact>]
      member _.``b async block colorCode`` () =
          async {
              do! Async.Sleep 1
              Assert.Equal(23, Logic.colorCode (Blue 3))
          }
          |> Async.StartAsTask
  
      [<Fact>]
      member _.``b Async.Parallel sval`` () =
          let r =
              [ async { return Logic.sval (SOk 1) }; async { return Logic.sval (SErr "ab") } ]
              |> Async.Parallel
              |> Async.RunSynchronously
  
          Assert.Equal<int[]>([| 1; 2 |], r)
  
      [<Theory>]
      [<InlineData(1)>]
      [<InlineData(20)>]
      member _.``b theory aboveThreshold`` (x: int) = Assert.Equal(x > 42, Logic.aboveThreshold x)
  
  type ClassC() =
      [<Fact>]
      member _.``c reads a repo file`` () =
          let root = Environment.GetEnvironmentVariable "TESTPRUNE_TRACE_REPO_ROOT"
          if not (String.IsNullOrEmpty root) then
              Assert.NotEmpty(Logic.readRepoFile (IO.Path.Combine(root, "global.json")))
  
      [<Fact>]
      member _.``c starts a child process`` () =
          let psi = ProcessStartInfo("/bin/echo", "hi", UseShellExecute = false, RedirectStandardOutput = true)
          use p = Process.Start psi
          p.WaitForExit()
          Assert.Equal(0, p.ExitCode)
  

TestPrune.slnx: under /tests/ add tests/TestPrune.Trace.Tests/TestPrune.Trace.Tests.fsproj; under /src/ add both new src projects. Add a new folder:

  <Folder Name="/tests/TraceFixtures/">
    <Project Path="tests/TraceFixtures/src/FxLib/FxLib.fsproj" />
    <Project Path="tests/TraceFixtures/src/FxDriver/FxDriver.fsproj" />
    <Project Path="tests/TraceFixtures/tests/FxTests/FxTests.fsproj" />
  </Folder>

semantic-tagger.json: add

{ "name": "TestPrune.Trace.Recorder", "fsproj": "src/TestPrune.Trace.Recorder/TestPrune.Trace.Recorder.fsproj", "tagPrefix": "trace-v", "fsProjsSharingSameTag": ["src/TestPrune.Trace/TestPrune.Trace.fsproj"] }

One tag covers both packages: they version in lockstep because the weaver resolves recorder methods by name.

.fshw.json: add a second entry to tests.projects:

{ "project": "TestPrune.Trace.Tests", "command": "dotnet", "args": "run --project tests/TestPrune.Trace.Tests --no-build --", "filterTemplate": "--filter-class {classes}", "classJoin": " ", "group": "trace" }

• [ ] Step 4: Run the tests to verify they pass

Run: dotnet build && dotnet run --project tests/TestPrune.Trace.Tests --no-build -- --filter-class TestPrune.Trace.Tests.ScaffoldTests Expected: PASS, 3 tests.

• [ ] Step 5: Gate and commit

Run mise run ci > /tmp/t1-gate.log 2>&1; echo "exit=$?", then read the log and the verdict. Commit:

jj commit -m "Scaffold TestPrune.Trace and its recorder, with F# probe-shape fixtures

Two new packages that version together under one trace-v tag: the recorder is
loaded into a test process and pins FSharp.Core 8.0.403 so it never raises the
host's floor; the tooling package will hold the weaver, shadow bin, store and
joiner. Probe names live in Recorder.Contract so the weaver cannot drift from
the recorder.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>"

Task 2: Recorder core, dump writer and dump reader

Files:

• Create: src/TestPrune.Trace.Recorder/Scope.fs, ContextSource.fs, RecorderState.fs, DumpWriter.fs
• Modify: src/TestPrune.Trace.Recorder/Probes.fs (real bodies + Runtime bootstrap), TestPrune.Trace.Recorder.fsproj (compile order; InternalsVisibleTo TestPrune.Trace.Tests)
• Create: src/TestPrune.Trace/DumpReader.fs (add to TestPrune.Trace.fsproj after Model.fs)
• Test: tests/TestPrune.Trace.Tests/RecorderStateTests.fs, XunitContextTests.fs, DumpRoundTripTests.fs

Interfaces:

• Consumes: Contract literals and the Probes/Scopes signatures (Task 1); Model.ProcessDump and friends (Task 1).

• Produces (recorder, internal, visible to the tests):

  • RecorderState(idCount: int, source: IContextSource option, parentScope: string) with members Hit(id), EnterStatic(), ExitStatic(), EnterScope(key), ExitScope(), CurrentScope(): Scope (null when unattributed), Scopes: seq<Scope>, Counters(): int64[], and the constants IdCount and ParentScope.

  • Scope with Key, Set(id), Ids(): int[], Links, Inputs, Children, and the test identity fields.

  • IContextSource, ContextInfo, XunitContextSource.tryCreate(): IContextSource option.

  • DumpWriter.write (path: string) (state: RecorderState) : unit.

  • Runtime.state : RecorderState (null when TESTPRUNE_TRACE_OUT is unset).

• Produces (tooling): DumpReader.readFile (path: string) : Result<Model.ProcessDump, string> and DumpReader.readDirectory (dir: string) : Model.ProcessDump list * (string * string) list, which returns the good dumps plus a (file, reason) for each rejected one.

• [ ] Step 1: Write the failing tests

tests/TestPrune.Trace.Tests/RecorderStateTests.fs:

module TestPrune.Trace.Tests.RecorderStateTests

open System.Threading
open System.Threading.Tasks
open Xunit
open Swensen.Unquote
open TestPrune.Trace.Recorder

/// A context source whose "current test" is an AsyncLocal the test sets, exactly
/// like xUnit's TestContext.Current.
type FakeSource() =
    let current = AsyncLocal<obj>()
    member _.Set(ctx: obj) = current.Value <- ctx

    interface IContextSource with
        member _.Current() = current.Value

        member _.Describe(ctx) =
            let name = string ctx

            { Key = "T:" + name
              TestClass = "Ns.C"
              TestMethod = name
              TestDisplay = "Ns.C." + name
              Parents = [| "C:Ns.C"; "A:assembly" |] }

let private idsOf (state: RecorderState) key =
    state.Scopes |> Seq.find (fun s -> s.Key = key) |> fun s -> s.Ids() |> Set.ofArray

[<Fact>]
let ``hits go to the current test, and parallel tests never contaminate each other`` () =
    let src = FakeSource()
    let state = RecorderState(1000, Some(src :> IContextSource), null)

    let work (name: string) (ids: int list) =
        Task.Run(fun () ->
            src.Set(box name)

            for _ in 1..200 do
                for id in ids do
                    state.Hit id)

    Task.WaitAll([| work "a" [ 1; 2; 3 ]; work "b" [ 3; 4 ]; work "c" [ 999 ] |])
    test <@ idsOf state "T:a" = set [ 1; 2; 3 ] @>
    test <@ idsOf state "T:b" = set [ 3; 4 ] @>
    test <@ idsOf state "T:c" = set [ 999 ] @>

[<Fact>]
let ``attribution flows through awaits and Task.Run`` () =
    let src = FakeSource()
    let state = RecorderState(64, Some(src :> IContextSource), null)

    let t =
        task {
            src.Set(box "flow")
            do! Task.Yield()
            let! () = Task.Run(fun () -> state.Hit 7)
            state.Hit 8
        }

    t.Wait()
    test <@ idsOf state "T:flow" = set [ 7; 8 ] @>

[<Fact>]
let ``static init wins over the test, an explicit scope wins over the test`` () =
    let src = FakeSource()
    let state = RecorderState(64, Some(src :> IContextSource), null)
    src.Set(box "t")
    state.EnterStatic()
    state.Hit 1
    state.ExitStatic()
    state.EnterScope "P:pool"
    state.Hit 2
    state.ExitScope()
    state.Hit 3
    test <@ idsOf state "S:static-init" = set [ 1 ] @>
    test <@ idsOf state "P:pool" = set [ 2 ] @>
    test <@ idsOf state "T:t" = set [ 3 ] @>

[<Fact>]
let ``no context means ambient, or the parent scope in a child process`` () =
    let lone = RecorderState(64, None, null)
    lone.Hit 5
    test <@ idsOf lone "A:ambient" = set [ 5 ] @>
    let child = RecorderState(64, None, "T:parent")
    child.Hit 6
    test <@ idsOf child "T:parent" = set [ 6 ] @>

[<Fact>]
let ``an id outside the manifest is counted as overflow and never indexes out of range`` () =
    let state = RecorderState(10, None, null)
    // Counters are per THREAD and summed process-wide, so compare against a snapshot.
    let before = state.Counters()
    state.Hit 10
    state.Hit -1
    test <@ state.Counters().[7] - before.[7] = 2L @>

[<Fact>]
let ``counters split hits by bucket`` () =
    let src = FakeSource()
    let state = RecorderState(64, Some(src :> IContextSource), null)
    let before = state.Counters()
    state.Hit 1 // ambient
    src.Set(box "t")
    state.Hit 1 // test
    state.EnterStatic()
    state.Hit 1 // static
    state.ExitStatic()
    let c = Array.map2 (-) (state.Counters()) before
    test <@ (c.[0], c.[5], c.[6]) = (1L, 1L, 1L) @>

tests/TestPrune.Trace.Tests/XunitContextTests.fs. This test runs under real xUnit v3, so the source binds the real TestContext:

module TestPrune.Trace.Tests.XunitContextTests

open Xunit
open Swensen.Unquote
open TestPrune.Trace.Recorder

[<Fact>]
let ``the xUnit source names the running test and its parents`` () =
    let src = (XunitContextSource.tryCreate ()).Value
    let info = src.Describe(src.Current())
    test <@ info.Key.StartsWith "T:" @>
    test <@ info.TestMethod = "the xUnit source names the running test and its parents" @>
    test <@ info.TestClass = "TestPrune.Trace.Tests.XunitContextTests" @>
    test <@ info.Parents |> Array.contains ("C:" + info.TestClass) @>
    test <@ info.Parents |> Array.last = "A:assembly" @>

tests/TestPrune.Trace.Tests/DumpRoundTripTests.fs:

module TestPrune.Trace.Tests.DumpRoundTripTests

open System.IO
open Xunit
open Swensen.Unquote
open TestPrune.Trace.Recorder
open TestPrune.Trace
open TestPrune.Trace.Model

[<Fact>]
let ``a dump reads back exactly what the recorder held`` () =
    let dir = Directory.CreateTempSubdirectory().FullName
    let state = RecorderState(128, None, null)
    state.EnterScope "T:x"
    state.Hit 3
    state.Hit 70
    state.CurrentScope().Links.TryAdd("P:pool", 0uy) |> ignore
    state.ExitScope()
    let path = Path.Combine(dir, "trace-1.ndjson")
    DumpWriter.write path state
    let dump = DumpReader.readFile path |> Result.defaultWith failwith
    let x = dump.Scopes |> List.find (fun s -> s.Key = "T:x")
    test <@ x.Ids = [| 3; 70 |] @>
    test <@ x.Links = [ "P:pool" ] @>
    test <@ dump.IdCount = 128 @>

[<Fact>]
let ``a dump without its end marker is rejected`` () =
    let dir = Directory.CreateTempSubdirectory().FullName
    let path = Path.Combine(dir, "trace-2.ndjson")
    File.WriteAllLines(path, [| """{"format":"testprune-trace/1","pid":2,"parentScope":null,"runtime":"r","os":"o","arch":"a","ids":1,"cpuMs":0,"counters":{"test":0,"class":0,"collection":0,"assembly":0,"override":0,"staticInit":0,"ambient":0,"overflow":0}}""" |])
    test <@ DumpReader.readFile path = Error "truncated" @>

[<Fact>]
let ``readDirectory ignores tmp files and reports rejected dumps by name`` () =
    let dir = Directory.CreateTempSubdirectory().FullName
    File.WriteAllText(Path.Combine(dir, "trace-3.ndjson.tmp"), "partial")
    File.WriteAllText(Path.Combine(dir, "trace-4.ndjson"), "not json")
    let good, bad = DumpReader.readDirectory dir
    test <@ good = [] @>
    test <@ bad |> List.map fst = [ Path.Combine(dir, "trace-4.ndjson") ] @>

• [ ] Step 2: Run the tests to verify they fail

Run: dotnet build Expected: FAIL. RecorderState, XunitContextSource, DumpWriter and DumpReader are undefined.

• [ ] Step 3: Implement the recorder

Scope.fs:

namespace TestPrune.Trace.Recorder

open System.Collections.Concurrent
open System.Threading

/// One attribution bucket: a bitset of probe ids plus what the host needs to name,
/// link and join it. One word per 64 ids; set with a racy read then an atomic OR, so
/// the common "already set" case is a plain load.
[<Sealed; AllowNullLiteral>]
type Scope(key: string, idCount: int) =
    let bits: uint64[] = Array.zeroCreate ((idCount + 63) >>> 6)
    member _.Key = key
    member val TestClass: string = null with get, set
    member val TestMethod: string = null with get, set
    member val TestDisplay: string = null with get, set
    member val Parents: string[] = [||] with get, set
    member val Links = ConcurrentDictionary<string, byte>()
    member val Inputs = ConcurrentDictionary<struct (string * string), byte>()
    member val Children = ConcurrentQueue<struct (int * string * bool)>()

    member _.Set(id: int) =
        let w = id >>> 6
        let m = 1UL <<< (id &&& 63)

        if (Volatile.Read(&bits.[w]) &&& m) = 0UL then
            Interlocked.Or(&bits.[w], m) |> ignore

    member _.Ids() : int[] =
        let out = ResizeArray()

        for w in 0 .. bits.Length - 1 do
            let mutable word = bits.[w]

            while word <> 0UL do
                let b = System.Numerics.BitOperations.TrailingZeroCount word
                out.Add((w <<< 6) + b)
                word <- word &&& (word - 1UL)

        out.ToArray()

    member this.IsEmpty =
        bits |> Array.forall ((=) 0UL)
        && this.Links.IsEmpty
        && this.Inputs.IsEmpty
        && this.Children.IsEmpty

ContextSource.fs:

namespace TestPrune.Trace.Recorder

open System
open System.Collections.Concurrent
open System.Reflection

/// What the recorder needs to know about one attribution context.
type ContextInfo =
    { Key: string
      TestClass: string
      TestMethod: string
      TestDisplay: string
      Parents: string[] }

/// Where "the current test" comes from. Production binds xUnit v3 by reflection
/// (no compile-time dependency); tests substitute a fake.
type IContextSource =
    /// The current context object, compared by reference for caching; null when none.
    abstract Current: unit -> obj
    abstract Describe: context: obj -> ContextInfo

module XunitContextSource =
    let private props = ConcurrentDictionary<struct (Type * string), PropertyInfo>()

    /// Read `name` from `o` through its type or any interface it implements. xUnit's
    /// concrete context types are internal; their interfaces are the public surface.
    let private read (o: obj) (name: string) : obj =
        if isNull o then
            null
        else
            let t = o.GetType()

            let p =
                props.GetOrAdd(
                    struct (t, name),
                    fun _ ->
                        Seq.append [ t ] (t.GetInterfaces())
                        |> Seq.tryPick (fun i -> i.GetProperty name |> Option.ofObj)
                        |> Option.toObj
                )

            if isNull p then null else p.GetValue o

    let private str (o: obj) = if isNull o then null else string o

    type private Source(current: Func<obj>) =
        interface IContextSource with
            member _.Current() = current.Invoke()

            member _.Describe(ctx) =
                let testObj = read ctx "Test"
                let cls = read ctx "TestClass"
                let meth = read ctx "TestMethod"
                let coll = read ctx "TestCollection"
                let className = str (read cls "TestClassName")
                let collName = str (read coll "TestCollectionDisplayName")

                let collParent =
                    if isNull collName then [||] else [| "L:" + collName |]

                if not (isNull testObj) then
                    { Key = "T:" + str (read testObj "UniqueID")
                      TestClass = className
                      TestMethod = str (read meth "MethodName")
                      TestDisplay = str (read testObj "TestDisplayName")
                      Parents = Array.concat [ [| "C:" + className |]; collParent; [| "A:assembly" |] ] }
                elif not (isNull className) then
                    { Key = "C:" + className
                      TestClass = className
                      TestMethod = null
                      TestDisplay = null
                      Parents = Array.append collParent [| "A:assembly" |] }
                elif not (isNull collName) then
                    { Key = "L:" + collName
                      TestClass = null
                      TestMethod = null
                      TestDisplay = null
                      Parents = [| "A:assembly" |] }
                else
                    { Key = "A:assembly"
                      TestClass = null
                      TestMethod = null
                      TestDisplay = null
                      Parents = [||] }

    /// Bind `Xunit.TestContext.Current` once. None when xUnit v3 is not loaded.
    let tryCreate () : IContextSource option =
        match Type.GetType("Xunit.TestContext, xunit.v3.core", false) with
        | null -> None
        | t ->
            match t.GetProperty("Current", BindingFlags.Public ||| BindingFlags.Static) with
            | null -> None
            | p ->
                // A bound delegate, not MethodInfo.Invoke: this runs on every probe hit. Valid
                // because the getter returns a reference type and delegate returns are covariant.
                let f = Delegate.CreateDelegate(typeof<Func<obj>>, p.GetGetMethod()) :?> Func<obj>
                Some(Source f :> IContextSource)

RecorderState.fs:

namespace TestPrune.Trace.Recorder

open System
open System.Collections.Concurrent
open System.Runtime.CompilerServices
open System.Threading

/// Per-thread hot-path cache. One instance per thread, registered so the exit dump
/// can sum its counters.
[<Sealed; AllowNullLiteral>]
type internal ThreadState() =
    member val StaticDepth = 0 with get, set
    member val LastCtx: obj = null with get, set
    member val LastScope: Scope = null with get, set
    member val LastOwner: obj = null with get, set
    /// test, class, collection, assembly, override, staticInit, ambient, overflow
    member val Counts: int64[] = Array.zeroCreate 8

[<AbstractClass; Sealed>]
type internal Threads =
    [<ThreadStatic; DefaultValue>]
    static val mutable private current: ThreadState

    static member val All = ConcurrentBag<ThreadState>()

    static member Get() =
        let t = Threads.current

        if isNull t then
            let fresh = ThreadState()
            Threads.current <- fresh
            Threads.All.Add fresh
            fresh
        else
            t

[<Sealed; AllowNullLiteral>]
type RecorderState(idCount: int, source: IContextSource option, parentScope: string) =
    let byKey = ConcurrentDictionary<string, Scope>()
    let byCtx = ConditionalWeakTable<obj, Scope>()
    let newScope key = byKey.GetOrAdd(key, fun k -> Scope(k, idCount))

    let ambient =
        newScope (if String.IsNullOrEmpty parentScope then "A:ambient" else parentScope)

    let ambientBucket = if String.IsNullOrEmpty parentScope then 6 else 4
    let staticInit = newScope "S:static-init"
    let overrideScope = AsyncLocal<Scope>()
    let mutable anyOverride = false

    let bucketOf (s: Scope) =
        match s.Key.[0] with
        | 'T' -> 0
        | 'C' -> 1
        | 'L' -> 2
        | _ -> 3

    let resolve (src: IContextSource) (ctx: obj) =
        match byCtx.TryGetValue ctx with
        | true, s -> s
        | _ ->
            let info = src.Describe ctx
            let s = newScope info.Key

            if isNull s.TestClass then
                s.TestClass <- info.TestClass
                s.TestMethod <- info.TestMethod
                s.TestDisplay <- info.TestDisplay
                s.Parents <- info.Parents

            byCtx.AddOrUpdate(ctx, s)
            s

    member _.IdCount = idCount
    member _.ParentScope = parentScope
    member _.Scopes: seq<Scope> = byKey.Values :> _

    /// The scope a hit would go to now (ignoring static init); null when unattributed.
    member this.CurrentScope() : Scope =
        let o = if anyOverride then overrideScope.Value else null

        if not (isNull o) then
            o
        else
            match source with
            | Some src ->
                match src.Current() with
                | null -> if ambientBucket = 4 then ambient else null
                | ctx -> resolve src ctx
            | None -> if ambientBucket = 4 then ambient else null

    member _.EnterScope(key: string) =
        anyOverride <- true
        overrideScope.Value <- newScope key

    member _.ExitScope() = overrideScope.Value <- null
    member _.EnterStatic() = let t = Threads.Get() in t.StaticDepth <- t.StaticDepth + 1
    member _.ExitStatic() = let t = Threads.Get() in t.StaticDepth <- t.StaticDepth - 1

    [<MethodImpl(MethodImplOptions.AggressiveInlining)>]
    member this.Hit(id: int) =
        let t = Threads.Get()

        if uint32 id >= uint32 idCount then
            t.Counts.[7] <- t.Counts.[7] + 1L
        elif t.StaticDepth > 0 then
            staticInit.Set id
            t.Counts.[5] <- t.Counts.[5] + 1L
        else
            let o = if anyOverride then overrideScope.Value else null

            if not (isNull o) then
                o.Set id
                t.Counts.[4] <- t.Counts.[4] + 1L
            else
                match source with
                | None ->
                    ambient.Set id
                    t.Counts.[ambientBucket] <- t.Counts.[ambientBucket] + 1L
                | Some src ->
                    let ctx = src.Current()

                    if isNull ctx then
                        ambient.Set id
                        t.Counts.[ambientBucket] <- t.Counts.[ambientBucket] + 1L
                    else
                        let s =
                            if Object.ReferenceEquals(ctx, t.LastCtx) && Object.ReferenceEquals(this, t.LastOwner) then
                                t.LastScope
                            else
                                let r = resolve src ctx
                                t.LastCtx <- ctx
                                t.LastScope <- r
                                t.LastOwner <- this
                                r

                        s.Set id
                        let b = bucketOf s
                        t.Counts.[b] <- t.Counts.[b] + 1L

    /// Sum of every thread's counters (see ThreadState.Counts for the bucket order).
    member _.Counters() : int64[] =
        let sum = Array.zeroCreate 8

        for t in Threads.All do
            for i in 0..7 do
                sum.[i] <- sum.[i] + t.Counts.[i]

        sum

Threads.All is process-wide: counters are per thread and summed over every thread, so two RecorderState instances in one test process share them. A real traced process has exactly one RecorderState, so production counters are exact. It is a [<ThreadStatic>] rather than a per-instance ThreadLocal<T> because the probe cost budget is a few nanoseconds. In tests, the counter assertions compare against a snapshot, and the whole RecorderStateTests module runs outside parallelization so that no other class's hits land between the snapshot and the assertion. Add to RecorderStateTests.fs:

[<CollectionDefinition("recorder-counters", DisableParallelization = true)>]
type RecorderCountersCollection() = class end

and put [<Collection("recorder-counters")>] on the module (F# modules compile to static classes, and xUnit reads the attribute from the class).

DumpWriter.fs:

module TestPrune.Trace.Recorder.DumpWriter

open System
open System.Diagnostics
open System.IO
open System.Runtime.InteropServices
open System.Text.Json

let private osName () =
    if RuntimeInformation.IsOSPlatform OSPlatform.OSX then "OSX"
    elif RuntimeInformation.IsOSPlatform OSPlatform.Linux then "Linux"
    elif RuntimeInformation.IsOSPlatform OSPlatform.Windows then "Windows"
    else "Other"

/// Write every non-empty scope (every T scope, even an empty one) to `path` via a
/// `.tmp` sibling and a rename, so a reader never sees a half-written file.
let write (path: string) (state: RecorderState) =
    let tmp = path + ".tmp"

    do
        use stream = File.Create tmp
        use w = new Utf8JsonWriter(stream)
        let line () =
            w.Flush()
            stream.WriteByte(byte '\n')
            w.Reset()

        let c = state.Counters()
        w.WriteStartObject()
        w.WriteString("format", Contract.DumpFormat)
        w.WriteNumber("pid", Environment.ProcessId)

        if String.IsNullOrEmpty state.ParentScope then
            w.WriteNull "parentScope"
        else
            w.WriteString("parentScope", state.ParentScope)

        w.WriteString("runtime", RuntimeInformation.FrameworkDescription)
        w.WriteString("os", osName ())
        w.WriteString("arch", string RuntimeInformation.ProcessArchitecture)
        w.WriteNumber("ids", state.IdCount)
        w.WriteNumber("cpuMs", int64 (Process.GetCurrentProcess().TotalProcessorTime.TotalMilliseconds))
        w.WriteStartObject "counters"

        for i, name in
            [ 0, "test"; 1, "class"; 2, "collection"; 3, "assembly"
              4, "override"; 5, "staticInit"; 6, "ambient"; 7, "overflow" ] do
            w.WriteNumber(name, c.[i])

        w.WriteEndObject()
        w.WriteEndObject()
        line ()

        for s in state.Scopes do
            if s.Key.StartsWith "T:" || not s.IsEmpty then
                w.WriteStartObject()
                w.WriteString("key", s.Key)

                if isNull s.TestMethod then
                    w.WriteNull "test"
                else
                    w.WriteStartObject "test"
                    w.WriteString("class", s.TestClass)
                    w.WriteString("method", s.TestMethod)
                    w.WriteString("display", s.TestDisplay)
                    w.WriteEndObject()

                w.WriteStartArray "parents"
                for p in s.Parents do w.WriteStringValue p
                w.WriteEndArray()
                w.WriteStartArray "links"
                for l in s.Links.Keys |> Seq.sort do w.WriteStringValue l
                w.WriteEndArray()
                w.WriteStartArray "ids"
                for id in s.Ids() do w.WriteNumberValue id
                w.WriteEndArray()
                w.WriteStartArray "inputs"

                for KeyValue(struct (kind, p), _) in s.Inputs |> Seq.sortBy (fun kv -> kv.Key) do
                    w.WriteStartObject()
                    w.WriteString("kind", kind)
                    w.WriteString("path", p)
                    w.WriteEndObject()

                w.WriteEndArray()
                w.WriteStartArray "children"

                for struct (pid, file, env) in s.Children do
                    w.WriteStartObject()
                    w.WriteNumber("pid", pid)
                    w.WriteString("file", file)
                    w.WriteBoolean("env", env)
                    w.WriteEndObject()

                w.WriteEndArray()
                w.WriteEndObject()
                line ()

        w.WriteStartObject()
        w.WriteBoolean("end", true)
        w.WriteEndObject()
        line ()

    File.Move(tmp, path, true)

/// The file name for this process inside the host-provided output directory.
let pathIn (dir: string) =
    Path.Combine(dir, $"trace-%d{Environment.ProcessId}.ndjson")