Fixture Recording & Replay

Fixture Recording & Replay

The fixture system makes flow tests deterministic by recording every external call during a baseline execution and replaying those recordings during test runs.

How Recording Works

When a play runs with recordFixture: true, each component call is intercepted and saved:

1. The HTTP request module makes a call — request + response saved to play_component_events
2. The AI module makes a call — prompt + response saved
3. A file is read from storage — path + content saved
4. A database query runs — query + result saved

All events are saved with a sequence number (seq) to ensure replay order.

Fixture Document Structure

{
  "id": "65fixture1",
  "flowId": "65flow1",
  "baselinePlayId": "65play-baseline",
  "events": [
    {
      "seq": 1,
      "playLayerId": "65pl1",
      "componentKind": "request",
      "operation": "http.get",
      "input": {
        "method": "GET",
        "url": "https://api.example.com/invoices/INV-TEST-001",
        "headers": {"Authorization": "Bearer [REDACTED]"}
      },
      "output": {
        "statusCode": 200,
        "body": {"id": "INV-TEST-001", "amount": 1250.00}
      }
    },
    {
      "seq": 2,
      "playLayerId": "65pl2",
      "componentKind": "ai",
      "operation": "chat.completion",
      "input": {
        "messages": [{"role": "user", "content": "Extract invoice data from: ..."}]
      },
      "output": {
        "content": "{\"invoiceNumber\":\"INV-TEST-001\",\"amount\":1250.00}"
      }
    }
  ]
}

How Replay Works

During a test run with a fixture:

1. The test fixture is loaded and indexed by (componentKind, operation, seq)
2. When a flowlet’s component would normally make an external call, the test component intercepts it
3. Instead of making the real call, it returns the recorded output from the fixture
4. The flow continues as if the real call succeeded

No network calls, no AI API costs, no side effects.

Updating a Fixture

If the flow’s behavior intentionally changes (new step, different API response shape), update the fixture:

# Re-run baseline with real external calls
POST /api/flows/65flow1/run
{"args": {"invoiceId": "INV-TEST-001"}, "recordFixture": true}

# Promote new play as baseline
POST /api/plays/65new-play/promote-baseline
{"testCaseId": "65case1"}

The old fixture is replaced. Commit the updated expected outputs.

What Gets Recorded

Component	Operations Recorded
request	All HTTP requests (method, URL, headers, body, response)
ai	All chat/embedding/extraction calls (prompt, response)
storage	File reads and writes (path, content)
db	All queries and mutations (SQL/query, results)
console	Log output (for assertion)

Sensitive data (Authorization headers, API keys) is redacted in the fixture before saving.