In this note
Your integration fetched two orders. Both look ready for dispatch. The request returned 200, the JSON parsed, and the job turned green. Meanwhile, a third order is on hold. It was on the next page, which the integration never requested.
A model reading the extract has no reason to mention that order. A rule checking hold === false would miss it too. The problem starts before either one sees the data.
Which orders should be here?
A coordinator needs the active west-region orders at 09:00 UTC, excluding archived orders. In this exercise, the source owner supplies a fixed snapshot and a separate control report showing which three orders should be there, with their versions and hold values. We can check the extract against that report. A real source may not offer that guarantee.
| Record | Expected place in the extract |
|---|---|
| O-101 · revision 1 · hold false | Page 1; included in the review population |
| O-102 · revision 1 · hold false | Page 1; included in the review population |
| O-103 · revision 1 · hold true | Page 2; included even though dispatch is on hold |
| O-900 · archived | Excluded by the declared population rule |
Include held orders in this check. Dropping them before comparing the lists would remove the cases the coordinator most needs to see. Agree any exclusions before fetching the data, and keep the reason for each one.
Start by writing which orders belong in the list. Ask the source owner to confirm the account, filters, identifiers, and point in time that produce it. Counting the extract itself won’t find a missing page: it will correctly count the two rows you already have.
Keep response checks and population checks separate
The offline companion uses JavaScript objects marked status: 200 rather than sending HTTP requests. It checks that every page belongs to the expected source, snapshot, and account, then compares order IDs and fields with its answer key. A failed check returns no accepted partial list. Nothing is dispatched or changed outside the exercise.
Remove page 2 and page 1 still reports 200. The checker catches the missing page and O-103. A real connector should retain the cursor or continuation details showing where it stopped. Report an expired cursor or lost permission as a fetch problem, rather than pretending the remaining records don’t exist.
Counts help, but three records can still be the wrong three. The second negative test replaces O-103 with O-999. The unique count stays at three. Identity reconciliation rejects the substitute and identifies the missing order. The duplicate test adds O-101 again; an extra copy cannot compensate for a missing identity.
The right order ID can still carry the wrong version. Check whether its hold value belongs to the agreed snapshot, rather than an earlier or later state. The companion compares fixed test records; a live service still needs evidence that its source report is correct, complete, current enough, and permitted for the requester.
An older amendment must not clear a newer hold
After the snapshot, O-103 changes twice. At 09:05, revision 2 clears the hold. At 09:10, revision 3 restores it. A consumer that overwrites its row with whichever message arrived last can receive revision 3 and then revision 2, ending with hold false. The last arrival contains the older business state.
The test source numbers each order’s revisions consecutively and never backdates them. If revision 3 arrives while we still have revision 1, the updater returns UNKNOWN and preserves the old row: revision 2 is missing. Apply 2 and then 3, and a later repeat of 2 returns STALE. It cannot clear the newer hold.
revision 3 before revision 2: UNKNOWN (gap; reconcile)
revision 2 after revision 3: STALE (hold remains true)An exact repeat of the current revision is harmless here. Different values under the same revision are rejected. The code returns a copy without changing its input; it has no durable event log, transaction handling, concurrent writers, recovery, or live replay queue.
Check your source before copying that rule. Some systems allow backdated corrections or use versions that aren’t consecutive numbers. Ask how updates, late messages, deletions, and restored records work. The fixture excludes deletion; a connector that supports it needs separate tests for removing records and checking the resulting list.
Rejecting revision 2 after revision 3 still tells us nothing about an unseen revision 4. Before using the list, establish how far source updates have been received and how old the data may be. Software making a dispatch decision must check current order state and permission again. This review extract cannot authorize dispatch.
Break the extract on purpose
Download the companion and run it with Node.js 22 or later. It compares a complete snapshot, a missing page, substituted identity, duplicate record, unavailable reference, revision gap, and stale arrival. Each bad case has an explicit expected result. A failed assertion stops the program instead of printing a successful summary.
node extraction-check.mjsUse your connector’s actual paging and update rules when adapting the tests. Include partial pages, wrong filters, duplicate or missing records, late changes, deletions, and lost access. Keep the connector version and source evidence beside the test result so someone else can investigate a failure. This exercise estimates no production error rate.
What if the source cannot establish completeness?
A live endpoint may change while you page through it. Without a stable snapshot or trustworthy population reference, you may be unable to establish that the decision’s required records arrived. In the companion, removing the independent reference produces UNKNOWN, even when all sample pages look correct. The checker returns no accepted rows.
Ask the source owner for a way to check what’s missing. That might be a bounded export, a control report, or overlapping fetches that you compare and reconcile. A current-record lookup may answer a narrower question. If none is sufficient, keep the manual check, narrow the use, or pause the recommendation. Tell the next person which part of the data remains unknown.
Take an extract your team uses today and ask its owner which records should be in it. Remove a page or swap an ID without breaking the response format. Run the job again. If it stays green, decide which independent check should catch that missing or substituted record before the extract is used.