Checking an integration after the job says it finished

Android Bench 2.0 came out this week with long-horizon coding tasks, and it reports completion separately from pass rate. An agent can get most of a multi-day task done and still miss an important requirement. I'd like the same split in production integrations, because a job finishing and the data being right are two different things to check.

Say you've got a dashboard that gets account updates from another application (this is a hypothetical). The scheduled job runs, the dashboard shows a recent update time, and then someone opens an account that still has yesterday's details. Before rerunning anything, I'd want to know what the job's success status actually means. It might only mean the source request came back without an error, or that every fetched record went onto a queue. Neither tells you the destination accepted the changes, and if the dashboard reads from a separate reporting table there's another step after that. Each part can finish fine while a later part is still waiting or has rejected a record. That's also why a single "last synced" timestamp is hard to read: I'd rather see "source checked" and "updates waiting to be applied" as separate things.

Webhooks have the same issue. Stripe's webhook documentation asks endpoints to return a successful response quickly, before doing complex processing, and it says you can get duplicate deliveries and that delivery order isn't guaranteed. So in your own app, a 200 back to Stripe should mean something specific, like "we've durably accepted this for processing", and the work after that needs its own status and a way to recover.

For the dashboard, I'd start with a small read-only comparison against the source, matched on stable record IDs (not names) and limited to the fields the integration owns. A total count can match while individual records don't, since one missing account and one duplicate cancel each other out. I'd want the result to show:

• source records missing from the destination
• records where the mapped fields differ
• records still waiting, and how long they've waited
• rejected records, with the reason
• records that couldn't be compared because a lookup failed

That last group matters more than it looks. If the comparison can't read a destination record and quietly treats it as absent, a temporary read problem can turn into a duplicate creation attempt. Watch the timing too. Comparing an export taken before a change with a live read taken after it will throw up differences that are perfectly valid, so use a source snapshot if there is one, or record the comparison window and recheck before acting (and be careful with timezone conversions when a date filter picks the records).

Once you've got the list, scope the repair to those records, keep the uncertain ones out of the automatic batch, and run the comparison again afterwards. So when the job says it finished, I'd take that as the job's own status and check the records separately, the same way Android Bench reports completion apart from pass rate. For the next maintenance task, pick one integration that currently reports a last-run time, trace a record through to the screen that uses it, and write down where each status gets set, so you know what "finished" actually covers before you rely on it.