Define the stop
Separate an unknown result from a rejected request.
A missing billing account is a different case from an unavailable service. Waiting will not supply a required account identifier, and changing an identifier will not repair a network outage. Give those conditions different routes.
For the timeout, hold this deal at Invoice outcome unknown. Do not mark it invoiced, send a customer confirmation or create a fresh request identifier just to make the error disappear. Other valid deals may continue if the failure is isolated; stop the wider flow if there is evidence of a shared fault.
Write the rule before the automation
Missing business input: ask the role responsible for that input to correct it.
Temporary technical fault: use a limited retry only when the action is safe to repeat.
Unknown downstream result: look up the original operation before trying again.
Decision outside policy: hold for someone with the required authority.
A case record the next operator can use
Keep one durable exception record linked to the original job. A notification points to that record; it should not become a second, conflicting account of the work. This template uses fictional identifiers and no customer details.
| Field | Worked example |
|---|---|
| Case and operation | EX-104; deal D-482; draft-invoice operation invoice:D-482:v1. |
| Stopped step and state | Create draft invoice; outcome unknown after timeout. Customer confirmation blocked. |
| Evidence | Attempt time, run ID, approved deal version, receiver lookup and any response ID. |
| Impact | Invoice preparation is delayed. No customer-facing completion is confirmed. |
| Decision owner | Billing operations investigates receiver state; account owner resolves commercial input disputes. |
| Fallback and next review | Billing lead takes over if the assigned owner is unavailable. Agree the review deadline for this workflow. |
| Closure condition | One approved draft is matched to this deal and operation, or an authorised decision to cancel is recorded. CRM status agrees. |
Recover the timed-out invoice request
These are the proposed steps for the fictional case. They assume the receiver can look up a logical operation and honours a documented idempotency contract. If it cannot, keep the case with an operator rather than enabling blind retries.
01
Inspect the original operation
Search the receiver using the recorded operation key and deal version. A missing response does not establish that creation failed. Check whether processing is complete, still pending or genuinely absent.
02
Reuse a confirmed result
If the expected draft already exists, link its receiver ID to the original job. Verify its account and approved amounts rather than treating any draft on the account as a match.
03
Repeat only under a safe contract
If a retry is warranted, reuse the original operation key under the receiver's deduplication rules. Respect the key lifetime and retry limit. A changed commercial instruction is a new decision, not the same request with silently altered values.
04
Verify both records
Confirm the approved draft and its link in the CRM, then release the next permitted step. Record the outcome and supporting IDs before closing EX-104. Leave an unresolved discrepancy open.
Recover the invoice request
The same Pending CRM status can hide different billing results. This synthetic case makes the evidence and effects of each recovery action visible.
A repeat using the original operation key is safe only under the receiving system's actual contract. Creating a fresh request is a different instruction. Accepting the case is not the same as resolving it.
In this model, a complete receiver lookup can confirm that no work is pending. Real systems may have delayed records or incomplete lookup APIs; keep an uncertain case open rather than treating a blank search as proof.
Synthetic case EX-104. No billing connection or customer data. A prepared draft is not an issued invoice.
- Billing evidence
- Unknown: no receiver evidence inspected
- CRM
- Pending: draft not verified in CRM
- Decision owner
- Awaiting billing operations
- Exception
- Open: customer confirmation held
Event log
EX-104 opened. CRM Pending; original operation invoice:D-482:v1 has no confirmed completion.
Full case history (1)
- EX-104 opened. CRM Pending; original operation invoice:D-482:v1 has no confirmed completion.
Account for work that already happened
The failure may happen after the draft was created but before the CRM was updated. Re-running the entire workflow can repeat the completed step. Recovery should identify the last confirmed result and resume only the work still required.
If the wrong draft exists, do not assume restoring application code undoes it. The responsible billing role needs to decide the permitted correction in the receiving system. The exception record must retain the relationship between the original action and the correction. Some effects cannot be undone; a message already delivered needs a different response from an unsent draft.
Before any correction
List the effects already confirmed in each system.
Check whether the correction itself can be repeated safely.
Name the role allowed to approve that correction.
Verify the resulting state rather than closing on a successful button press.
Test the recovery procedure before relying on it
Use synthetic records in a test environment. Ask someone who did not build the workflow to follow the exception record; note where they need an explanation that the record does not provide.
| Test | Expected result to agree and verify |
|---|---|
| Required billing input missing | No draft request is sent; the input owner receives a specific correction task. |
| Receiver completes but response is lost | Lookup finds the original draft; recovery creates no additional draft. |
| Receiver remains unavailable | Retries stop at the agreed limit; the job stays visible with a next owner and review time. |
| Primary owner is absent | The named fallback can accept the case with the original evidence and age. |
| Two operators attempt recovery | A claim or concurrency control prevents simultaneous conflicting actions; receiver deduplication remains a separate safeguard. |
| Draft exists but CRM update failed | Recovery repairs the link and status without replaying draft creation. |
| Operator selects Resolved without a matched outcome | The closure check rejects the incomplete result or exposes it for review. |
