Skip to content

docs(integration-toolkit): add the interactive registration recipe - #166

Open
karlrank wants to merge 12 commits into
mainfrom
docs/interactive-registration
Open

karlrank wants to merge 12 commits into
mainfrom
docs/interactive-registration

Conversation

@karlrank

@karlrank karlrank commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

What

Documents the Interactive Inbound Lane for integration partners: an "Interactive registration" recipe under the Integration Toolkit, plus the reference updates that go with it.

New page — docs/integrations/integration-toolkit/interactive-registration.md:

  • The problem, from the integrator's side. Today's advice is implicitly "push during the portal's poll window and hope". The portal polls entity search blind and cannot tell "not yet" from "never", which is why support tickets of this class exist and why none of them can be investigated afterwards.
  • The flow as a Mermaid sequence diagram: the portal mints a correlation id and passes it to the extension hook, the partner pushes with it, the portal waits on the correlation.
  • The partner minimum: two fields on the push — the portal's correlation_id and interactive: true — and nothing in the hook response. Returning the id from the hook is documented as optional (only for partners that push under an id of their own). correlation_complete: true on the last request is strongly recommended, never required.
  • The two failure patterns: customer found but bundle incomplete → push what exists and close the correlation in the same request; customer not found → answer the hook with an error and push nothing.
  • Request and response examples for POST /v3/erp/updates/events and the three read endpoints — one event, the correlation wait, and the correlations listing (GET /v3/erp/updates/correlations, newest first, 7-day horizon).
  • The correlation wait's two requirement sets, wait_for_entities and wait_for_use_cases: what "arrived" means, the selector rule (a selector may only name a member of the target's configured unique_ids), the selector syntax, the configuration rule for use case requirements (one use case per partner event type, every use case with a slug), and partial results — every requirement is met by the first matching event, so only correlation_complete says nothing further is coming.
  • The use case option interactive: disabled | on_request | always.
  • Rate limiting, honestly: over budget, the events run on the org's normal lane, the wait still works, it is just slower, and a monitoring warning is raised.
  • Event-stream partners (Kafka and similar): echo the id as a CloudEvents extension as the minimum, terminal event and full event set as improvements.
  • What not to do: wait on ingest (ignored, never synchronous), calling the wait from inside the hook, opening a correlation for a customer you could not find, selecting on an attribute that is not a configured unique id.
  • Monitoring: all ten codes with their levels.

Reference updates, slotted into the existing information architecture:

File Change
integration-toolkit/use-cases.md Inbound quick-reference row + an "Interactive Registration" use case section linking to the recipe
integration-toolkit/overview.md Component table row and a pointer in the Inbound direction section
integration-toolkit/configuration.md New "Inbound Use Case Options" table covering direct and interactive, plus the use case modelling rule
integration-toolkit/inbound/getting-started.md Note that ingestion is asynchronous, plus a Next Steps entry; the v3 example fixed (request-level correlation_id, no event_type, the required use case slug, the real response body)
integration-toolkit/configuration.md, inbound/examples.md v3 request shape in the event structure, correlation and example snippets

Navigation is autogenerated from the directory, so the new page appears under Integrations → Integration Toolkit between Use Cases and Mapping Examples (sidebar_position: 5).

Why

Registrations that time out today do so silently: the portal's entity-search poll returns nothing whether the push is queued, arrived without a contract, or was never sent. Nothing records which it was. The recipe replaces that with a contract an integrator can implement with two fields on the push and a status endpoint that answers the only question that matters — did the data land, and if not, what was missing.

Related

  • Specification: Interactive Inbound Lane RFC — integration-toolkit-api MR !278
  • The API documented here ships in the integration-toolkit-api MR stack (feat/interactive-inbound-lane). This docs PR merges after those, so the page does not describe fields the API still rejects. The page carries an "Availability — rolling out" admonition in the meantime.
  • The runnable reference, erpilot-integration, is being updated in parallel and is linked from the page.

Checks

  • npm run build (what CI runs) is clean on a fresh npm ci checkout of this branch — 293 doc pages, no broken links; every new in-page anchor resolves in the built HTML.
  • No src changes, so the repo's existing npm run lint findings are untouched.

Follow-ups (not in this PR)

  • monitoring/codes.md is autogenerated from the backend snapshot. The ten interactive codes (INTERACTIVE_RATE_LIMITED, INTERACTIVE_NOT_ALLOWED, INTERACTIVE_CROSS_LANE_WRITE, CORRELATION_COMPLETED, CORRELATION_CLOSED_EMPTY, CORRELATION_LATE_EVENT, CORRELATION_ID_MISSING, CORRELATION_REQUIREMENT_FAILED, CORRELATION_WAIT_TIMEOUT, CORRELATION_WAIT_REJECTED) are described on the recipe page and land in the reference via npm run update-monitoring-codes once the toolkit change is merged.

Corrections after the dev e2e run (2026-10-14)

Six commits on top, from the dev e2e findings triage and the decisions taken on them:

  • v3 examples (Getting Started, configuration, examples): meta.correlation_id and event_type were v1 shape and silently dropped by v3, so a partner copying them could never wait; step 2 lacked the required slug; step 3's response body was wrong.
  • Selector encoding: one wait_for_entities requirement per parameter occurrence, never comma-split, values encoded exactly once (curl --data-urlencode, URLSearchParams, the SDK); comma form for wait_for_use_cases only. curl, SDK and axios examples.
  • Wait verdicts: an entity requirement on an open correlation runs to timeout (error only once complete); zero-write events (disabled / unconfigured use case, extraction failure) end error/partial; partial counts as error for use case requirements; a complete correlation's answer is final; the impossible MISSING_UNIQUE_IDENTIFIERS example replaced; wait=0 on an undecided correlation reads timeout; unknown correlation ids answer at once.
  • Close rules, always, interactive: false reasons, monitoring table: duplicate-only retried close and close beyond the cap still close; mixed close; deduplication_id on retries and closes; always makes every event interactive and charged; wait on ingest ignored.
  • Grants and budget: ingest requires integration:ingest, reads integration:view; the budget is charged per request for admitted events, min(events, burst), all or nothing, a zero-event close costs one token.
  • Examples match the API: sort order, event_name as routing key, message casing, an opaque cursor.

Some of this describes the decided contract ahead of the toolkit fixes that implement it; merge after they land.

Karl Rankla added 12 commits October 9, 2026 14:32
The only guidance we gave an integration partner for portal self-registration
was implicit: push your data while the portal is still polling entity search,
and hope. That poll cannot tell "not yet" from "never", so a registration that
fails produces a generic timeout and a support ticket nobody can investigate
afterwards — not the partner, not epilot.

This documents the correlation-id contract that replaces it. The portal mints
an id and hands it to the extension hook, the partner returns it and sends it
on every push for that customer, and the portal waits on that correlation
instead of searching blind. The recipe states the minimum ask in two lines
before anything else, then the optional interactive lane and completion flag,
the request and response contract for the ingest call and both status
endpoints, the two requirement sets of the correlation wait and when to use
which, the `interactive` use case option, what the rate limit actually does to
an org that exceeds it, the event-stream variant, and the two things not to do.

The reference updates slot it into the existing structure rather than beside
it: a quick-reference row and a use case section in Use Cases, the component
table and the inbound direction in the Overview, a new inbound use case
options table in Configuration, and a pointer from the inbound Getting
Started where the asynchronous response is first explained.

Documents the Interactive Inbound Lane RFC (integration-toolkit-api !278).
The API described here ships in the toolkit MR stack, so this merges after it.
An event-stream bridge whose terminal marker arrives alone in a batch has
no data to send with the close, so it closes the correlation with an empty
events array. That is the one case in which the ingest endpoint accepts one;
without correlation_complete an empty events array is still a 400. Spelling
out both halves, because the asymmetry is otherwise only discoverable by
getting the error.
…nal contract

The recipe was written against an earlier draft of the API. Several
things have since been decided or found to be wrong, and an integrator
building against the old page would ship code that silently never works.

- The partner minimum is two fields on the push: the portal's
  correlation_id and interactive: true. Processing is only recorded for
  interactive events, so the id alone leaves nothing to wait on.
  Returning the id from the hook is optional, needed only when a partner
  pushes under an id of its own, because the portal waits on the id it
  minted when the hook returns none.
- A requirement is met by the first matching event. Without
  correlation_complete a wait can answer requirements_met while more of
  the customer's data is still arriving, so the flag is presented as
  strongly recommended and never required, with a worked example of
  where several use cases help and where one use case firing twice
  needs a selector instead.
- A selector can only name a member of the target's configured
  unique_ids, because that is all the response carries per entity; any
  other attribute never matches and fails silently. The examples now
  select on configured unique ids only, and the claim that the identity
  id is returned and that bundles carry a separate billing contact is
  gone, as both were verified false.
- Use case requirements are only as fine-grained as the use case
  configuration: one inbound use case per partner event type, each with
  a slug, or the requirement cannot tell messages apart or cannot be
  named at all.
- The correlations listing is its own endpoint,
  GET /v3/erp/updates/correlations, newest first with a declared 7-day
  horizon.
- Response field names follow the final schema (mapped_at, writes_total,
  writes_done, the use_cases list, the errors[] item shape,
  first_ingested_at and last_done_at), and the close, wait-range and
  completion rules follow the decided behaviour.
- Monitoring lists the five added codes, CORRELATION_WAIT_TIMEOUT is a
  warning carrying missing_entities and missing_use_cases, and
  CORRELATION_CLOSED_EMPTY is info, so a partner that correctly reports
  "no data" never alerts louder than a user who waited for nothing.
- The two partner failure patterns are documented: customer found but
  bundle incomplete pushes what exists and closes in the same request;
  customer not found answers the hook with an error and pushes nothing.
…d rules and the new codes

Partners that push in parallel close with a zero-event request once
every push is accepted; sequential pushers keep the close on the last
request. A correlation id is unique per identification, 1-255 characters
of [A-Za-z0-9_-] on waitable requests, and holds at most 32 events. A
selector key outside the configured unique ids is now a 400. Monitoring
lists CORRELATION_CLOSE_IGNORED, CORRELATION_CLOSED_INCOMPLETE and
CORRELATION_EVENT_LIMIT_EXCEEDED, and the timeout counts once per
correlation.
…rrounding whitespace

A wait_for_entities selector value is compared the way the toolkit's entity lookup compares unique ids: trimmed and lowercased on both sides, with whitespace inside the value still significant.
Getting Started, the configuration reference and the examples showed the
v1 request shape on the v3 endpoint: meta.correlation_id and event_type,
both of which v3 drops without an error, so a partner copying them got a
generated correlation id and could never wait on its own. Step 2 created a
use case without the required slug, and step 3 documented a response body
the API never returns.
One wait_for_entities requirement per parameter occurrence, never split on
commas, each value percent-encoded exactly once - as curl --data-urlencode,
URLSearchParams and the SDK do. The comma-separated form stays for
wait_for_use_cases only. The page previously offered comma-joined entity
requirements and said --data-urlencode carries a comma, which the dev e2e
run showed splitting a value into an injected second requirement.
… and complete

- An entity requirement whose event failed answers error only once the
  correlation is complete; on an open correlation it runs to timeout. The
  page claimed a wait never hangs on a failed event.
- Events that write nothing because the use case is disabled or
  unconfigured, or because extraction failed, end error or partial.
- partial counts as an error for wait_for_use_cases.
- A complete correlation's answer is final; late events are processed and
  counted but never change it.
- The error example used MISSING_UNIQUE_IDENTIFIERS, which ingest rejects
  with 422 before anything is recorded; it now shows
  ENTITY_REFERENCE_NOT_FOUND.
- wait=0 on an undecided correlation reads timeout, CLOSED_INCOMPLETE is
  raised only by waits above 0, and a wait on an unknown correlation id
  answers at once instead of holding a long poll.
… and the interactive false reasons

- always makes every event of the use case interactive and charged,
  including requests without the flag and without a correlation id.
- interactive:false also happens at the 32-event cap, for an always event
  with an invalid correlation id, and when the state store is unavailable.
- A retried closing request closes even when all its events are
  duplicates, a close beyond the cap still closes, a mixed close covers
  only the interactive events, and retries and closes need a
  deduplication_id.
- A wait field on ingest is ignored, not a 400.
- Monitoring table: CLOSE_IGNORED also covers the retry case,
  CLOSED_INCOMPLETE and REQUIREMENT_FAILED are raised once and never by
  wait=0, CROSS_LANE_WRITE is reported in either order, and
  CORRELATION_ID_MISSING carries no correlation id.
…w the interactive budget is charged

Ingest requires integration:ingest on the target integration (403
otherwise; existing scoped tokens need the grant) and the status reads
require integration:view, which the page described as the same
organization-scoped token. The rate limit is charged per request for the
events admitted as interactive after validation, deduplication and the
cap - min(events, burst), all or nothing - and a zero-event close costs
one token, not one token per event.
…returns

Sort use_cases and entities the way the API does (by slug, then _id) and
say so, show event_name on the event record as the routing key, use the
message casing the API returns for a disabled use case, and replace the
listing cursor example, which decoded to a key the API rejects, with an
opaque placeholder.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant