Offline forms on the phone

Experimental

The offline send queue is experimental (3.0 GA): the declaration, the wire field and the header may still change in a minor release.

A warehouse, a basement, a lift: the user fills a form on the phone, presses Save and there is no signal. By default that save fails, as it does on the web — the change has not been sent and the user is told so. For the writes that still make sense later, the Mateu app can keep them in a send queue and send them when the connection is back.

It is opt-in, per action: only an action that declares it is ever queued. It works in both kinds of build of the app: the hub (the generic Mateu app, one queue per UI in its list) and a fixed build bound to one UI (your own branded app, MATEU_VARIANT=fixed — see Mobile app stores), whose single UI has its queue.

Declare it

@Button
@Action(offline = OfflinePolicy.QUEUE)
public Message save() {
  stock.record(location, counted);
  return Message.success("Count saved");
}

Language	Declaration
Java (annotation)	@Action(offline = OfflinePolicy.QUEUE)
Java (fluent)	Action.builder().id("save").offline(OfflinePolicy.QUEUE)
YAML (actions: of a definition)	offline: QUEUE
C#	[ActionOptions(Offline = OfflinePolicy.Queue)]
Python	@action_options(offline=OfflinePolicy.QUEUE)

The action then travels with offline: "queue" on the wire (ActionDto.offline; absent otherwise).

Only for writes that still make sense later, possibly after other changes happened in between: a stock count, a delivery note, an inspection report, a meter reading. Not for anything whose outcome depends on the moment it runs (booking a slot that may no longer be free, a payment): leave those failing, so the user decides.

What the user sees

1. They press Save without connection (or the server does not answer in time). The app keeps the request and says “Pending — will be sent when you’re back online.” The form keeps what they typed.
2. While something is waiting, a strip at the bottom of the screen says how many changes are pending, with Send now and View. View lists them; each one can be cancelled.
3. The UI’s row in the app’s list (the hub) shows “N changes pending”.
4. When the connection comes back — or the UI is opened, or the app returns to the foreground — the app sends them in order, one at a time, and says how many were sent. If the server is there but fails, is busy or does not answer in time, the item stays pending and the app tries again on its own after a while (30 seconds, doubling up to 10 minutes).
5. If the server refuses one (a validation error, a 4xx, an error message), the queue stops there and shows why: what follows it may depend on it. The user can Try again (after fixing things elsewhere) or Cancel it.
6. If the sign-in expired, the app refreshes it silently and sends it. If that cannot happen, the item keeps waiting and the user is told to sign in again; it goes once they have.
7. If the stored queue could not be read back whole, it is never shown as an empty list — see below.

A queued item belongs to its UI: removing the UI from the list drops what it had waiting.

Whose queue it is — one per signed-in person

A queued request is sent later with the sign-in that is current then, so a queue belongs to the person who filled it — its owner — and to nobody else:

• On a UI that signs in, the owner is the signed-in identity: the issuer and subject (iss, sub) of the id token — or of a JWT access token when there is no id token — plus the tenant when the token names one (tid, tenant_id, tenant). A renewed or refreshed token of the same person is the same owner. On a UI with no sign-in, everything goes to one anonymous queue, as before.
• When the tokens do not say who signed in (opaque tokens, no JWT at all), the app asks the provider at sign-in and at each token refresh — while online — through its OpenID Connect userinfo_endpoint (from the discovery document, HTTPS only), and keeps the answer’s sub (plus tenant) with the stored tokens, at the configured issuer, so the owner is known later offline. A refresh that answers another sub is another owner; a refresh whose userinfo call fails keeps the owner of the sign-in it refreshed. Claims in the tokens always win over userinfo. If the provider offers no userinfo endpoint, or it fails at sign-in, nothing is guessed: that sign-in queues nothing, and those writes fail as an ordinary write does.
• Each owner has their own queue on the device. Before each send the app checks that the one signed in now is the queue’s owner; otherwise nothing is sent. A send that has to refresh an expired token does it silently and is repeated only if the same person is still signed in — it never opens the sign-in browser on its own.
• Signing out suspends the queue: it stays on the device, untouched — not sent, not shown and not counted in the hub’s badge. Someone else signing in to the same UI starts with an empty queue of their own and sees nothing of it. When the same person signs in again, their queue is back and is sent.
• Someone else signing in mid-session — the sign-in expired, a request opened the sign-in browser and a different person signed in (a shared phone) — gets a fresh session of the UI with their own queue: the screen loaded as the previous person is not kept, and nothing they save is filed in the previous person’s queue. Signing in where nobody was signed in yet (a UI that signs in only when a request needs it) keeps the screen and only opens that person’s queue.
• The hub’s “N changes pending” counts only the queue of whoever is signed in to that UI now.

When the stored queue cannot be read

The queue is written so that an interrupted write (the app killed half-way, a full keychain) never mixes two versions: each new version is written in full under a new generation, one pointer is then switched to it, and only after that is an older version deleted. Every version carries its length and a checksum, checked when it is read back.

• If the latest version does not read back whole, the app falls back to the previous one and says so: “The list of pending changes was restored from its previous copy. Check that it is complete.” (a change made just before may be missing, or one already sent may be back — the server’s Idempotency-Key handling keeps a repeat from being applied twice).
• If no version can be read, the queue is shown as damaged — never as an empty list: “Your pending changes saved on this device could not be read. They cannot be sent.”, with Discard them (asked for confirmation). Until the user discards it, nothing new is written over it — a write made offline in the meantime is not kept, and the user is told why.
• A keychain that cannot be read right now (a locked device) is not damage: nothing is shown or written, and the next use reads it again.

These messages are in the app’s chrome language (English, Spanish, French, German, Portuguese, Italian).

The contract with the server — Idempotency-Key

A queued write replays later, and the app may not know whether the first attempt arrived: when a request times out, the server may well have processed it and only the answer was lost. So every attempt of a queueable action — the first one included — carries the same key:

POST /mateu/v3/sync/count
Idempotency-Key: 6f1c2b9e-3d4a-4c1f-9b7e-2a8d5c0e1f34

The Java, .NET and Python servers honour it on the sync endpoint:

• A request with a key runs once per caller, action and key. A repeat of the same key gets the first answer back and the action does not run again; a repeat arriving while the first run is still in flight waits for it and gets its answer.

• The caller is the SAME identity your access checks read — Java’s IdentityResolver (a PrincipalResolver bean or the framework’s principal), .NET’s MateuOptions.Identity, Python’s identity_provider — taken whole: name, roles, groups, scopes and permissions, sorted and hashed. One user’s key never answers another user’s request, and a caller whose grants changed (a permission revoked) is not served the answer remembered for the old ones: the request runs again and meets the action’s checks. The view and the action id are part of the scope too: the request’s server-side type and route (trimmed, route without leading or trailing slashes — the same in the three servers), so the same key and action id on another screen is another entry. On .NET, a custom MateuOptions.Identity that does not read HttpContext.User should set Identity.Name, or two users with the same grants look alike.

• The key is looked up only once the request has passed its checks, so a request refused by any rule — identity or not — is never answered from memory. The three servers behave the same: the key stays pending while the request is handled and is claimed at the point the action is about to run — after the exposed-type and YAML route/action access checks, the view’s class gates, the action’s resolution and its access annotations (advertised / strict action, @EyesOnly / @DisabledUnless, Python’s @audience), the CRUD write gates (canEdit()/CanEdit/can_edit(), display()…), an editable view’s read-only gate and a capability listing’s declared writes. A request that never reaches that point (a load, a search) simply runs.

• A run that fails — an exception, or an answer carrying an error message (a refusal, the error boundary’s) — is forgotten, so the same key can be sent again once the problem is fixed.

• A key is 8–128 url-safe characters ([A-Za-z0-9_\-:.]); anything else is ignored (the request runs as if it had none). The app sends a v4 UUID.

• Keys are remembered in memory, per process — lost on a restart — in two tables:

  • runs in flight are never evicted (dropping one would let a retry start a second run while the first is still working). When 2000 keyed runs are already in flight, a request with a NEW key is refused with the error “Too many pending requests — The server is still processing too many requests. Try again in a moment.” The app does not take it as a refusal of the change: the item stays pending and is sent again later, like after a server failure (see step 4 above); a repeat of a key already in flight always joins its run.
  • finished answers: at most 2000 (least recently used first out), each for 24 hours. After that the key is forgotten and a repeat would run again.

  Tune both with mateu.idempotency.max-entries / mateu.idempotency.ttl-seconds (Java), MATEU_IDEMPOTENCY_MAX_ENTRIES / MATEU_IDEMPOTENCY_TTL_SECONDS (.NET, Python).

• Each caller has its own share of both tables (mateu.idempotency.max-entries-per-caller, 100): past it, the caller’s new key is refused with the same “Too many pending requests” while its runs are in flight, and its own least recently used answer is forgotten when it finishes more — one caller can never fill the tables for the others or evict their answers.

• The same key sent with a different request (another component state or parameters) is not a retry: it is refused with “Idempotency key reused — This key was already used for a different request. Send this one with a new key.”

• A run does not depend on the connection that started it: if the client gives up (its timeout drops the connection), the run goes on and keeps its key, and the retry joins it.

Several instances

The memory is per process: it does not survive a restart and is not shared. Behind a load balancer, the retries of one device must reach the same instance (sticky sessions) for the guarantee to hold across instances; a retry that lands after a restart, on another instance or after the 24 hours runs the action again. This is a best-effort guard, not a persistent exactly-once guarantee. If your write is naturally idempotent (an upsert by a client-generated id), you can also read the header yourself (HttpRequest.getHeaderValue("Idempotency-Key")) and deduplicate in your own store.

The header is honoured for any action that sends it, not only queued ones; only the app sends it today, and only for actions declared offline = QUEUE. With a key, the app also repeats the request after a timeout or a 5xx (the server deduplicates), which it never does for an ordinary write.

Storage and security

• The queue lives on the device in the platform keychain / keystore (expo-secure-store, encrypted at rest), one per UI and owner, split in chunks because a secure-store value is small, written generationally (see above). On the web build of the app it is the browser’s localStorage, like the sign-in tokens there.
• Only the request body is stored — the form’s state, the action, the route, the parameters. Never a token or a header: the request is signed with the token current when it is sent.
• A queue holds at most 50 items and 64 KB per item (a form with a large photo is not kept: it fails as before).
• Only the UI’s own backend: an action of a remote island (another backend) is not queued.
• A static UI (a bundle with no server) never queues: it has nobody to send it to.

What is not covered

• The web renderers ignore the declaration: a browser tab without connection fails as before.
• Streamed actions (@Action(sse = true), a LongTask) are not queued.
• Nothing is merged: if the record changed on the server in between, your action’s own rules decide (e.g. an optimistic @Version check refuses it, and the refusal is shown).