Read Replica Lag: You Saved It. Why Did It Disappear?

00:00:00 — Maya
I move the workshop to seven-thirty, hit Save, and the invitation preview still says seven. There's a Send button underneath it.

00:00:08 — Alex
Eighty people are about to get one of those times. I'm not clicking that yet. This is Development Radio. I'm Alex.

00:00:15 — Maya
I'm Maya. Let's build a small invitation tool and follow one edit. One organizer, eighty guests. She changes seven to seven-thirty, saves, then previews the invitation. Nobody edits it again. The save goes to the main database, the primary. Preview reads a separate copy, a replica.

00:00:34 — Alex
Then replica lag is already a good explanation. The primary has the new time, but the replica hasn't caught up. Did her save have to wait for that copy?

00:00:44 — Maya
In this tool, no. We're using asynchronous replication. The primary commits the change and the application says Saved. The replica applies that change separately. Until it does, a read there can still return seven. The success message and the old preview can both be telling the truth about different copies.

00:01:03 — Alex
Two honest answers, one organizer wondering whether to press Save again. She's checking the invitation she's about to send. I wouldn't ask her to untangle our database copies.

00:01:14 — Maya
No. I'd have Preview read the primary with a fresh query after the save commits. The database that accepted seven-thirty can supply seven-thirty. But I'd follow Send before calling the repair complete. A correct screen isn't evidence of what the sending code reads.

00:01:30 — Alex
Then let's follow it. Preview knows to read the primary. What happens when she presses that Send button?

00:01:35 — Maya
It hands the event ID to a background worker. The worker loads the event, builds the invitation, then sends copies to the guests. In this tool, that worker also reads the replica.

00:01:46 — Alex
Then she could check a correct preview and still send the wrong time. We fixed what she sees. We haven't fixed what she sends.

00:01:54 — Maya
That's worse than the first screen. At least seven in the preview makes her stop. Now we've reassured her, and another reader can quietly pick up seven again. Changing Preview's read route doesn't change the worker's.

00:02:07 — Alex
The worker is still carrying out her decision to send the corrected invitation. It needs that edit too. Would you take its event read to the primary as well?

00:02:16 — Maya
For this workflow, yes. The worker starts after the successful save, and it makes a fresh query against the primary. We fetch the event once to build this batch, not once for every guest. That gives both Preview and the invitation builder the saved seven-thirty, while this organizer leaves the event unchanged.

00:02:35 — Alex
One batch read helps. But I'm wary of 'use the primary' becoming our answer to every stale screen. We added replicas to take work off it. Where would you draw the line?

00:02:45 — Maya
Around the edit she's acting on. Preview needs the saved change. The worker building that invitation needs it too. Browsing older workshops is a different promise. I'd keep those routes separate and measure the extra primary load from these two operations.

00:03:00 — Alex
I can live with that split. But now the worker's primary read fails, and the replica is still answering. The organizer has a correct preview in front of her. Are we really leaving Send waiting?

00:03:12 — Maya
Yes. I'd leave sending pending. The replica being available doesn't tell us it includes her edit. We can't use it as a fallback just to clear the job. She may have to wait even while that copy can return a perfectly readable record.

00:03:26 — Alex
You're taking away an escape route. If I'm trying to get this invitation out, a working replica is awfully tempting.

00:03:32 — Maya
It is. But what would we be sending? Seven, possibly. She didn't ask us to get any invitation out; she asked us to send the corrected one. I'd show 'Saved, sending is waiting' rather than put a Sent badge over a version we haven't checked.

00:03:49 — Alex
Then let's test that choice with the replica deliberately behind. Leave it at seven, save seven-thirty, then open Preview and build the invitation. Both should say seven-thirty.

00:04:00 — Maya
Then make the worker's primary read fail: sending must stay pending. It mustn't quietly switch back to seven. Pick one save in your product and follow the next action to its read. Name the edit it must include, and what waits if it can't. Those eighty guests shouldn't need to understand replica lag to arrive at seven-thirty.