Repository navigation
Why do we need a max queue delay? #29081
Replies: 1 comment
|
You've hit the documented ceiling of Deno Queues (KV-backed): the The why has to do with how the queue is built on top of Deno KV: a delayed message is effectively invisible-until-time-T, and the whole scheduling/lease machinery is built around KV operations with finite, predictable visibility windows. Very long delays interact badly with that model — TTL/visibility semantics, queue lease handling, and the fact that KV is eventually consistent storage, not a durable scheduler. A message invisible for 6 months has to survive KV compaction, region moves, and GC pressure with no way to inspect or manage it mid-flight, which is exactly the kind of "works until it silently doesn't" case queues are supposed to eliminate. So the platform bounded it. The standard workaround for "remind me in N months" is a tick chain — enqueue a short-delay "tick" message that, when it fires, checks how much time remains and either does the work or re-enqueues itself with another bounded delay: const RESCHEDULE_SLICE_MS = 30 * 24 * 60 * 60 * 1000; // 30 days
export function scheduleAt(target: Date, payload: unknown) {
const delay = Math.min(
Math.max(target.getTime() - Date.now(), 0),
RESCHEDULE_SLICE_MS,
);
return kv.enqueue({ target: target.toISOString(), payload }, { delay });
}
async function handler(msg) {
if (Date.now() < Date.parse(msg.target)) {
await scheduleAt(new Date(msg.target), msg.payload); // re-arm
return;
}
// target reached — do the real work
}One thing worth knowing before you lean on very long chains: KV itself has data-lifetime semantics (unwritten entries get GC'd after ~90 days in some tiers) — if your scheduled work truly matters months out, the more robust architecture is an external scheduler (cron on Deploy, a one-shot task in your own infra) writing into the queue when the moment comes, and using Deno Queues only for the near-term execution part. |
Uh oh!
There was an error while loading. Please reload this page.
Hello everyone! 👋
I'm creating an application to notify me using the Deno queue delay, but I just discovered we have a maximum of 30 days' delay. - related PR
I have some questions regarding this constraint:
a. If so, is anyone working on this?
If this is something open to change, I can work on it.
All reactions