Implement queue rate limiting with GCRA algorithm #2795
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: f325c8d The changes in this PR will be included in the next version bump. This PR includes changesets to release 26 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
WalkthroughAdds GCRA-based Redis rate limiting across the task queue system: new duration parsing utilities and schemas, queue-level rateLimit configuration and per-run rateLimitKey propagation, a GCRARateLimiter implementation (Lua-backed) and RunQueue extensions to store/enforce configs, database schema and queue record changes, environment flag to disable rate limits, API/SDK plumbing for rateLimit and rateLimitKey, and unit/integration tests for duration parsing, key generation, enqueue/dequeue, and rate-limit behaviors. Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Areas requiring extra attention:
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Repository UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
.changeset/forty-houses-doubt.md (1)
6-6: Consider hyphenating compound adjective (optional refinement).While not incorrect, "rate-limiting algorithm" is the more standard hyphenation for a compound adjective. However, this is a minor style preference and the current text is perfectly acceptable.
If you prefer strict grammatical convention:
-Add GCRA rate limiting algorithm for task queue management +Add GCRA rate-limiting algorithm for task queue managementinternal-packages/run-engine/src/engine/systems/enqueueSystem.ts (1)
86-104: LGTM! Rate limit key fallback logic is correct.The implementation correctly handles rate limit key propagation with a clear fallback chain: explicit parameter → stored run value → undefined. The comment on lines 86-89 helpfully explains the behavior for re-enqueue scenarios (checkpoint, delay, waitpoint, pendingVersion).
Optional simplification for Line 90:
- const effectiveRateLimitKey = rateLimitKey ?? run.rateLimitKey ?? undefined; + const effectiveRateLimitKey = rateLimitKey ?? run.rateLimitKey;The final
?? undefinedis redundant since the nullish coalescing already returnsundefinedwhen both operands are nullish.packages/core/src/v3/schemas/schemas.ts (1)
149-161: Consider consolidating duplicate duration regex and aligning burst constraints.The
RateLimitConfigschema duplicates the duration regex fromDurationStringSchema(line 154 vs line 106). Also,bursthere hasmin(1)but no max, whilelimitis capped at 100000.Consider:
- Reusing
DurationStringSchemainstead of inline regex- Adding a max constraint to
burstfor consistencyexport const RateLimitConfig = z.object({ /** Maximum number of requests allowed in the period */ limit: z.number().int().min(1).max(100000), /** Time window - must be a valid duration string (e.g., "1s", "100ms", "5m", "1h") */ - period: z.string().regex(/^\d+(\.\d+)?(ms|s|m|h|d)$/, { - message: 'Period must be a valid duration string (e.g., "1s", "100ms", "5m", "1h")', - }), + period: DurationStringSchema, /** Optional burst allowance - allows temporary exceeding of rate limit (defaults to limit) */ - burst: z.number().int().min(1).optional(), + burst: z.number().int().min(1).max(100000).optional(), });Note: If you need the custom error message, you could add
.refine()or keep the inline approach, but the duplicate regex increases maintenance burden.internal-packages/run-engine/src/rate-limiter/gcra.ts (1)
6-75: Consider using types instead of interfaces per coding guidelines.The coding guidelines specify "Use types over interfaces for TypeScript". The current implementation uses
interfaceforQueueRateLimitConfig,StoredQueueRateLimitConfig,GCRAParams,GCRARateLimiterOptions, andRateLimitResult.While functionally equivalent here, for consistency with project conventions:
-export interface QueueRateLimitConfig { +export type QueueRateLimitConfig = { /** Maximum number of requests allowed within the period */ limit: number; /** Time period in milliseconds */ periodMs: number; /** Optional burst capacity (defaults to 1) */ burst?: number; -} +};Apply similar changes to the other interfaces.
apps/webapp/app/v3/services/createBackgroundWorker.server.ts (1)
373-388: Consider whether silently continuing without rate limiting is the desired behavior.When
parseDurationToMsthrows (e.g., invalid period format like"invalid"), the error is logged butrateLimitConfigremainsnull, and the queue is created without rate limiting. This is a graceful degradation approach.If rate limiting is critical for the user's use case, they might expect the deployment to fail rather than silently omit the rate limit. Consider whether this should throw a
ServiceValidationErrorinstead to surface the configuration error to the user.If you want to fail fast on invalid rate limit configuration:
} catch (error) { logger.error("createWorkerQueue: invalid rate limit period format", { queueName, rateLimit: queue.rateLimit, error, }); + throw new ServiceValidationError( + `Invalid rate limit period format "${queue.rateLimit.period}" for queue "${queueName}". Expected format: number + unit (ms, s, m, h, d)` + ); }
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (29)
.changeset/forty-houses-doubt.md(1 hunks)apps/webapp/app/env.server.ts(1 hunks)apps/webapp/app/runEngine/services/triggerTask.server.ts(1 hunks)apps/webapp/app/v3/runEngine.server.ts(1 hunks)apps/webapp/app/v3/runQueue.server.ts(2 hunks)apps/webapp/app/v3/services/createBackgroundWorker.server.ts(9 hunks)apps/webapp/app/v3/services/enqueueRun.server.ts(3 hunks)apps/webapp/app/v3/utils/durations.ts(1 hunks)apps/webapp/test/parseDurationToMs.test.ts(1 hunks)internal-packages/database/prisma/schema.prisma(1 hunks)internal-packages/run-engine/src/engine/index.ts(5 hunks)internal-packages/run-engine/src/engine/systems/enqueueSystem.ts(4 hunks)internal-packages/run-engine/src/engine/types.ts(2 hunks)internal-packages/run-engine/src/index.ts(1 hunks)internal-packages/run-engine/src/rate-limiter/gcra.ts(1 hunks)internal-packages/run-engine/src/rate-limiter/index.ts(1 hunks)internal-packages/run-engine/src/run-queue/index.ts(8 hunks)internal-packages/run-engine/src/run-queue/keyProducer.ts(2 hunks)internal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.ts(1 hunks)internal-packages/run-engine/src/run-queue/tests/keyProducer.test.ts(1 hunks)internal-packages/run-engine/src/run-queue/tests/rateLimit.test.ts(1 hunks)internal-packages/run-engine/src/run-queue/types.ts(2 hunks)packages/core/src/v3/schemas/__tests__/rateLimit.test.ts(1 hunks)packages/core/src/v3/schemas/api.ts(2 hunks)packages/core/src/v3/schemas/schemas.ts(3 hunks)packages/core/src/v3/types/queues.ts(1 hunks)packages/core/src/v3/types/tasks.ts(2 hunks)packages/trigger-sdk/package.json(1 hunks)packages/trigger-sdk/src/v3/shared.ts(29 hunks)
🧰 Additional context used
📓 Path-based instructions (11)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
Files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/run-queue/types.tsapps/webapp/app/v3/utils/durations.tsinternal-packages/run-engine/src/index.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/test/parseDurationToMs.test.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tspackages/core/src/v3/types/queues.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsinternal-packages/run-engine/src/rate-limiter/gcra.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/rate-limiter/index.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.tsinternal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/schemas.tspackages/trigger-sdk/src/v3/shared.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/run-queue/types.tsapps/webapp/app/v3/utils/durations.tsinternal-packages/run-engine/src/index.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/test/parseDurationToMs.test.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tspackages/core/src/v3/types/queues.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsinternal-packages/run-engine/src/rate-limiter/gcra.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/rate-limiter/index.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.tsinternal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/schemas.tspackages/trigger-sdk/src/v3/shared.ts
**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use vitest for all tests in the Trigger.dev repository
Files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsapps/webapp/test/parseDurationToMs.test.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.ts
**/*.{js,ts,jsx,tsx,json,md,css,scss}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier
Files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/runEngine/services/triggerTask.server.tspackages/trigger-sdk/package.jsoninternal-packages/run-engine/src/run-queue/types.tsapps/webapp/app/v3/utils/durations.tsinternal-packages/run-engine/src/index.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/test/parseDurationToMs.test.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tspackages/core/src/v3/types/queues.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsinternal-packages/run-engine/src/rate-limiter/gcra.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/rate-limiter/index.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.tsinternal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/schemas.tspackages/trigger-sdk/src/v3/shared.ts
**/*.test.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.test.{ts,tsx,js,jsx}: Test files should live beside the files under test and use descriptivedescribeanditblocks
Avoid mocks or stubs in tests; use helpers from@internal/testcontainerswhen Redis or Postgres are needed
Use vitest for unit tests
Files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsapps/webapp/test/parseDurationToMs.test.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/v3/utils/durations.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/test/parseDurationToMs.test.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tspackages/core/src/v3/types/queues.tspackages/core/src/v3/types/tasks.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tspackages/core/src/v3/schemas/schemas.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/v3/utils/durations.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/services/enqueueRun.server.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webapp
Files:
apps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/v3/utils/durations.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/test/parseDurationToMs.test.tsapps/webapp/app/v3/services/enqueueRun.server.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
apps/webapp/**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Test files should only import classes and functions from
app/**/*.tsfiles and should not importenv.server.tsdirectly or indirectly; pass configuration through options instead
Files:
apps/webapp/test/parseDurationToMs.test.ts
apps/webapp/app/v3/services/**/*.server.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Organize services in the webapp following the pattern
app/v3/services/*/*.server.ts
Files:
apps/webapp/app/v3/services/enqueueRun.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
packages/trigger-sdk/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Files:
packages/trigger-sdk/src/v3/shared.ts
🧠 Learnings (48)
📓 Common learnings
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/index.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/v3/services/enqueueRun.server.tsinternal-packages/run-engine/src/run-queue/tests/rateLimit.test.tspackages/core/src/v3/types/queues.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/run-queue/keyProducer.ts.changeset/forty-houses-doubt.mdinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/v3/runQueue.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.tsinternal-packages/run-engine/src/engine/index.tspackages/core/src/v3/schemas/schemas.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeyTTL` option to define a time window during which duplicate triggers return the original run
Applied to files:
internal-packages/run-engine/src/engine/types.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Configure OpenTelemetry instrumentations and exporters in trigger.config.ts for enhanced logging
Applied to files:
internal-packages/run-engine/src/engine/types.tsapps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Use build extensions in trigger.config.ts (additionalFiles, additionalPackages, aptGet, prismaExtension, etc.) to customize the build
Applied to files:
internal-packages/run-engine/src/engine/types.tspackages/trigger-sdk/package.jsonapps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach metadata to task runs using the metadata option when triggering, and access/update it inside runs using metadata functions
Applied to files:
internal-packages/run-engine/src/engine/types.tsapps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.tsinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Subscribe to run updates using `runs.subscribeToRun()` for realtime monitoring of task execution
Applied to files:
internal-packages/run-engine/src/engine/types.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Specify runtime environment (node or bun) in trigger.config.ts using the `runtime` property
Applied to files:
internal-packages/run-engine/src/engine/types.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Limit task duration using the `maxDuration` property (in seconds)
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use the `task()` function from `trigger.dev/sdk/v3` to define tasks with id and run properties
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdapps/webapp/app/v3/services/createBackgroundWorker.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.batchTrigger()` to trigger multiple runs of a single task with different payloads
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.trigger()` to trigger a task from inside another task with specified payload
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/types/tasks.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export tasks with unique IDs within the project to enable proper task discovery and execution
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/trigger-sdk/package.jsonpackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schedules.task()` for scheduled/cron tasks instead of regular `task()`
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdapps/webapp/app/v3/services/createBackgroundWorker.server.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.trigger()` with type-only imports to trigger tasks from backend code without importing the task implementation
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tspackages/core/src/v3/types/tasks.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to packages/trigger-sdk/**/*.{ts,tsx} : In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Applied to files:
packages/trigger-sdk/package.json
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: The SDK at packages/trigger-sdk is an isomorphic TypeScript SDK
Applied to files:
packages/trigger-sdk/package.json
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `trigger.dev/sdk/v3` for all imports in Trigger.dev tasks
Applied to files:
packages/trigger-sdk/package.json
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Configure build process in trigger.config.ts using `build` object with external packages, extensions, and JSX settings
Applied to files:
packages/trigger-sdk/package.json
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : When importing from `trigger.dev/core` in the webapp, use subpath exports from the package.json instead of importing from the root path
Applied to files:
packages/trigger-sdk/package.json
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schemaTask()` from `trigger.dev/sdk/v3` with Zod schema for payload validation
Applied to files:
packages/trigger-sdk/package.jsonapps/webapp/app/env.server.tspackages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.ts.changeset/forty-houses-doubt.mdpackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Use the Run Engine 2.0 from `internal/run-engine` for new run lifecycle code in the webapp instead of the legacy run engine
Applied to files:
apps/webapp/app/v3/runEngine.server.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
apps/webapp/app/v3/runEngine.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/app/**/*.{ts,tsx} : Access all environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env` in the Trigger.dev webapp
Applied to files:
apps/webapp/app/env.server.tsapps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-08-19T09:49:07.011Z
Learnt from: julienvanbeveren
Repo: triggerdotdev/trigger.dev PR: 2417
File: apps/webapp/app/routes/api.v1.projects.$projectRef.envvars.$slug.import.ts:56-61
Timestamp: 2025-08-19T09:49:07.011Z
Learning: In the Trigger.dev codebase, environment variables should default to `isSecret: false` when not explicitly marked as secrets in the syncEnvVars functionality. This is the intended behavior for both regular variables and parent variables.
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webapp
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-11-10T09:09:07.399Z
Learnt from: myftija
Repo: triggerdotdev/trigger.dev PR: 2663
File: apps/webapp/app/env.server.ts:1205-1206
Timestamp: 2025-11-10T09:09:07.399Z
Learning: In the trigger.dev webapp, S2_ACCESS_TOKEN and S2_DEPLOYMENT_LOGS_BASIN_NAME environment variables must remain optional until an OSS version of S2 is available, to avoid breaking environments that don't have S2 provisioned.
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:27:48.109Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-27T16:27:48.109Z
Learning: Applies to **/*.test.{ts,tsx,js,jsx} : Use vitest for unit tests
Applied to files:
apps/webapp/test/parseDurationToMs.test.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to **/*.{test,spec}.{ts,tsx} : Use vitest for all tests in the Trigger.dev repository
Applied to files:
apps/webapp/test/parseDurationToMs.test.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerAndWait()` to batch trigger multiple different tasks and wait for results
Applied to files:
apps/webapp/app/v3/services/enqueueRun.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTriggerAndWait()` to batch trigger tasks and wait for all results from a parent task
Applied to files:
apps/webapp/app/v3/services/enqueueRun.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTaskAndWait()` to batch trigger tasks by passing task instances and wait for results
Applied to files:
apps/webapp/app/v3/services/enqueueRun.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTrigger()` to trigger multiple runs of a task from inside another task
Applied to files:
apps/webapp/app/v3/services/enqueueRun.server.tspackages/core/src/v3/schemas/api.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTask()` to batch trigger tasks by passing task instances for static task sets
Applied to files:
packages/core/src/v3/schemas/api.tspackages/core/src/v3/types/tasks.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Configure task retry behavior using the `retry` property with options like maxAttempts, factor, and timeout values
Applied to files:
packages/core/src/v3/types/tasks.tspackages/core/src/v3/schemas/schemas.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Specify machine resources using the `machine` property with preset options like 'large-1x'
Applied to files:
packages/core/src/v3/types/tasks.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach cron schedules declaratively using the `cron` property or imperatively using `schedules.create()`
Applied to files:
.changeset/forty-houses-doubt.mdapps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata methods (set, del, replace, append, remove, increment, decrement, stream, flush) to update metadata during task execution
Applied to files:
.changeset/forty-houses-doubt.mdinternal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use logger methods (debug, log, info, warn, error) from `trigger.dev/sdk/v3` for structured logging in tasks
Applied to files:
.changeset/forty-houses-doubt.md
📚 Learning: 2025-11-14T16:03:06.917Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2681
File: apps/webapp/app/services/platform.v3.server.ts:258-302
Timestamp: 2025-11-14T16:03:06.917Z
Learning: In `apps/webapp/app/services/platform.v3.server.ts`, the `getDefaultEnvironmentConcurrencyLimit` function intentionally throws an error (rather than falling back to org.maximumConcurrencyLimit) when the billing client returns undefined plan limits. This fail-fast behavior prevents users from receiving more concurrency than their plan entitles them to. The org.maximumConcurrencyLimit fallback is only for self-hosted deployments where no billing client exists.
Applied to files:
apps/webapp/app/v3/runQueue.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.trigger()` to trigger multiple different tasks at once from backend code
Applied to files:
apps/webapp/app/v3/services/createBackgroundWorker.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to internal-packages/database/**/*.{ts,tsx} : Use Prisma for database interactions in internal-packages/database with PostgreSQL
Applied to files:
apps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Use `trigger.dev/redis-worker` for background job and worker system needs in the webapp and run engine
Applied to files:
apps/webapp/app/v3/services/createBackgroundWorker.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Scope idempotency keys globally or to current run using the scope parameter
Applied to files:
internal-packages/run-engine/src/engine/index.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Configure global retry settings in trigger.config.ts using `retries` object with defaults for all tasks
Applied to files:
packages/core/src/v3/schemas/schemas.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.triggerAndWait()` to trigger a task and wait for its result from a parent task
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `runs.subscribeToBatch()` to subscribe to changes for all runs in a batch
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
🧬 Code graph analysis (12)
internal-packages/run-engine/src/run-queue/tests/keyProducer.test.ts (1)
internal-packages/run-engine/src/run-queue/keyProducer.ts (1)
queueKey(101-127)
internal-packages/run-engine/src/run-queue/types.ts (1)
packages/trigger-sdk/src/v3/shared.ts (1)
queue(131-149)
apps/webapp/app/v3/utils/durations.ts (1)
apps/webapp/app/v3/runQueue.server.ts (1)
parseDurationToMs(6-6)
apps/webapp/app/v3/runEngine.server.ts (2)
apps/webapp/app/env.server.ts (1)
env(1285-1285)apps/supervisor/src/env.ts (1)
env(119-119)
apps/webapp/test/parseDurationToMs.test.ts (1)
apps/webapp/app/v3/utils/durations.ts (1)
parseDurationToMs(5-31)
packages/core/src/v3/schemas/__tests__/rateLimit.test.ts (1)
packages/core/src/v3/schemas/schemas.ts (2)
DurationStringSchema(106-106)QueueRateLimitConfigSchema(110-114)
internal-packages/run-engine/src/run-queue/tests/rateLimit.test.ts (3)
internal-packages/run-engine/src/run-queue/keyProducer.ts (1)
RunQueueFullKeyProducer(24-394)internal-packages/database/src/transaction.ts (1)
Decimal(21-21)internal-packages/testcontainers/src/index.ts (2)
redisTest(167-167)redisContainer(116-130)
internal-packages/run-engine/src/run-queue/keyProducer.ts (1)
internal-packages/run-engine/src/run-queue/types.ts (1)
RunQueueKeyProducerEnvironment(50-53)
apps/webapp/app/v3/services/createBackgroundWorker.server.ts (3)
internal-packages/run-engine/src/rate-limiter/index.ts (2)
QueueRateLimitConfig(7-7)parseDurationToMs(4-4)apps/webapp/app/v3/runQueue.server.ts (3)
QueueRateLimitConfig(11-18)parseDurationToMs(6-6)updateQueueRateLimitConfig(60-66)apps/webapp/app/v3/utils/durations.ts (1)
parseDurationToMs(5-31)
internal-packages/run-engine/src/run-queue/index.ts (5)
internal-packages/run-engine/src/index.ts (2)
QueueRateLimitConfig(29-29)configToGCRAParams(25-25)internal-packages/run-engine/src/rate-limiter/gcra.ts (2)
QueueRateLimitConfig(6-13)configToGCRAParams(117-124)internal-packages/run-engine/src/rate-limiter/index.ts (2)
QueueRateLimitConfig(7-7)configToGCRAParams(3-3)apps/webapp/app/v3/runQueue.server.ts (1)
QueueRateLimitConfig(11-18)packages/core/src/v3/schemas/schemas.ts (1)
QueueRateLimitConfig(116-116)
internal-packages/run-engine/src/run-queue/tests/enqueueMessage.test.ts (3)
internal-packages/testcontainers/src/index.ts (3)
redisTest(167-167)redisContainer(116-130)assertNonNullable(19-19)internal-packages/run-engine/src/run-queue/types.ts (2)
InputPayload(5-17)InputPayload(18-18)internal-packages/testcontainers/src/utils.ts (1)
assertNonNullable(173-176)
packages/core/src/v3/schemas/schemas.ts (2)
internal-packages/run-engine/src/index.ts (1)
QueueRateLimitConfig(29-29)apps/webapp/app/v3/runQueue.server.ts (1)
QueueRateLimitConfig(11-18)
🪛 LanguageTool
.changeset/forty-houses-doubt.md
[grammar] ~6-~6: Use a hyphen to join words.
Context: ...gger.dev/core": minor --- Add GCRA rate limiting algorithm for task queue manage...
(QB_NEW_EN_HYPHEN)
Vibe Kanban Review CompleteYour code review is ready! |
guillermolg00
changed the title
guillermolg00
changed the title
Author
Declarative GCRA Rate Limiting for Queue ProcessingContext & NeedPreviously, rate limiting in Trigger.dev required manual implementation within task code. This approach had several limitations:
Goal: Enable declarative rate limiting at the queue level, enforced before task execution begins. Solution OverviewImplemented GCRA (Generic Cell Rate Algorithm) directly in the Redis dequeue operation via Lua scripting. This ensures:
Architectureflowchart TB
subgraph SDK["@trigger.dev/sdk"]
TaskDef["Task Definition\nqueue: { rateLimit: { limit, period } }"]
TriggerOpts["Trigger Options\n{ rateLimitKey: 'tenant-123' }"]
end
subgraph CLI["trigger.dev CLI"]
Manifest["Queue Manifest\nwith rateLimit config"]
end
subgraph Webapp["apps/webapp"]
Sync["createBackgroundWorker.server.ts"]
RunQueue1["runQueue.server.ts"]
EnvVar["TRIGGER_DISABLE_QUEUE_RATE_LIMITS"]
end
subgraph RunEngine["@internal/run-engine"]
direction TB
GCRA["gcra.ts\nconfigToGCRAParams()"]
RunQueue2["RunQueue"]
LuaScript["Lua Script\ndequeueMessagesFromQueue"]
end
subgraph Redis["Redis"]
QueueData["Sorted Set\n{queue}:messages"]
RLConfig["Hash\n{queue}:rl:config"]
RLBucket["String\n{queue}:rl:{rateLimitKey}"]
Concurrency["Concurrency Keys"]
end
TaskDef --> Manifest
TriggerOpts --> RunQueue1
Manifest --> Sync
Sync --> GCRA
GCRA --> RLConfig
EnvVar -.->|"disable flag"| RunQueue2
RunQueue1 --> RunQueue2
RunQueue2 --> LuaScript
LuaScript --> QueueData
LuaScript --> RLConfig
LuaScript --> RLBucket
LuaScript -.->|"if allowed"| Concurrency
Dequeue Flow with Rate LimitingsequenceDiagram
participant W as Worker
participant RQ as RunQueue
participant Lua as Lua Script
participant Redis as Redis
W->>RQ: dequeueFromMasterQueue()
RQ->>Lua: EVALSHA dequeueMessagesFromQueue
loop For each message in queue
Lua->>Redis: ZRANGEBYSCORE (get eligible messages)
Lua->>Redis: GET {queue}:rl:config
alt Rate limit config exists & not disabled
Lua->>Lua: Extract rateLimitKey from message
Lua->>Lua: gcra_check(bucket, emission, burst)
Lua->>Redis: GET/SET bucket TAT
alt GCRA allows
Lua->>Redis: Update concurrency
Lua->>Redis: ZREM from queue
Lua-->>RQ: Return message
else Rate limited
Lua->>Lua: Calculate retryAfter + jitter
Lua->>Redis: ZADD with new score (delayed)
Note over Lua: Skip concurrency increment
end
else No rate limit
Lua->>Redis: Update concurrency
Lua->>Redis: ZREM from queue
Lua-->>RQ: Return message
end
end
RQ-->>W: Dequeued messages
GCRA Algorithm DetailsThe Generic Cell Rate Algorithm uses a "virtual scheduling" approach: Example:
Test CoverageNew test file:
ConfigurationQueue-level rate limitexport const myTask = task({
id: "my-task",
queue: {
name: "my-queue",
rateLimit: {
limit: 100,
period: "1m",
burst: 10 // optional, defaults to 1
}
},
run: async (payload) => { /* ... */ }
});Per-tenant rate limiting (at trigger time)// Using task instance
await myTask.trigger(payload, {
rateLimitKey: `tenant-${tenantId}`
});
// Using tasks.trigger() - works consistently
await tasks.trigger("my-task", payload, {
rateLimitKey: `tenant-${tenantId}`
});
// Batch triggering with per-item keys
await myTask.batchTrigger([
{ payload: item1, options: { rateLimitKey: "tenant-A" } },
{ payload: item2, options: { rateLimitKey: "tenant-B" } },
]);Environment variable# Disable all queue rate limits (useful for testing/debugging)
TRIGGER_DISABLE_QUEUE_RATE_LIMITS=1Key Design Decisions
|
ericallam
reviewed
Dec 18, 2025
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
packages/core/src/v3/types/queues.ts(1 hunks)packages/core/src/v3/types/tasks.ts(2 hunks)packages/trigger-sdk/src/v3/shared.ts(9 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/core/src/v3/types/tasks.ts
- packages/core/src/v3/types/queues.ts
🧰 Additional context used
📓 Path-based instructions (4)
packages/trigger-sdk/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Files:
packages/trigger-sdk/src/v3/shared.ts
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
Files:
packages/trigger-sdk/src/v3/shared.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Files:
packages/trigger-sdk/src/v3/shared.ts
**/*.{js,ts,jsx,tsx,json,md,css,scss}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier
Files:
packages/trigger-sdk/src/v3/shared.ts
🧠 Learnings (11)
📓 Common learnings
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata methods (set, del, replace, append, remove, increment, decrement, stream, flush) to update metadata during task execution
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schemaTask()` from `trigger.dev/sdk/v3` with Zod schema for payload validation
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach metadata to task runs using the metadata option when triggering, and access/update it inside runs using metadata functions
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `.withStreams()` to subscribe to realtime streams from task metadata in addition to run changes
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `trigger.dev/sdk/v3` for all imports in Trigger.dev tasks
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.batchTrigger()` to trigger multiple runs of a single task with different payloads
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTask()` to batch trigger tasks by passing task instances for static task sets
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.trigger()` to trigger multiple different tasks at once from backend code
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
🧬 Code graph analysis (1)
packages/trigger-sdk/src/v3/shared.ts (1)
packages/core/src/v3/resource-catalog-api.ts (1)
resourceCatalog(5-5)
🔇 Additional comments (3)
packages/trigger-sdk/src/v3/shared.ts (3)
131-142: LGTM! Rate limit added to queue registration.The
rateLimitparameter is correctly added to queue metadata registration, following the same pattern asconcurrencyLimit.
249-257: LGTM! Consistent queue metadata registration.Both
createTaskandcreateSchemaTaskcorrectly propagaterateLimitfrom the queue configuration to the metadata registration, ensuring consistent handling across different task creation methods.Also applies to: 382-390
131-142: Well-structured rate limiting integration.The changes follow existing patterns and maintain consistency across the codebase. The implementation:
- Uses optional chaining appropriately to handle undefined values
- Follows the same pattern as existing features like
concurrencyKey- Adheres to SDK coding guidelines (isomorphic code, function declarations, types over interfaces)
- Maintains backward compatibility by making rate limit features optional
Also applies to: 249-257, 382-390, 1955-1955, 2009-2009, 2050-2050, 2129-2129, 2283-2283, 2377-2377
Contributor
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
packages/trigger-sdk/src/v3/shared.ts (4)
636-660: Add missingrateLimitKeyto batch-by-id array path.The array path for
batchTriggerByIdis missingrateLimitKeypropagation fromitem.options?.rateLimitKey. Per-item rate limit keys are supported by theBatchByIdItemtype (viaTriggerOptions) but not forwarded to the server.🔎 Apply this diff to add the missing field:
return { index, task: item.id, payload: payloadPacket.data, options: { queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType, delay: item.options?.delay,
892-915: Add missingrateLimitKeyto batch-by-id-and-wait array path.The array path for
batchTriggerByIdAndWaitis missingrateLimitKeypropagation fromitem.options?.rateLimitKey.🔎 Apply this diff to add the missing field:
return { index, task: item.id, payload: payloadPacket.data, options: { lockToVersion: taskContext.worker?.version, queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
1152-1176: Add missingrateLimitKeyto batch-by-task array path.The array path for
batchTriggerTasksis missingrateLimitKeypropagation fromitem.options?.rateLimitKey. Per-item rate limit keys are supported by theBatchByTaskItemtype but not forwarded.🔎 Apply this diff to add the missing field:
return { index, task: item.task.id, payload: payloadPacket.data, options: { queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
1413-1437: Add missingrateLimitKeyto batch-by-task-and-wait array path.The array path for
batchTriggerAndWaitTasksis missingrateLimitKeypropagation fromitem.options?.rateLimitKey.🔎 Apply this diff to add the missing field:
return { index, task: item.task.id, payload: payloadPacket.data, options: { lockToVersion: taskContext.worker?.version, queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
♻️ Duplicate comments (4)
packages/trigger-sdk/src/v3/shared.ts (4)
1749-1773: Add missingrateLimitKeyto batch-by-id streaming path.The streaming transform for
batchTriggerByIdis missingrateLimitKeypropagation fromitem.options?.rateLimitKey, preventing per-item rate limiting in streamed batch-by-id operations.🔎 Apply this diff to add the missing field:
yield { index: index++, task: item.id, payload: payloadPacket.data, options: { queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
1801-1825: Add missingrateLimitKeyto batch-by-id-and-wait streaming path.The streaming transform for
batchTriggerByIdAndWaitis missingrateLimitKeypropagation fromitem.options?.rateLimitKey.🔎 Apply this diff to add the missing field:
yield { index: index++, task: item.id, payload: payloadPacket.data, options: { lockToVersion: taskContext.worker?.version, queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
1852-1876: Add missingrateLimitKeyto batch-by-task streaming path.The streaming transform for
batchTriggerTasksis missingrateLimitKeypropagation fromitem.options?.rateLimitKey.🔎 Apply this diff to add the missing field:
yield { index: index++, task: item.task.id, payload: payloadPacket.data, options: { queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
1903-1927: Add missingrateLimitKeyto batch-by-task-and-wait streaming path.The streaming transform for
batchTriggerAndWaitTasksis missingrateLimitKeypropagation fromitem.options?.rateLimitKey.🔎 Apply this diff to add the missing field:
yield { index: index++, task: item.task.id, payload: payloadPacket.data, options: { lockToVersion: taskContext.worker?.version, queue: item.options?.queue ? { name: item.options.queue } : undefined, concurrencyKey: item.options?.concurrencyKey, + rateLimitKey: item.options?.rateLimitKey, test: taskContext.ctx?.run.isTest, payloadType: payloadPacket.dataType,
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
apps/webapp/app/env.server.ts(1 hunks)apps/webapp/app/runEngine/services/triggerTask.server.ts(1 hunks)apps/webapp/app/v3/runEngine.server.ts(1 hunks)internal-packages/database/prisma/schema.prisma(1 hunks)internal-packages/run-engine/src/engine/index.ts(5 hunks)internal-packages/run-engine/src/engine/types.ts(2 hunks)packages/core/src/v3/schemas/api.ts(2 hunks)packages/core/src/v3/types/tasks.ts(2 hunks)packages/trigger-sdk/src/v3/shared.ts(9 hunks)
🚧 Files skipped from review as they are similar to previous changes (7)
- apps/webapp/app/runEngine/services/triggerTask.server.ts
- packages/core/src/v3/schemas/api.ts
- internal-packages/run-engine/src/engine/types.ts
- packages/core/src/v3/types/tasks.ts
- internal-packages/run-engine/src/engine/index.ts
- internal-packages/database/prisma/schema.prisma
- apps/webapp/app/env.server.ts
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
Files:
apps/webapp/app/v3/runEngine.server.tspackages/trigger-sdk/src/v3/shared.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/app/v3/runEngine.server.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Files:
apps/webapp/app/v3/runEngine.server.tspackages/trigger-sdk/src/v3/shared.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/v3/runEngine.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webapp
Files:
apps/webapp/app/v3/runEngine.server.ts
**/*.{js,ts,jsx,tsx,json,md,css,scss}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier
Files:
apps/webapp/app/v3/runEngine.server.tspackages/trigger-sdk/src/v3/shared.ts
packages/trigger-sdk/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Files:
packages/trigger-sdk/src/v3/shared.ts
🧠 Learnings (21)
📓 Common learnings
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
apps/webapp/app/v3/runEngine.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Use the Run Engine 2.0 from `internal/run-engine` for new run lifecycle code in the webapp instead of the legacy run engine
Applied to files:
apps/webapp/app/v3/runEngine.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Specify runtime environment (node or bun) in trigger.config.ts using the `runtime` property
Applied to files:
apps/webapp/app/v3/runEngine.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Configure OpenTelemetry instrumentations and exporters in trigger.config.ts for enhanced logging
Applied to files:
apps/webapp/app/v3/runEngine.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach metadata to task runs using the metadata option when triggering, and access/update it inside runs using metadata functions
Applied to files:
apps/webapp/app/v3/runEngine.server.tspackages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
apps/webapp/app/v3/runEngine.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata methods (set, del, replace, append, remove, increment, decrement, stream, flush) to update metadata during task execution
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schemaTask()` from `trigger.dev/sdk/v3` with Zod schema for payload validation
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `trigger.dev/sdk/v3` for all imports in Trigger.dev tasks
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `.withStreams()` to subscribe to realtime streams from task metadata in addition to run changes
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeyTTL` option to define a time window during which duplicate triggers return the original run
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.batchTrigger()` to trigger multiple runs of a single task with different payloads
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.trigger()` to trigger multiple different tasks at once from backend code
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTask()` to batch trigger tasks by passing task instances for static task sets
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTaskAndWait()` to batch trigger tasks by passing task instances and wait for results
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerAndWait()` to batch trigger multiple different tasks and wait for results
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTrigger()` to trigger multiple runs of a task from inside another task
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTriggerAndWait()` to batch trigger tasks and wait for all results from a parent task
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `runs.subscribeToBatch()` to subscribe to changes for all runs in a batch
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
🧬 Code graph analysis (1)
packages/trigger-sdk/src/v3/shared.ts (1)
packages/core/src/v3/resource-catalog-api.ts (1)
resourceCatalog(5-5)
🔇 Additional comments (3)
apps/webapp/app/v3/runEngine.server.ts (1)
83-83: LGTM: Rate limit disable flag properly configured.The flag correctly reads from the environment variable and follows the webapp pattern for boolean environment variable conversion.
packages/trigger-sdk/src/v3/shared.ts (2)
132-136: LGTM: Queue rate limit metadata properly registered.The
rateLimitfield is consistently added to queue metadata registration across all task creation paths.Also applies to: 255-255, 388-388
1963-1963: LGTM: Rate limit key properly propagated in single-task operations.The
rateLimitKeyis correctly threaded through direct trigger operations and single-task batch operations (both array and streaming paths).Also applies to: 2018-2018, 2060-2060, 2140-2140, 2294-2294, 2389-2389
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ Checklist
Testing
internal-packages/run-engine/src/run-queue/tests/rateLimit.test.tspackages/core/src/v3/schemas/__tests__/rateLimit.test.tstest-triggerreference projectTo run the tests:
cd internal-packages/run-engine
pnpm run test ./src/run-queue/tests/rateLimit.test.ts---
Changelog
Add GCRA (Generic Cell Rate Algorithm) rate limiting support for task queues. This feature allows users to configure rate limits on their tasks to control execution throughput.
Changes:
GCRARateLimiterimplementation in@internal/run-engine@trigger.dev/core@trigger.dev/sdktask definitionsenqueueRunservice to respect rate limits💯