Skip to content

perf: cut Postiz memory footprint for single-tenant use (v0.3.0) - #3

Merged
fatherlinux merged 2 commits into
mainfrom
perf/memory-footprint
Aug 23, 2026
Merged

perf: cut Postiz memory footprint for single-tenant use (v0.3.0)#3
fatherlinux merged 2 commits into
mainfrom
perf/memory-footprint

Conversation

@fatherlinux

Copy link
Copy Markdown
Member

Summary

The container was pinned against its 2g cgroup limit with 1GB in swap — it had hit memory.max 133,238 times and been OOM-killed once. A restart reclaimed nothing: it returned to 2015M with 841M swapped within four minutes, which rules out a leak. This is a container sized for a workload it does not have.

Actual scale: 1 user, 6 integrations, 461 lifetime posts.

Root cause

temporal.module.ts registers one Temporal worker per supported provider rather than per connected one — 31 workers. Each carries its own @temporalio/core-bridge Rust core and thread pair, allocated outside the V8 heap where --max-old-space-size cannot bound it.

Confirmed by thread accounting in the running orchestrator:

threads name
31 workflow-proces
31 temporal-real-s
12 tokio-runtime-w
37 node
4 libuv-worker

117 total — exactly one workflow-proces/temporal-real-s pair per worker, against 1,088,416 kB of private dirty anonymous memory with the heap capped at 432MB.

Changes

  1. POSTIZ_ACTIVE_PROVIDERS (in fatherlinux/postiz-app@crunchtools-patches) trims workers 31 → 7. Unset preserves upstream behaviour exactly. Also pins maxCachedWorkflows instead of inheriting the SDK default, which derives from heap_size_limit and lands near 135 per worker.
  2. ecosystem.config.js runs node directly instead of pnpmdotenvnode, dropping six wrapper processes (~195MB). The dotenv layer reads /app/.env, which this image never creates.
  3. Per-app heap caps via PM2 interpreter_args, plus Temporal/postgres pools sized for one tenant.

Test plan

  • ecosystem.config.js parses; all three apps resolve to direct node invocations with distinct heap flags
  • Worker filter logic exercised against the real 32-provider list: unset → 31 workers (upstream preserved), 6 configured → 7 workers, main always present, typos and stray whitespace handled
  • Verified nestjs-temporal-core forwards workerOptions into Worker.create via Object.assign (temporal-worker.service.js:182-186)
  • Verified a CLI heap flag overrides inherited NODE_OPTIONS (128 → 176MB ceiling vs 432MB from env)
  • GHA build + container test
  • Post-deploy memory measurement on lotor

Caveat

A provider absent from POSTIZ_ACTIVE_PROVIDERS has nobody polling its task queue, so its posts enqueue and never execute. The boot log prints the active set and warns on unknown identifiers. Add the identifier and restart when connecting a new provider.

Note: postiz-pg-init.sh only runs on first init, so its new values do not affect the existing volume — that postgresql.conf is edited separately.

fatherlinux and others added 2 commits August 23, 2026 07:37
Bump to v0.3.0.

The container was pinned against its 2g cgroup limit with 1GB in swap,
having hit memory.max 133,238 times and been OOM-killed once. Restarting it
reclaimed nothing — it returned to 2015M with 841M swapped inside four
minutes, which rules out a leak: this is a container sized for a workload it
does not have. Actual scale is 1 user, 6 integrations, 461 lifetime posts.

Three changes, in order of impact:

1. POSTIZ_ACTIVE_PROVIDERS (see fatherlinux/postiz-app@crunchtools-patches)
   trims Temporal workers from 31 to 7. This is the ~700MB of native memory
   that the existing --max-old-space-size=384 could never reach, since it
   lives in the SDK's Rust core rather than the V8 heap.

2. ecosystem.config.js runs node directly instead of pnpm -> dotenv -> node,
   dropping six wrapper processes (~195MB). The dotenv layer reads
   /app/.env, which this image never creates, so it was loading nothing.

3. Per-app heap caps via PM2 interpreter_args, and Temporal/postgres
   connection pools sized for one tenant. A CLI flag overrides an inherited
   NODE_OPTIONS (verified: 128 yields a 176MB ceiling vs 432MB from the env),
   so per-app limits win over the global.

postiz-pg-init.sh only runs on first init, so its new values do not affect
the existing volume; that postgresql.conf is edited separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014dGq9ivDC6sqeJJfkra6C2
Testing the staged config against a disposable container caught this before
it shipped. Lowering max_connections to 30 leaves no headroom:

  - Prisma defaults to num_cpus*2+1 per client and DATABASE_URL sets no
    connection_limit, so on 6 vCPUs that is 13 each for the backend and the
    orchestrator = 26.
  - Temporal opens a pool PER SERVICE (frontend/history/matching/worker),
    not one global pool, so maxConns in config.yaml is not the ceiling.
    Measured 13 at idle in production, 12 in the test container.
  - Background workers = 4.

Worst case ~43 against an idle baseline of 22. At 30 this would have failed
with "sorry, too many clients already" under any real load.

Fresh installs now get 60 with the arithmetic documented inline. The
shared_buffers reduction to 128MB is unaffected and stays — it has no bearing
on connection limits.

The Temporal pool reduction (10/5 -> 6/2, 5/3 -> 4/1) is kept: the test
container ran clean on it with zero pool-exhaustion errors. Worth noting its
measured benefit is roughly one connection, so it is close to neutral. The
memory win in this release is entirely the worker-count and wrapper-process
changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014dGq9ivDC6sqeJJfkra6C2
@fatherlinux
fatherlinux merged commit e8d7c3e into main Aug 23, 2026
1 check passed
@fatherlinux
fatherlinux deleted the perf/memory-footprint branch August 23, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant