prp-local

Stateless per-chunk article generation for any OpenAI-compatible local LLM server (LM Studio, Ollama, llama.cpp HTTP, vLLM).

prp-local lets a small local model (e.g. Gemma 4 e4b on a 16 GB Mac) produce long-form content that would not fit in its context window. It does this by splitting the work into chunks, calling the model with a fresh context per chunk, and using the disk as the persistent memory.

Why

Small local models have small context windows. A 4K window cannot hold a 10,000-word article and its own working trace at the same time. Most tools try to fit everything into one long conversation and break.

prp-local does the opposite. Each chunk is its own short API call. The model sees only the outline and the title of the section it is writing right now. Everything else lives on disk.

This is the MemGPT pattern applied to a single writing task: a small working frame plus a large external store.

Install

pip install prp-local

Or from source:

git clone https://github.com/capitansuat/prp-local
cd prp-local
pip install -e .

Prerequisites

Any OpenAI-compatible local LLM server. Tested with LM Studio on macOS. Start the server on port 1234 (or any port) with a chat model loaded.

Check it is reachable:

curl http://127.0.0.1:1234/v1/models

Quick start

Generate a 2000-word article in English:

prp-local new "the history of public-key cryptography"

Same in Turkish, longer target:

prp-local new "yapay zeka etiği" --target 5000 --lang tr

The article ends up at runs/<task_id>/final.md. The task id is derived from the subject; you can override it with --task my_id.

Example: 6,000-word Turkish article on Gemma 4 e4b

This is an actual run on a 16 GB MacBook Pro M1 using LM Studio with Gemma 4 e4b in a 4K context window. The model was asked for ~10,000 words; it chose its own chunk count and per-section length.

$ prp-local new "yapay zeka etiğinin felsefi, hukuki, sosyal ve teknik boyutları" \
    --target 10000 --lang tr

[1/4] Health check
  LM Studio OK, model: google/gemma-4-e4b
[2/4] Plan
  Plan written (12 chunks, sum target ~8800 words):
    1.  [750w] Giriş: Yapay Zeka ve Etik Sorumluluk
    2.  [800w] Felsefi Boyut I: Deontoloji ve Konseküansyalizm
    3.  [750w] Felsefi Boyut II: Erdem Etiği ve Sorumluluk
    4.  [800w] Hukuki Boyut I: Düzenleyici Çerçeveler ve AB AI Act
    5.  [700w] Hukuki Boyut II: Sorumluluk, Hesap Verebilirlik ve İspat Yükü
    6.  [800w] Sosyal Boyut I: Algoritmik Önyargı ve Eşitsizlik
    7.  [800w] Sosyal Boyut II: İşgücü Piyasası Dönüşümü
    8.  [750w] Teknik Boyut I: Şeffaflık ve Açıklanabilirlik
    9.  [800w] Teknik Boyut II: Veri Gizliliği, Güvenlik ve Sağlamlık
    10. [650w] Etik Tasarım Prensipleri
    11. [700w] Yönetişim Modelleri: Ulusal, Kurumsal ve Küresel Yaklaşımlar
    12. [500w] Sonuç: İnsan Merkezli Bir Gelecek Vizyonu
[3/4] Chunk loop
    [1] OK 482w (target 750), max_tok=2437, 57.3s
    [2] OK 622w (target 800), max_tok=2600, 68.3s
    [3] OK 534w (target 750), max_tok=2437, 64.4s
    [4] OK 602w (target 800), max_tok=2600, 51.0s
    [5] OK 488w (target 700), max_tok=2275, 56.8s
    [6] OK 577w (target 800), max_tok=2600, 64.2s
    [7] OK 554w (target 800), max_tok=2600, 49.4s
    [8] OK 571w (target 750), max_tok=2437, 65.5s
    [9] OK 596w (target 800), max_tok=2600, 65.1s
    [10] OK 487w (target 650), max_tok=2112, 50.4s
    [11] OK 521w (target 700), max_tok=2275, 55.3s
    [12] OK 380w (target 500), max_tok=1625, 47.9s
[4/4] Assemble
  Final: ~/Downloads/prp-local/runs/ai_ethics_10k/final.md
  Total: 6414 words across 12 chunks

The full run took about 12 minutes. The model never saw more than one section's worth of context at any moment. If the process had died at chunk 7, prp-local resume ai_ethics_10k --subject "..." would have picked up from chunk 8 by reading the disk.

The same task in a single long conversation would have overflowed Gemma 4 e4b's 4K context long before reaching the assembly step.

How it works

Five phases, all run automatically:

1. Plan. Python asks the model for a JSON outline with per-section word targets. The model decides how many sections, and how long each one should be. The user only provides a total target.
2. Write plan.md. Python writes the plan to disk in a strict format. The model never writes plan.md.
3. Chunk loop. For each section, Python makes a fresh API call. The prompt contains the subject, the outline (titles only), and the section to write. Previous chunk content is never sent back to the model.
4. Reconcile. After each chunk, plan.md is updated. If the process dies, the next resume reads the disk and continues from the first missing chunk.
5. Assemble. Python reads every chunk file in order and writes final.md. The model is not involved in assembly.

Commands

prp-local new <subject> [--task ID] [--target N] [--lang tr|en]
prp-local resume <task_id> [--subject TEXT] [--lang tr|en]
prp-local status <task_id>
prp-local assemble <task_id>
prp-local config [--init]

All commands accept --url URL and --runs-dir DIR for one-off overrides.

Configuration

Settings come from, in order: **CLI flag > environment variable > config file

defaults**. The first source that provides a value wins.

Defaults

Setting	Default
URL	http://127.0.0.1:1234
Language	en
Target word count	2000

Environment variables

export PRP_LOCAL_URL=http://192.168.1.50:1234
export PRP_LOCAL_LANG=tr
export PRP_LOCAL_TARGET=5000
export PRP_LOCAL_RUNS_DIR=~/my-articles

Config file

Initialize:

prp-local config --init

This creates ~/.config/prp-local/config.toml (or $XDG_CONFIG_HOME/prp-local/config.toml). Edit:

url = "http://127.0.0.1:1234"
default_lang = "tr"
default_target = 5000

View the effective config:

prp-local config

Resume

If something fails (server crash, network blip), restart:

prp-local resume <task_id> --subject "the original subject"

prp-local reads runs/<task_id>/plan.md, checks which <n>.md files exist on disk, marks them done, and continues from the first missing one. Existing chunk files are never overwritten.

Plan format

plan.md is plain text, parseable by both humans and the tool:

task:my_article
1:done|450|Introduction
2:done|500|Main argument
3:pending|400|Counterexamples
4:pending|350|Conclusion

Each line after the header is <n>:<status>|<target_words>|<title>.

Token budget formulas

Token-per-word ratios come from public tokenizer benchmarks, not from arbitrary choices:

• English: ~1.3 tokens per word
• Turkish: ~2.5 tokens per word (agglutinative)
• Safety margin: 1.3x

A chunk's max_tokens is computed from its own target_words and the language. The plan call's budget is computed from the total target and the maximum possible chunk count. Nothing is hard-coded against a specific model.

To add a language, add an entry to TOKENS_PER_WORD in src/prp_local/budget.py.

Project layout

src/prp_local/
  __init__.py
  __main__.py     # python -m prp_local
  cli.py          # argparse entry, config overrides
  config.py       # 4-tier config loader
  budget.py       # token formulas
  lms.py          # OpenAI-compatible HTTP client
  plan.py         # plan generation + plan.md I/O
  core.py         # orchestrator
tests/
  test_budget.py
  test_plan.py
  test_config.py

Development

pip install -e .[dev]
pytest

License

MIT.