feat: configuration tunables

[thushan]

Extends #164 to add extra tunables to Olla for tweaking things.

Setting	Section	Default	Env var	Description
read_header_timeout	server	10s	OLLA_SERVER_READ_HEADER_TIMEOUT	Max time to read request headers. Guards the inbound listener against Slowloris-style slow-header attacks.
connection_keep_alive	proxy	30s	OLLA_PROXY_CONNECTION_KEEP_ALIVE	TCP keep-alive interval for backend connections. Was previously hardcoded in two places.
response_header_timeout	proxy	30s	OLLA_PROXY_RESPONSE_HEADER_TIMEOUT	Max wait for the backend's first response header. Raise for backends that load models on demand (e.g. Lemonade), where the cold start would otherwise abort at 30s.
Now honoured by both Olla and Sherpa engines (#164 was Olla-only).
tls_handshake_timeout	proxy	10s	OLLA_PROXY_TLS_HANDSHAKE_TIMEOUT	Max time allowed for a TLS handshake with a backend.

All four follow zero-value-means-default, so existing configs are unaffected. Each is also overridable via its OLLA_ env var, consistent with the rest of the config.

server:
  read_header_timeout: 10s

proxy:
  connection_keep_alive: 30s
  response_header_timeout: 30s   # raise for on-demand model loaders (e.g. Lemonade)
  tls_handshake_timeout: 10s

Also in this PR (follow-ups to #164)

• Faxed UpdateConfig dropping ResponseHeaderTimeout/TLSHandshakeTimeout on config reload
• Extended response_header_timeout to the Sherpa engine (was hardcoded)
• Aded the missing keys to the shipped config/config.yaml and the canonical docs config block

Summary by CodeRabbit

• New Features

  • Added configurable server read header timeout, proxy connection keep-alive, proxy response header timeout, and proxy TLS handshake timeout settings; applied to HTTP server and proxy transports.

• Documentation

  • Updated configuration reference and default YAML examples to document new timeout and connection options.

• Tests

  • Added tests covering environment-variable overrides and proxy transport timeout/keep-alive behaviour.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 41ce0097-e3e3-444c-bd99-728cd0be0265

📥 Commits

Reviewing files that changed from the base of the PR and between 54a5665 and 0d2db79.

📒 Files selected for processing (1)

• internal/adapter/proxy/factory.go

🚧 Files skipped from review as they are similar to previous changes (1)

• internal/adapter/proxy/factory.go

---

Walkthrough

Adds configurable server read_header_timeout and proxy connection_keep_alive, response_header_timeout, and tls_handshake_timeout; threads these through config types, app wiring, env overrides, proxy factory/adapters (Sherpa/Olla), service implementations, tests, and documentation.

Changes

Proxy Timeout and Connection Pooling Configuration

Layer / File(s)	Summary
Configuration schema and defaults internal/config/types.go, internal/adapter/proxy/config/unified.go, internal/adapter/proxy/config.go, config/config.yaml	ProxyConfig adds connection_keep_alive and tls_handshake_timeout; BaseProxyConfig adds DefaultTLSHandshakeTimeout and getter; adapter Configuration gains TLSHandshakeTimeout; default YAML includes new fields.
Application-level configuration and environment variable support internal/app/config.go, internal/config/config.go	App constants added for read-header timeout; updateProxyConfiguration respects configured keep-alive and TLS handshake timeout; applyEnvOverrides parses OLLA_SERVER_READ_HEADER_TIMEOUT, OLLA_PROXY_CONNECTION_KEEP_ALIVE, OLLA_PROXY_RESPONSE_HEADER_TIMEOUT, and OLLA_PROXY_TLS_HANDSHAKE_TIMEOUT.
HTTP server and proxy service implementation internal/app/services/http.go, internal/app/services/proxy.go	HTTP server Start() applies configurable ReadHeaderTimeout with 10s fallback; proxy creation uses configurable ConnectionKeepAlive with 30s fallback and populates additional proxy timeout fields.
Proxy adapter factory and Sherpa service internal/adapter/proxy/factory.go, internal/adapter/proxy/sherpa/service.go	Factory wires optional getters for ResponseHeaderTimeout and TLSHandshakeTimeout into Sherpa and Olla configs; Sherpa transport reads timeouts from provided configuration.
Olla service transport and configuration internal/adapter/proxy/olla/service.go	createOptimisedTransport uses config.GetTLSHandshakeTimeout(); UpdateConfig copies/preserves ResponseHeaderTimeout and TLSHandshakeTimeout across updates.
Transport and configuration testing internal/adapter/proxy/olla/service_transport_test.go, internal/config/config_test.go	Added tests for ConnectionKeepAlive getter/default, TLSHandshakeTimeout default and configurable behaviour on transports, UpdateConfig preserving timeouts, and env var parsing for new tunables.
Configuration reference documentation docs/content/configuration/reference.md	Server timeouts section documents read_header_timeout; proxy connection settings expanded with connection_keep_alive, response_header_timeout, and tls_handshake_timeout; default config snippet updated.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• thushan/olla#164: Overlaps on threading response_header_timeout through proxy configuration into transports.
• thushan/olla#59: Earlier unified proxy configuration changes that this PR extends with TLS handshake timeout and getters.

Suggested labels

chore

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'feat: configuration tunables' is vague and generic, using non-descriptive terms that don't convey meaningful information about the specific changes in the changeset.	Consider a more descriptive title that specifies the key tunables added (e.g., 'feat: add configuration tunables for timeouts and connection settings').

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/config-tunables

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/adapter/proxy/factory.go`:
- Around line 50-56: The current single type assertion that requires both
GetTLSHandshakeTimeout and GetResponseHeaderTimeout causes all optional mappings
to be skipped if any single method is missing; update the mapping to perform
separate, independent type assertions (or method-existence checks) for each
optional getter so each optional field on sherpaConfig (e.g.,
TLSHandshakeTimeout, ResponseHeaderTimeout and other optional Olla/pool
tunables) is set only when its specific getter exists — locate the block using
tlsCfg/GetTLSHandshakeTimeout/GetResponseHeaderTimeout and the similar block
around lines 71-83 and replace the combined assertion with per-method checks
that individually assign sherpaConfig fields when the corresponding getter is
present.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 020a9fe6-89b9-48c8-bf34-911fc434660b

📥 Commits

Reviewing files that changed from the base of the PR and between 6b61f56 and 54a5665.

📒 Files selected for processing (14)

• config/config.yaml
• docs/content/configuration/reference.md
• internal/adapter/proxy/config.go
• internal/adapter/proxy/config/unified.go
• internal/adapter/proxy/factory.go
• internal/adapter/proxy/olla/service.go
• internal/adapter/proxy/olla/service_transport_test.go
• internal/adapter/proxy/sherpa/service.go
• internal/app/config.go
• internal/app/services/http.go
• internal/app/services/proxy.go
• internal/config/config.go
• internal/config/config_test.go
• internal/config/types.go