[BUG] Docker container reports missing Python.h upon startup

[ThoughtProcess]

OS

Linux

GPU Library

CUDA 12.x

Python version

3.12

Describe the bug

When I attempt to run docker compose -f docker/docker-compose.yml up using either the ghcr.io image, or building from the Dockerfile in the repo, the container logs a number of "fatal error: Python.h: No such file or directory" errors when it first starts.

Reproduction steps

1. Clone repo
2. (Optional): edit docker/docker-compose.yml to comment out the image section and uncomment the build section
3. Run docker compose -f docker/docker-compose.yml up

Expected behavior

I would expect the startup log to not show that error.

Logs

Attaching to tabbyapi-1
tabbyapi-1  | 2026-08-19 16:14:51.488 INFO:     Configured backend: cudaMallocAsync
tabbyapi-1  | /tmp/tmpe68t6sye/cuda_utils.c:6:10: fatal error: Python.h: No such file or directory
tabbyapi-1  |     6 | #include <Python.h>
tabbyapi-1  |       |          ^~~~~~~~~~
tabbyapi-1  | compilation terminated.
tabbyapi-1  | /opt/venv/lib/python3.12/site-packages/fla/utils/_device.py:100: UserWarning: Triton is not supported on current platform, roll back to CPU.
tabbyapi-1  |   _cpu_device_warning()
tabbyapi-1  | /tmp/tmp9c2io6ts/cuda_utils.c:6:10: fatal error: Python.h: No such file or directory
tabbyapi-1  |     6 | #include <Python.h>
tabbyapi-1  |       |          ^~~~~~~~~~
tabbyapi-1  | compilation terminated.
tabbyapi-1  | /opt/venv/lib/python3.12/site-packages/fla/utils/_device.py:177: UserWarning: Triton is not supported on current platform, roll back to CPU.
tabbyapi-1  |   _cpu_device_warning()
tabbyapi-1  | /tmp/tmponk2s15n/cuda_utils.c:6:10: fatal error: Python.h: No such file or directory
tabbyapi-1  |     6 | #include <Python.h>
tabbyapi-1  |       |          ^~~~~~~~~~
tabbyapi-1  | compilation terminated.
tabbyapi-1  | /tmp/tmpepoevlye/cuda_utils.c:6:10: fatal error: Python.h: No such file or directory
tabbyapi-1  |     6 | #include <Python.h>
tabbyapi-1  |       |          ^~~~~~~~~~
tabbyapi-1  | compilation terminated.
tabbyapi-1  | /tmp/tmpi7ww2y47/cuda_utils.c:6:10: fatal error: Python.h: No such file or directory
tabbyapi-1  |     6 | #include <Python.h>
tabbyapi-1  |       |          ^~~~~~~~~~
tabbyapi-1  | compilation terminated.
tabbyapi-1  | 2026-08-19 16:14:54.674 INFO:     Your API key is: [redacted]
tabbyapi-1  | 2026-08-19 16:14:54.674 INFO:     Your admin key is:
tabbyapi-1  | [redacted]
tabbyapi-1  | 2026-08-19 16:14:54.674 INFO:     If these keys get compromised, make sure to
tabbyapi-1  | delete api_tokens.yml and restart the server. Have fun!
tabbyapi-1  | 2026-08-19 16:14:54.674 INFO:     Generation logging is disabled
tabbyapi-1  | 2026-08-19 16:14:54.676 INFO:     Applied sampler overrides from file.
tabbyapi-1  | 2026-08-19 16:14:54.676 INFO:     Developer documentation:
tabbyapi-1  | http://0.0.0.0:5000/redoc
tabbyapi-1  | 2026-08-19 16:14:54.677 INFO:     Starting OAI API
tabbyapi-1  | 2026-08-19 16:14:54.677 INFO:     Completions:
tabbyapi-1  | http://0.0.0.0:5000/v1/completions
tabbyapi-1  | 2026-08-19 16:14:54.677 INFO:     Chat completions:
tabbyapi-1  | http://0.0.0.0:5000/v1/chat/completions
tabbyapi-1  | 2026-08-19 16:14:54.680 INFO:     Started server process [1]
tabbyapi-1  | 2026-08-19 16:14:54.681 INFO:     Waiting for application startup.
tabbyapi-1  | 2026-08-19 16:14:54.681 INFO:     Application startup complete.
tabbyapi-1  | 2026-08-19 16:14:54.682 INFO:     Uvicorn running on http://0.0.0.0:5000 (Press
tabbyapi-1  | CTRL+C to quit)

Additional context

As a simple test, I added python3-dev to the initial apt package list in the Dockerfile, as shown below:

# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    curl \
    ca-certificates \
    python3.12 \
    python3-pip \
    python3.12-venv \
    python3-dev \
    && rm -rf /var/lib/apt/lists/*

And then the image built and launched cleanly:

Attaching to tabbyapi-1
tabbyapi-1  | 2026-08-19 16:30:04.829 INFO:     Configured backend: cudaMallocAsync
tabbyapi-1  | 2026-08-19 16:30:08.093 INFO:     Your API key is: [redacted]
tabbyapi-1  | 2026-08-19 16:30:08.093 INFO:     Your admin key is:
tabbyapi-1  | [redacted]
tabbyapi-1  | 2026-08-19 16:30:08.093 INFO:     If these keys get compromised, make sure to
tabbyapi-1  | delete api_tokens.yml and restart the server. Have fun!
tabbyapi-1  | 2026-08-19 16:30:08.094 INFO:     Generation logging is disabled
tabbyapi-1  | 2026-08-19 16:30:08.095 INFO:     Applied sampler overrides from file.
tabbyapi-1  | 2026-08-19 16:30:08.096 INFO:     Developer documentation:
tabbyapi-1  | http://0.0.0.0:5000/redoc
tabbyapi-1  | 2026-08-19 16:30:08.096 INFO:     Starting OAI API
tabbyapi-1  | 2026-08-19 16:30:08.096 INFO:     Completions:
tabbyapi-1  | http://0.0.0.0:5000/v1/completions
tabbyapi-1  | 2026-08-19 16:30:08.097 INFO:     Chat completions:
tabbyapi-1  | http://0.0.0.0:5000/v1/chat/completions
tabbyapi-1  | 2026-08-19 16:30:08.100 INFO:     Started server process [1]
tabbyapi-1  | 2026-08-19 16:30:08.100 INFO:     Waiting for application startup.
tabbyapi-1  | 2026-08-19 16:30:08.101 INFO:     Application startup complete.
tabbyapi-1  | 2026-08-19 16:30:08.101 INFO:     Uvicorn running on http://0.0.0.0:5000 (Press
tabbyapi-1  | CTRL+C to quit)

Distro: Ubuntu 24.04.4 LTS
Kernel: 7.0.0-1016-nvidia
Docker version 29.7.2, build a7dcaa6

Acknowledgements

• I have looked for similar issues before submitting this one.
• I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
• I understand that the developers have lives and my issue will be answered when possible.
• I understand the developers of this program are human, and I will ask my questions politely.