Skip to content

[Fix] Fix segmentation fault on concurrent requests #36

Description

@b-re-w

Concurrency guard missing on shared model

  • server.py invokes model.chat() directly for each request without checking whether the shared model is already busy, allowing concurrent execution on a singleton model instance and leading to crashes or segmentation faults. The model itself exposes chat() without any locking mechanism, so simultaneous calls from different sessions can overlap unchecked.

Issue: Reject concurrent requests and fix session initialization

  • Model sessions currently share a single model instance without concurrency control. If a second request arrives while model.chat() is running, the process may fail or segfault. Additionally, Session uses a class-level _initialized flag, so after the first session is created, subsequent sessions skip initialization and lack session_id/model_id.

Expected Behavior

  • If the model is busy processing a request, additional requests should be rejected immediately.
  • Each Session instance should initialize independently and receive unique identifiers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions