The documentation appears to be conflicting on whether or not multiple workers are supported
The deployment modes document seems to imply that you can run multiple workers to improve scaling.
However the ARCHITECTURE.md document appears to contradict that and only a single worker can be used.
The use of SQLite also would imply only a single worker. Testing also appears to confirm a lack of support for multiple workers. I setup a distributed deployment with a dedicated worker, mcp, and web pod. Running multiple web and mcp pods certainly appears to work well, however running multiple replicas with a shared DB or separate DBs definitely causes issues.
The workers are going to be the big load for a central service as they're what are running the scraping and retrieving data.
I'm testing using this as an internal alternative to Context7 or Nia as advertised, but am honestly confused by the sole dependency on SQLite. That's great for running locally, and I could see that as a default. But for running as a central service, supporting a proper database system like PostgreSQL or MongoDB, which both have vector support, would be the best option.
The documentation appears to be conflicting on whether or not multiple workers are supported
The deployment modes document seems to imply that you can run multiple workers to improve scaling.
However the ARCHITECTURE.md document appears to contradict that and only a single worker can be used.
The use of SQLite also would imply only a single worker. Testing also appears to confirm a lack of support for multiple workers. I setup a distributed deployment with a dedicated worker, mcp, and web pod. Running multiple web and mcp pods certainly appears to work well, however running multiple replicas with a shared DB or separate DBs definitely causes issues.
The workers are going to be the big load for a central service as they're what are running the scraping and retrieving data.
I'm testing using this as an internal alternative to Context7 or Nia as advertised, but am honestly confused by the sole dependency on SQLite. That's great for running locally, and I could see that as a default. But for running as a central service, supporting a proper database system like PostgreSQL or MongoDB, which both have vector support, would be the best option.