(Maybe?) Implement Rate-Limiting? #2
Description
Use something like slowapi to ensure ppl can't spam the LLMs, ideally you can also use this to figure out how to route to different LLMs but one step at a time.
{{ message }}
Use something like slowapi to ensure ppl can't spam the LLMs, ideally you can also use this to figure out how to route to different LLMs but one step at a time.