Skip to content

Commit 083f987

Browse files
committed
Update RELEASE_NOTES-v3.md
1 parent 24de93c commit 083f987

File tree

1 file changed

+27
-9
lines changed

1 file changed

+27
-9
lines changed

RELEASE_NOTES-v3.md

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,20 @@
11
# v3 Release Notes
22

3+
## Rewritten for extensibility
4+
5+
A major rewrite of llms has been completed to make it extensible and allow for easy addition of new features and providers. Including built-in UI components which have been [refactored into modules](https://github.com/ServiceStack/llms/tree/main/llms/ui/modules) utilizing the same extensibility APIs any extension will be able to use. In addition to adding new features, extensions are also to replace existing components by registering new components with the same name.
6+
7+
Likewise, the server has adopted a server extension model, where major provider implementations can now be dropped into the [llms/providers](https://github.com/ServiceStack/llms/tree/main/llms/providers) folder where it will be automatically loaded and registered at runtime.
8+
9+
This allows [main.py](https://github.com/ServiceStack/llms/blob/main/llms/main.py) to continue to retain a lean functional core in a single file whislt still being extensible.
10+
311
## New Provider Configuration Model
412

5-
The internal provider configuration is now a **superset** of `models.dev/api.json`. Definitions are merged, allowing you to simply enable providers using `"enabled": true` while inheriting all standard provider configurations from `models.dev`.
13+
The most disruptive change is the migration to utilize the same [models.dev](https://models.dev) open provider and model catalogue as used and maintained by [OpenCode](https://opencode.ai).
14+
15+
llms provider configuration is now a **superset** of `models.dev/api.json` where its definitions are merged, allowing you to enable providers using just `"enabled": true` where it inherits all standard provider configurations from `models.dev`.
616

7-
The switch to models.dev greatly expands our model selection to not more than 530 providers from 23 different providers. Including new support for:
17+
The switch to models.dev greatly expands our model selection to more than 530 models from 23 different providers. Including new support for:
818

919
- Alibaba (39 models)
1020
- Chutes (56 models)
@@ -24,11 +34,11 @@ Please raise an issue to add support for any missing providers from [models.dev]
2434

2535
### Up to Date Providers
2636

27-
If your providers.json is older than 1 day, it will be automatically updated when you run `llms`. You can also use the `--update-providers` command at anytime to update your local `providers.json` with the latest provider list from `models.dev`.
37+
This also allows `llms` to automatically update your `providers.json` with the latest provider list from `models.dev` daily. You can also use the `--update-providers` command at anytime to update your local `providers.json` with the latest provider list from `models.dev`.
2838

29-
It filters and saves only the providers that are referenced in your `llms.json`. Any additional providers you want to use that are not included in `models.dev` can be added to your `~/.llms/providers-extra.json` which will be merged into your `providers.json` when updated.
39+
`llms` filters and saves only the providers that are referenced in your `llms.json`. Any additional providers you want to use that are not included in `models.dev` can be added to your `~/.llms/providers-extra.json` which will be merged into your `providers.json` when updated.
3040

31-
This optimization keeps your local configuration file lightweight by only containing the providers you actually use.
41+
This optimization keeps your local configuration file lightweight by only containing the providers that are available for use.
3242

3343
## New Model Selector UI
3444

@@ -128,7 +138,9 @@ Use `map_models` to explicitly whitelist and map specific models. Only the model
128138
```
129139

130140
### 4. Custom Provider & Model Definitions
131-
You can still fully define custom providers and models that aren't in `models.dev`. For example, adding Mistral's free `codestral` endpoint:
141+
142+
You can still fully define custom OpenAI compatible providers and models that aren't in `models.dev`. For example, adding Mistral's free `codestral` endpoint:
143+
132144
```json
133145
"codestral": {
134146
"enabled": true,
@@ -147,6 +159,8 @@ You can still fully define custom providers and models that aren't in `models.de
147159
}
148160
```
149161

162+
If you just want to enable access to new models for existing providers whilst you're waiting for them to be added to models.dev, you can add them to your ``~/.llms/providers-extra.json` where they'll be merged into your `providers.json` when updated.
163+
150164
### 5. NPM SDK Alignment
151165
The provider configuration is now closely aligned with the `models.dev` npm configuration. The `"npm"` field is used to map the provider configuration to the correct Python provider implementation. This generic mapping allows for flexible provider support, including **Anthropic Chat Completion** requests, which are correctly handled for both **Anthropic** and **MiniMax** providers.
152166

@@ -165,16 +179,20 @@ Extensions can be installed from GitHub or by creating a local folder:
165179
Extensions are Python modules that plug into the server lifecycle using special hooks defined in their `__init__.py`:
166180

167181
- **`__parser__(parser)`**: Add custom CLI arguments.
168-
- **`__install__(ctx)`**: Enhance the server instance (e.g., add routes, register providers, filters, etc). `ctx` gives you access to the `ExtensionContext`.
182+
- **`__install__(ctx)`**: Enhance the server instance (e.g., add routes, register providers, request/response filters, etc). `ctx` gives you access to the `ExtensionContext`.
169183
- **`__run__(ctx)`**: Execute custom logic when running in CLI mode.
170184

171185
### How it Works (UI)
172-
Extensions can also include a frontend component.
186+
Extensions can also just include a frontend component.
173187
1. **Placement**: Add a `ui` folder within your extension directory.
174188
2. **Access**: Files in this folder are automatically served at `/ext/<extension_name>/*`.
175189
3. **Integration**: Create a `ui/index.mjs` file. This is the entry point and must export an `install` function:
176190

177191
```javascript
192+
const MyComponent = {
193+
template: `...`
194+
}
195+
178196
// ui/index.mjs
179197
export default {
180198
install(ctx) {
@@ -315,7 +333,7 @@ v3 includes built-in support for audio generation on Google's new TTS models:
315333
- **Gemini 2.5 Flash Preview TTS**
316334
- **Gemini 2.5 Pro Preview TTS**
317335

318-
This is available both in the UI and on the command-line using `--out audio`, e.g:
336+
This is available in both the UI and on the command-line using `--out audio`, e.g:
319337

320338
```bash
321339
llms --out audio "Merry Christmas"

0 commit comments

Comments
 (0)