Skip to content

Add memra as a model library - #2382

Merged
Wauplin merged 2 commits into
huggingface:mainfrom
avifenesh:add-memra-library
Aug 26, 2026
Merged

Add memra as a model library#2382
Wauplin merged 2 commits into
huggingface:mainfrom
avifenesh:add-memra-library

Conversation

@avifenesh

@avifenesh avifenesh commented Aug 17, 2026

Copy link
Copy Markdown

Adds memra to model-libraries.ts with a snippet, per Adding a library to the Hub.

What it is. A from-scratch inference engine in Rust + CUDA for NVIDIA Blackwell workstation and consumer cards (sm_120a, with a compile-gated Hopper lane), serving GGUF and safetensors over an OpenAI-compatible HTTP API. MIT. I maintain it.

Prerequisite. A model already carries the tag: ?filter=memra returns Avifenesh/Qwen3.8-27B-NVFP4-MTP-GGUF (~3k downloads).

filter: false — one tagged model today, far under the 100-model bar for the models-page filter.

countDownloads: path_extension:"gguf" OR path:"config.json". Two artifact shapes, one counted file each:

  • GGUF files are self-contained, so one file is one load — the same reasoning behind the Hub's own GGUF special case, and the pattern rkllm and litert-lm already use.
  • safetensors checkpoints are counted by config.json, which is one per repo. Counting path_extension:"safetensors" would overestimate a sharded model by its shard count, which the docs explicitly warn against.

Snippet. Three short blocks:

  1. Install, with the hardware requirement stated first — Blackwell-class card, Linux x86_64, driver 580+, no CUDA toolkit — so nobody installs it expecting an unsupported card to work.
  2. One chat-templated generation via the engine's hf: resolver, noting that a repo containing several GGUF files needs :<substring> to pick one (which is exactly what the resolver's error asks for).
  3. The OpenAI-compatible server, one line.

Checks. In packages/tasks: pnpm run check (tsc) clean, pnpm test 22 passed, pnpm run format:check clean.


Note

Low Risk
Additive registry and static snippet strings in packages/tasks; no auth, inference, or download-count logic changes in this diff.

Overview
Registers memra as a Hub model library so repos tagged with library_name: memra get the standard model-page treatment (label, repo link, usage snippets). The library is not added to the models-page filter (filter: false), consistent with a small catalog.

Model pages show three snippets: install via the upstream install.sh (with Blackwell / driver notes), one-shot chat generation with run-gen and hf:${model.id} (including multi-GGUF :<substring> selection), and an OpenAI-compatible memra-server one-liner bound to the current model.

Reviewed by Cursor Bugbot for commit 88f4041. Bugbot is set up for automated code reviews on this repo. Configure here.

memra is a Rust + CUDA inference engine for NVIDIA Blackwell workstation and
consumer cards, serving GGUF and safetensors over an OpenAI-compatible API. I
maintain it.

- filter: false — one model carries the tag today, well under the 100-model bar.
- countDownloads: GGUF files are self-contained, so one file is one load;
  safetensors checkpoints are counted by config.json, which is one per repo,
  because counting shards would overestimate a sharded model by its shard count.
- The snippet leads with the hardware requirement, so nobody installs it expecting
  it to run on an unsupported card, and shows both the one-shot generation and the
  server. It also notes that a repo containing several GGUF files needs a
  :<substring> to pick one, which is what the engine's hf: resolver asks for.

pnpm check, pnpm test and pnpm format:check all pass in packages/tasks.
Comment thread packages/tasks/src/model-libraries.ts Outdated

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good thank you!

Expect 2-3 days before getting it live on the Hub 🤗

@Wauplin
Wauplin merged commit a121260 into huggingface:main Aug 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants