Add ANEForge to the model libraries registry - #2347
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b1a2df8. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1a2df87a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
PR bug bot suggestions were mostly correct and now fixed. Maintainers please have a peek when you get a moment. Thanks! |
|
Hi @sbryngelson, super cool project, congrats! 🙌 However, the Hub integration you propose is unusual. Repos are intended to host model weights that users can download and run with the associated library; in your case, the four repos you created only contain usage instructions. I understand that ANEForge can run many existing architectures by dynamically creating a graph for the ANE, which is quite impressive. To bootstrap usage of your library, perhaps you could duplicate a few representative model repos (whose license allows) under your namespace, assign them the I would also recommend you include the four snippets in your GitHub's readme page, as they demonstrate a super low-friction way to test your library with many models from the Hub. Beyond the context of this PR, I'd also be curious to know what kind of adoption and interest you are seeing. The ANE remains the unreachable but highly-efficient compute unit in Apple Silicon, and there appear to be more experimental efforts trying to leverage it through private APIs (oMLX, for instance). |
|
Thanks, this was helpful. I've made the changes. The repos were unusual for the reason you spotted: ANEForge doesn't ship its own weights. It compiles an existing Hub model's architecture into an ANE program on the fly and pulls that model's weights from its original repo via huggingface_hub, so the four aneforge/* repos were instructions rather than weights. I've duplicated four permissively-licensed models under the namespace, weights unchanged, each tagged library_name: aneforge with attribution and the source license:
Each loads and runs on the Neural Engine straight from its aneforge/ repo id, verified on an M5 Pro. Happy to grow this as a community effort. Since those span four tasks, I updated the snippet in this PR to branch on pipeline_tag, so a text-generation, image-classification, or ASR model shows the right ANEForge call instead of the embeddings one. I also added a section to the GitHub README with the four snippets, the low-friction path you pointed out: https://github.com/sbryngelson/ANEForge#run-a-hub-model-on-the-neural-engine On adoption: still early and research-stage. The recent work has been about lowering friction, a sentence-transformers drop-in and a langchain-aneforge package on PyPI, this Hub integration, and a reverse-engineering guide of the engine on arXiv, plus a community effort to map the ANE roofline across chips. Interest so far is mostly from the Apple-silicon and reverse-engineering side rather than production users. I'm aware of oMLX and a few other private-API efforts; the shared motivation is that the ANE is the efficient unit on Apple silicon but reachable only through CoreML. I'd welcome your thoughts on where a library like this is most useful to the Hub community. Thanks again. |
pcuenca
left a comment
There was a problem hiding this comment.
Thanks for iterating @sbryngelson 🙌 This looks good to me, but let's maybe wait for another reviewer before merging.
ANEForge (https://github.com/sbryngelson/ANEForge) runs computation on the Apple Neural Engine directly, without CoreML. Its sentence-transformers drop-in loads any Hub model by repo id and runs the encoder on the engine, so this adds the library entry + a 'Use this model' snippet.
f7ba64a to
1778f89
Compare
|
@pcuenca rebased on master. no other reviewers yet 😮💨 |
|
Thank you, Spencer! |
…2433) Follow-up to #2347 (which added the `aneforge` library entry). The `aneforge` snippet branched on `text-generation`, `image-classification`, and `automatic-speech-recognition`, falling back to the sentence-transformers drop-in for everything else. CLIP models (`zero-shot-image-classification`) hit that fallback and rendered the sentence-transformers snippet, which does not apply to them. This adds a CLIP branch dispatching to `af.load_clip`, plus a test for the per-pipeline dispatch. This now covers every model type ANEForge can load from a Hub repo id: text-generation (`load_llm`), zero-shot-image-classification (`load_clip`), image-classification (`load_vit`), automatic-speech-recognition (`load_whisper`), and sentence-similarity / feature-extraction (the sentence-transformers drop-in). I ran each snippet verbatim against a live model in the `aneforge` org on the Neural Engine to confirm it works as written: - `aneforge/gpt2` -> `load_llm` generates text - `aneforge/clip-vit-base-patch32` -> `load_clip().classify(image, [...])` returns sorted (label, prob) - `aneforge/vit-base-patch16-224` -> `load_vit().classify(image)` returns top-k (label, logit) - `aneforge/whisper-tiny.en` -> `load_whisper().transcribe(audio)` returns text - `aneforge/all-MiniLM-L12-v2` -> `SentenceTransformer().encode(..., normalize_embeddings=True)` The org now has 9 tagged models across these pipelines. I left `filter: false` as-is, since the field is documented for libraries with >100 models. `pnpm --filter @huggingface/tasks test` (snippets spec) passes, and `oxfmt --check` / `eslint` are clean. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Changes only affect generated model-card code samples in `@huggingface/tasks`, with no auth, inference, or data-path impact. > > **Overview** > **ANEForge Hub snippets** now route CLIP / zero-shot image models to `af.load_clip` instead of the sentence-transformers fallback. > > The `aneforge` snippet generator gains a branch when `pipeline_tag` is `zero-shot-image-classification` or the model is tagged `clip`, emitting a short example that calls `clip.classify(image, labels)`. A Vitest case asserts dispatch for text-generation (`load_llm`), zero-shot CLIP (`load_clip`), and the sentence-similarity default (`SentenceTransformer`). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f312d90. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->

This adds ANEForge to the model-libraries registry, with a matching "Use this model" snippet.
What ANEForge is
ANEForge runs computation on the Apple Neural Engine (ANE) directly, without CoreML. It compiles a lazy tensor graph into a single ANE program and dispatches it through the same daemon/kernel-driver stack Apple's own frameworks use.
It integrates with the Hub through a
sentence_transformers-compatible drop-in that loads any Hub model by repo id and runs the encoder on the engine:Checklist
pip install aneforge)huggingface_hub.hf_hub_downloadunder the hood)Changes
model-libraries.ts:aneforgeentry (alphabetical, betweenallennlpandanemoi),filter: true.model-libraries-snippets.ts: ananeforgesnippet that loads the model with the sentence-transformers drop-in and runs.encode()on the ANE.I matched the existing tab/quote/trailing-comma style by hand but was not able to run the full monorepo toolchain locally; happy to adjust formatting or the snippet to your conventions.
Note
Low Risk
Adds only static registry metadata and a display snippet; no changes to auth, downloads, or inference paths in this repo.
Overview
Registers ANEForge in the Hub model-libraries registry so models tagged with
library_name: aneforgeget the correct label, docs/repo links, and Use this model code on model pages.The new entry sits alphabetically between AllenNLP and AnemoI, with
filter: trueso ANEForge appears in the models library filter. The snippet shows loading a Hub repo viaaneforge.sentence_transformers.SentenceTransformerand running.encode()on the Apple Neural Engine (no CoreML).Reviewed by Cursor Bugbot for commit b1a2df8. Configure here.