Skip to content

fix: inspect_ai Model passed API class not instance - #85

Open
andrewwhitecdw wants to merge 1 commit into
NVlabs:mainfrom
andrewwhitecdw:bugfix/evaluator-35a51c50
Open

fix: inspect_ai Model passed API class not instance#85
andrewwhitecdw wants to merge 1 commit into
NVlabs:mainfrom
andrewwhitecdw:bugfix/evaluator-35a51c50

Conversation

@andrewwhitecdw

Copy link
Copy Markdown

This PR addresses the following issue in fouroversix/scripts/ptq/evaluators/evaluator.py: inspect_ai Model passed API class not instance.

Changes

  • fouroversix/scripts/ptq/evaluators/evaluator.py: inspect_ai Model passed API class not instance.

Details

--- a/fouroversix/scripts/ptq/evaluators/evaluator.py
+++ b/fouroversix/scripts/ptq/evaluators/evaluator.py
@@ -1,6 +1,6 @@
-                from .utils import local_hf
-
-                config = GenerateConfig()
-                full_results = inspect_ai.eval(
-                    tasks=tasks,
-                    model=Model(local_hf(model_name, model, config), config, None),
+                from .utils import LocalHuggingFaceAPI
+
+                config = GenerateConfig()
+                full_results = inspect_ai.eval(
+                    tasks=tasks,
+                    model=Model(LocalHuggingFaceAPI(model_name, model, config), config, None),

Tests

Let me know if you want tests added for this fix or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant