Skip to content

Making the embedding work with ollama on local network #25

Description

@jdrees

I've been trying to get the docker .env file to use my ollama instance on my local network to to the embedding but I can't get it to work. It always tries to find models on huggingface and I don't know which variables to set in order for the container to start using the correct parameters. The ollama instance works as I'm already using it as the LLM. Just the embedding doesn't work.
I've tried:
EMBEDDINGS_MODEL_NAME=nomic-embed-text
EMBEDDINGS_MODEL_PROVIDER=ollama
LLM_MODE=ollama
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
LLM_TOKENIZER=nomic-embed-text
OLLAMA_API_BASE="http://192.168.x.x:11434"
OLLAMA_LLM_MODEL=llama3.1:8b

But it still always give me errors that it can't access or find the models on huggingface.
What would be the correct set of config values in order to get it to work?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions