How to Transfer Ollama Model Settings Between Hugging Face Models

Posted on by — No Comments ↓

Situation

I had a model called hf.co/aaa/Example-XXX:YYY in Ollama, and I found the model at Hugging Face.

Today, I found a fine-tuned model and tried to use it, but noticed that ollama show does not show TEMPLATE etc, and the output is unexpected (e.g. It does not stop answering etc).

Say, ollama pull names the fine-tuned model as:
hf.co/bbb/FineTuned-Example-XXX:Q4_M.

And you decided to apply the same settings from:
hf.co/aaa/Example-XXX:YYY.

Commands

# Delete the model downloaded
# - I did this, but you can actually keep it and just replace it
#   with a new configuration
ollama rm hf.co/bbb/FineTuned-Example-XXX:Q4_M

mkdir my_modelfiles

cd my_modelfiles

ollama show --modelfile hf.co/aaa/Example-XXX:YYY > example.txt

# Amend "example.txt" to say:
# FROM hf.co/bbb/FineTuned-Example-XXX:Q4_M

# Create the model with the modified file:
ollama create hf.co/bbb/FineTuned-Example-XXX:Q4_M -f example.txt

# Check the model:
ollama show hf.co/bbb/FineTuned-Example-XXX:Q4_M

# Run the model:
ollama run hf.co/bbb/FineTuned-Example-XXX:Q4_M "Hello."

Leave a Reply