Hugging Face is a machine learning platform that's home to nearly 500,000 open source models. In this video, I show you how to download, transform, and use them in your local Ollama setup. Get access to the latest and greatest without having to wait for it to be published to Ollama's model library. Let's go!
# Modelfile
FROM "./capybarahermes-2.5-mistral-7b.Q4_K_M.gguf"
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
# Create and run model
ollama create capy-hermes -f Modelfile
ollama run capy-hermes
# Pull the model weights and config
git lfs install
git clone https://huggingface.co/argilla/CapybaraHermes-2.5-Mistral-7B
cd CapybaraHermes-2.5-Mistral-7B
docker run --rm -v .:./model ollama/quantize -q q4_K_M /model
# then use q4_K_M.bin GGUF file to create the model in ollama