Hugging Face is a machine learning platform that's home to nearly 500,000 open source models. In this video, I show you how to download, transform, and use them in your local Ollama setup. Get access to the latest and greatest without having to wait for it to be published to Ollama's model library. Let's go!
Check out my other Ollama videos - https://www.youtube.com/playlist?list=PL4041kTesIWby5zznE5UySIsGPrGuEqdB
Links:
Code from the video - https://decoder.sh/videos/importing-open-source-models-to-ollama
Ollama docs for adding a model - https://github.com/ollama/ollama/blob/main/docs/import.md
CapybaraHermes model - https://huggingface.co/argilla/CapybaraHermes-2.5-Mistral-7B
The Bloke GGUF model - https://huggingface.co/TheBloke/CapybaraHermes-2.5-Mistral-7B-GGUF
Ollama Quantization Docker Image - https://hub.docker.com/r/ollama/quantize
Dolphin dataset - https://erichartford.com/dolphin
OpenHermes dataset - https://huggingface.co/datasets/teknium/OpenHermes-2.5
Capybara dataset - https://huggingface.co/datasets/LDJnr/Capybara
Git LFS - https://git-lfs.com/
GGUF - https://github.com/ggerganov/ggml/blob/master/docs/gguf.md
Safetensor - https://huggingface.co/docs/safetensors/index
ChatML - https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B#prompt-format
Timestamps:
00:00 - Intro
00:13 - But Why?
00:57 - Where do new models come from?
01:06 - Who makes models?
01:22 - Our target model
01:37 - What even is a model?
01:50 - GGUF
02:02 - Safetensors
02:18 - Importing GGUF to Ollama
02:33 - TheBloke Quantized Models
03:07 - Creating Modelfile
03:48 - Create Model
03:55 - Run Model
04:21 - DIY Quantization of safetensors
05:12 - Download safetensors
05:56 - Quantization
06:49 - What's Next?
07:09 - Feel better soon