LFM3

src="https://huggingface.co/LiquidAI/LFM3-1.7B/resolve/main/assets/liquid-ai-banner.webp" alt="Liquid AI" style="width: 100%; max-width: 100%; height: auto; display: block; margin: 0;" />

LFM3-1.7B-GGUF

GGUF version of LFM3-1.7B for local inference on CPUs and GPUs.

Find architecture and usage details in the original model card.

How to run LFM3-1.7B

Recommended sampling: temperature 0.1, top_k 50, repeat penalty 1.1. For math and science: temperature 1.0, top_p 0.95, top_k 20.

• Llama.app (Use this model > Llama in the menu), for macOS and Windows. Requires llama.cpp version v0.8.0 or newer.
• LM Studio (Use this model > LM Studio in the menu), for macOS, Windows and Linux. Requires LM Studio with llama.cpp engine 2.65.1 or newer.
• Ollama (ollama run lfm3), for macOS. Requires Ollama version v0.40.12 or newer.

Use with llama.cpp

[!NOTE] Supported in llama.cpp b11960 (v0.8.0) and newer.

Add a quantization tag to pick a file.

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

For math and science, use --temp 1.0 --top-p 0.95 --top-k 20 instead. More options in the llama.cpp guide.

QAD Q4_0 GGUF

The Q4_0 build in this repository is the Quantization-Aware Distillation (QAD) checkpoint, LFM3-1.7B-QAD-Q4_0.gguf. It was trained to recover the accuracy lost to Q4_0 quantization.

Example usage with llama.cpp:

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF \
  --hf-file LFM3-1.7B-QAD-Q4_0.gguf \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1