JiRackUltra

A fast and efficient ~1.5B model optimized for CPU inference. The model was refactored with BitNet features and an updated tokenizer that includes new Routing, Tool call, and Robotics tags. Built on a redesigned DeepSeek R1 architecture with native ternary (BitNet-style) support and ready-to-run GGUF quantizations.

• JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.

JiRack Ternary Architedure & JiRack Tokenizer

• Benefits high quality CPU inference TQ_2 on Llama.cpp and Ollama via QAT
• Robotcs, Routing, Coding, Multimedia, Advanced tool calling via CMSManhattan/JiRackPrecisionTokenizer

PARTNERSHIP

• NVIDIA
• FISERV

Ollama production support

• We are working to support JiRack on Ollama for production systems also
• added Jirack chat without reasoning feature https://ollama.com/cmsmanhattan
• Follow fresh Ollama platform updates

JiRack sevice options

• Current quantizations were done from the FP16 model, but the model allows for more compression thanks to its ternary architecture.
• If you need to do ternary compression, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.
• Plus double QAT via ONNX QAT.
• Adapt train process to avoid catastrophic forgetting with NDA
• Adapt train process to avoid fast plato in training with NDA
• Convert model to TQ2_0 with support AVX2 and AVX-512 CPU instructions for high performance on CPU
• QAT for TQ_2 Llama.cpp Ternarization docs https://huggingface.co/CMSManhattan/JiRackUltra_1b/blob/main/QAT_to_Llama.cpp_GGUF_TQ2_0_JirackUltra_1b.md
• Adapts to agentic or instruct models for tool calling, using the JiRak tokenizer to enable high-quality tool calling on small models — built as a domain-specific tool expert.
• Deployment and scale

JiRack Codding Agent IDE

• It is Agent Coding IDE for JiRack Models to run via Ollama on home PC
• It good choose for Agent Coding IDE such as Cursor , Windsurf IDE or Devin IDE etc but more safe that ask you to apply changes and review.
• Plugin https://marketplace.eclipse.org/content/jirack-coding-agent
• Web site https://www.jirack.com
• Final release version https://huggingface.co/CMSManhattan/JiRackDeltaNet_27b/resolve/main/jirack_ide_final.zip

Spring Boot AI tool calls examples for JiRack Ultra series

• Tool call library on java for Enterprise https://github.com/alibaba/spring-ai-alibaba

GoEx AI tool calls examples for JiRack Ultra series

• Tool call library on python https://github.com/ShishirPatil/gorilla

JiRack Ultra 1 tool calls to boost tool call quality

• Use JiRack Precision tokenzer tags for tool calls with ToolBench https://github.com/OpenBMB/ToolBench
• https://huggingface.co/xalss/Qwen2-7B-Instruct-glaive-function-calling
• https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1
• Add JiRack tool call tags in the dataset and modify tool call processor if needed

JiRack RoboTech

• Advanced Tokenizer with Robotics & Routing & Tool calls Tokenizer and other
• CMSManhattan/JiRackPrecisionTokenizer

Available Variants

Quick Start

Run with Docker

Default CPU (Q4 recommended)

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q4:latest

Q3

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q3:latest

Q2 (lowest memory)

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16  \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu-q2:latest

Full precision

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16  \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  cmsmanhattan/jirack-ultra-1b-cpu:latest

Multi CPU

docker run -d \
  --name jirack_ultra_1b \
  -p 7869:7869 \
  --cpus=16 \
  -e THREADS=16 \
  -e THREADS_BATCH=16 \
  --restart unless-stopped \
  --memory=4g \
  --cpus=4 \
  cmsmanhattan/jirack-ultra-1b-cpu-q4:latest

Docker Compose Example

services:
  jirack:
    image: cmsmanhattan/jirack-ultra-1b-cpu-q4:latest
    container_name: jirack_ultra_1b
    ports:
      - "7869:7869"
    volumes:
      - .:/app
      - ./web:/app/web
    environment:
      - MAX_TOKENS=2048
      - TEMPERATURE=0.7
      - TOP_P=0.9
      - DEFAULT_STREAM=False
      - INTRA_THREADS=4
      - USE_ENV_ALLOCATOR=1
      - THREADS=16
      - THREADS_BATCH=16
    deploy:
      resources:
        limits:
          memory: 4g

Access the UI

Once the container is running, open your browser and navigate to: http://localhost:7869 This opens the JiRack UI — a clean web interface.

Changing the Port

The listening port can be easily modified directly from the Settings panel within the JiRack UI.

Licensing

• Model weights are released under the MIT License — free to use, modify, and distribute for any purpose, including commercial. No royalties, no per-user fees, no subscription.
• The Docker image with UI and the pre-built Ollama quantizations are separate paid products. I