vllm-translategemma

This is a modified version of google/translategemma-4b-it optimized for deployment with vLLM.

Changes from Original Model

Chat Template

The original TranslateGemma model requires a structured payload with dedicated source_lang_code and target_lang_code fields:

{
  "role": "user",
  "content": [
    {
      "type": "text",
      "source_lang_code": "cs",
      "target_lang_code": "de-DE",
      "text": "V nejhorším případě i k prasknutí čočky."
    }
  ]
}

However, vLLM does not support these custom content parameters. To maintain compatibility, the chat template has been modified to encode language codes directly in the message content using a delimiter-based format:

{
  "model": "model",
  "messages": [
    {
      "role": "user",
      "content": ">>cs>>de-DE>>V nejhorším případě i k prasknutí čočky."
    }
  ]
}

Format: >>{source_lang}>>{target_lang}>>{text_to_translate}

If you need to provide a custom prompt input

Format: >>{text}

Model Configuration (RoPE)

The original model uses the new Transformers RoPE configuration format with separate attention type settings:

"rope_parameters": {
  "full_attention": {
    "factor": 8.0,
    "rope_type": "linear"
  },
  "sliding_attention": {
    "rope_type": "default"
  }
}

This has been simplified for vLLM compatibility:

"rope_parameters": {
  "factor": 8.0,
  "rope_type": "linear"
}

Generation Configuration

The EOS token has been corrected from to to ensure proper sequence termination.

Original readme

Resources and Technical Documentation:

• Technical Report
• Responsible Generative AI Toolkit
• TranslateGemma on Vertex Model Garden

Terms of Use: Terms Authors: Google Translate

Model Information

Summary description and brief definition of inputs and outputs.

Description

TranslateGemma is a family of lightweight, state-of-the-art open translation models from Google, based on the Gemma 3 family of models. TranslateGemma models are designed to handle translation tasks across 55 languages. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art translation models and helping foster innovation for everyone.

Inputs and outputs

• Input:

  • Text string, representing the text to be translated
  • Images, normalized to 896 x 896 resolution and encoded to 256 tokens each
  • Total input context of 2K tokens

• Output:

  • Text translated into the target language

Usage

TranslateGemma is designed to work with a specific chat template that supports direct translation of a text input, or text-extraction-and-translation from an image input. This chat template has been implemented with Hugging Face transformers' chat templating system and is compatible with the apply_chat_template() function provided by the Gemma tokenizer and Gemma 3 processor. Notable differences from other models' chat templates include:

• TranslateGemma supports only User and Assistant roles.

• TranslateGemma's User role is highly opinionated:

  • The content property must be provided as a list with exactly one entry.

  • The content list entry must provide:

    • A "type" property where the value must be either "text" or "image".
    • A "source_lang_code" property as a string
    • A "target_lang_code" property as a string

  • The content list entry should provide one of these:

    • A "url" property, if the entry's type is "image", from which the image will be loaded
    • A "text" property, if the entry's type is "text", containing only the text to translate

  • The "source_lang_code" and "target_lang_code" property values can take one of one of two forms:

    • An ISO 639-1 Alpha-2 language code, e.g., en; or
    • A "regionalized" variant as an ISO 639-1 Alpha-2 language code and an ISO 3166-1 Alpha-2 country code pair separated by a dash or an underscore, e.g., en_USoren-GB, similar to the Unicode Common Locale Data Repository format.

  • If the "source_lang_code" and "target_lang_code" property value is not supported by the model, an error will be raised when the template is applied.

With Pipelines

from transformers import pipeline
import torch

pipe = pipeline(
    "image-text-to-text",
    model="google/translategemma-4b-it",
    device="cuda",
    dtype=torch.bfloat16
)

# ---- Text Translation ----
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "source_lang_code": "cs",
                "target_lang_code": "de-DE",
                "text": "V nejhorším případě i k prasknutí čočky.",
            }
        ],
    }
]

output = pipe(text=messages, max_new_tokens=200)
print(output[0]["generated_text"][-1]["content"])

# ---- Text Extraction and Translation ----
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "source_lang_code": "cs",
                "target_lang_code": "de-DE