Gemma 4 31B

An open-weight model hosted by LiveKit themselves, and the option that honours sampling controls.

Reached through the LiveKit Inference gateway.

temperature and maxTokens are set on the model catalogue entry rather than in a provider body; everything else here is sent to the gateway exactly as written, so these use the upstream spelling. Gemini and Gemma honour different settings, so they are documented separately. There is no list of permitted settings on this route, so a name that is not a chat-completions parameter at all is not caught by OnCore: it fails inside the client LiveKit calls the provider with, before any request goes out, and automatic failover answers that turn instead, so the reply comes from a different model than the one you selected. A value the provider refuses ends the same way, and that is what the If invalid column means on this page. Every language model on this platform runs behind automatic failover, so a refused setting never reaches the caller as an error: another model answers that turn on its own settings, and the reply reads perfectly normally. Measured on 2026-09-03, an in-range frequency_penalty that Gemini refuses outright came back as an ordinary answer written by the failover model, with nothing in it to say so. Send only the settings listed here.

Settings#

Every setting is optional. Leave one out and OnCore sends nothing for it, so the Default applies. If invalid is what the provider does with a value it does not accept.

Overriding on a single call#

Return this from your assistant-request webhook to apply these settings to one call. This is the only route that carries them. A model in your catalogue stores the model id, temperature and maxTokens, and nothing else, so every other setting on this page is sent per call or not at all.

{
  "data": {
    "overrides": {
      "model": {
        "settings": {
          "temperature": 0.8,
          "maxTokens": 250,
          "max_completion_tokens": 2000,
          "top_p": 0.9,
          "frequency_penalty": 0.3,
          "presence_penalty": 0.3,
          "stop": [
            "\n\n"
          ],
          "reasoning_effort": "low"
        }
      }
    }
  }
}

Send only the keys you want. A key you leave out is not sent at all, so the Default applies rather than a stored value. Add "id" beside "settings" to swap the whole model for another one from your catalogue at the same time.

temperature: Genuinely applied on this model, but not range checked: a value outside 0 to 2 is taken without complaint, so keep it in range yourself.

max_completion_tokens: The raw form of maxTokens, for overriding the limit on a single call. max_tokens is accepted here as a synonym. No limit outside the allowed range was ever sent on this model.

top_p: An alternative to temperature for controlling variation. Set one or the other, not both. Not range checked, so keep it within 0 to 1 yourself.

frequency_penalty: Discourages repeating the same wording. Small positive values are enough; high values make the reply read oddly.

presence_penalty: Pushes the model toward new topics, which on a phone call can make it wander off the caller's question.

stop: Generation halts the moment a sequence appears, and the sequence itself is not spoken. Verified live: asking the model to count to six with stop set to four returned one two three. Choose something a caller could never make the assistant say, or it will cut a reply mid-sentence. LiveKit publishes no limit on how many sequences these models take, and the four-sequence cap it does publish belongs to the xAI table, so do not carry that figure across.

reasoning_effort: A switch rather than a dial on this model. Leaving it out, or sending none, keeps reasoning off. Any other value turns it on and roughly quadruples the tokens spent on a turn. The level itself carries no meaning here: low, medium, high and an arbitrary string all produced the same spend, so there is no way to ask for less thinking once it is on.

Hosted by LiveKit#

Unlike every other language model in this section, this one is not served by a model vendor. LiveKit hosts it themselves on LiveKit Inference and tunes the deployment for the low latency a voice agent needs, which is why they recommend it as the default model for voice. There is no separate provider account or key involved, and capacity and rate limits are handled by LiveKit rather than by Google.

Reasoning is a switch here#

reasoning_effort works, but not the way it does on a model with reasoning levels. Measured on one hard question: 381 tokens with the setting left out, 346 with none, then 1,398 at low, 2,000 at medium, 1,765 at high, and 2,000 for an arbitrary string. Turning it on costs roughly four times the tokens; which value you use makes no difference.

seed and tool_choice are accepted without being validated, so neither gives a result you can rely on.

Sampling works here, unlike on Gemini#

This is the one Google model on the platform where temperature and top_p genuinely apply. Asked for the same invented name six times, it returned one distinct answer at temperature: 0 and six at temperature: 2, and one distinct answer at top_p: 0.01. Neither value is range checked, so keep temperature within 0 to 2 and top_p within 0 to 1 yourself.

If an assistant needs consistent, predictable phrasing, this is the Google model that can give it to you.

Next steps#

• Gemini: the previous Google model.
• xAI: language from xAI.
• Providers introduction: per-call overrides, and how a wrong value behaves across providers.