Expressive mode

Overview#

Expressive mode lets the language model decide how a reply should sound, not only which words to say. On each turn it adds delivery markup that the voice understands: emotion, pace, pauses, and, on voices that can do it, sounds such as a laugh or a breath. The caller hears the performance. Transcripts and chat history stay as clean text.

One mode applies per assistant. Set settings.conversationConfig.deliveryMode to "expressive". The value is read when the call starts, so a change takes effect on the next call.

This is the mode for Cartesia, Fish Audio, xAI, and Inworld voices. It is a different path from emotion tagging, which classifies a finished reply and injects one tag. Do not turn both on: deliveryMode accepts one value.

The field reference lives on Create and configure an assistant. This page is the walkthrough.

Prerequisites#

• An assistant you can PATCH, or the same settings object on POST /assistants. See Create and configure an assistant.
• A Cartesia, Fish Audio, xAI, or Inworld voice from GET /voices. Store the voice id. Other voices keep their normal delivery. The request still succeeds. See Choosing a voice.
• A voice call. Expressive mode shapes spoken replies. It does not change SMS or web chat.

Pick this mode when#

Use expressive mode when you want the assistant to match the register of the moment: brighter on good news, quieter on bad news, a short laugh only when one is earned.

Stay on "none" when the line must stay flat and predictable.

Use emotion tagging instead when the voice is Cartesia Sonic 3, 3.5, or 3.6, or ElevenLabs eleven_v3, and you want one consistent emotion on each English reply rather than the model inventing its own markup.

Enable it#

Send deliveryMode on create or on a later PATCH. settings merges. You do not have to resend the rest of conversationConfig.

curl -X PATCH https://core-api.heysadie.ai/assistants/0195f1e2-0000-0000-0000-000000000000 \
  -H "Authorization: ApiKey YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "settings": {
      "conversationConfig": {
        "deliveryMode": "expressive"
      }
    }
  }'

A value other than "none", "emotion_tagging", or "expressive" is a 400.

Omitting deliveryMode leaves the assistant on plain delivery, unless the older emotionTaggingEnabled: true flag is still stored and deliveryMode is absent. If both are present, deliveryMode wins.

Tune it with expressiveOptions#

Leaving expressiveOptions out keeps the built-in defaults: light fillers, the full sound vocabulary the voice can render, and a pace the model chooses for the turn.

Only the keys you send override a default. Send expressiveOptions: null to clear every stored override.

The per-category keys on nonverbalSounds are laughing, breathing, sighing, crying, vocalizing, mouthSounds, and reflexSounds. vocalizing is humming or sing-song delivery. mouthSounds is a tsk, tongue-click, or lip-smack. reflexSounds is a cough, throat-clear, or yawn.

A category you omit from the object stays on. A key that is not in that list is a 400.

Each voice renders only the sounds it actually supports. Turning laughing on does not invent a laugh on a voice that cannot produce one.

curl -X PATCH https://core-api.heysadie.ai/assistants/0195f1e2-0000-0000-0000-000000000000 \
  -H "Authorization: ApiKey YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "settings": {
      "conversationConfig": {
        "deliveryMode": "expressive",
        "expressiveOptions": {
          "disfluencies": false,
          "nonverbalSounds": { "laughing": false, "crying": false },
          "pace": "slow",
          "instructionsAppend": "Keep laughter rare. Never sound sarcastic."
        }
      }
    }
  }'

What each voice can do#

The model is told which markup the active voice understands, then those tags are translated into that provider's native form before synthesis. Switching voice therefore changes the available delivery, not only the timbre.

xAI publishes these inline tags: [pause], [long-pause], [hum-tune], [laugh], [chuckle], [giggle], [cry], [tsk], [tongue-click], [lip-smack], [breath], [inhale], [exhale], [sigh]. Wrapping tags include at least <whisper>, <slow>, and <soft>. You do not write these into the prompt when expressive mode is on. The model inserts them. They are here so you know what the caller can hear.

xAI steers delivery through those tags. It does not publish a mood label you can read from a frontend.

Inworld's voice setting deliveryMode (STABLE, BALANCED, CREATIVE) is a different field. It controls how much the Inworld model varies from take to take. It is not settings.conversationConfig.deliveryMode. See inworld-tts-2.

Keep delivery out of the prompt#

Your prompt still owns what the assistant says. Expressive mode owns how it sounds.

Do not write "sound warm", "add ums", or "laugh at jokes" in developerPrompt or a custom instruction. Those lines fight the built-in guide and often make the model write stage directions as words the voice then reads aloud.

If you need an extra rule, put it in expressiveOptions.instructionsAppend. That text is appended to the delivery guide, not to the assistant's task prompt.

Delivery rules that belong in instructionsAppend:

• "Keep laughter rare."
• "Never sound sarcastic."
• "Stay calm when the caller is upset."

Rules that still belong in the prompt:

• What the assistant may promise
• When to transfer
• How to take a booking

See Writing prompts for voice.

What happens when it cannot run#

There is no extra field on the end-of-call report that says expressive mode fired. Confirm it by listening: browser calling is the fastest check.

Limits#

• One deliveryMode per assistant. You cannot combine expressive mode with emotion tagging.
• instructionsAppend is capped at 2000 characters.
• Nonverbal categories the voice cannot render stay silent even when set to true.
• The saved first message is spoken as you wrote it. Expressive markup starts on the model's own replies.
• Cartesia's static voice emotion setting (on the voice body) is a whole-call bias. It is not this mode. See sonic-3.6.

Next steps#

• Emotion tagging: one classifier-picked emotion per English reply on Cartesia Sonic 3, 3.5, 3.6, and ElevenLabs eleven_v3.
• Eleven v3 audio tags: the tag catalogue for eleven_v3, including the set emotion tagging injects.
• Create and configure an assistant: the deliveryMode and expressiveOptions fields next to the rest of conversationConfig.