---
title: "Grok Voice API Tutorial: How to Build a Realtime AI Voice Agent for Customer Support"
description: "A hands-on Grok Voice Agent API tutorial: connect over WebSocket, configure voice and turn detection, wire up function calling for live order lookups, and build a production customer support voice agent in Python."
image: "https://foundrysoft.co/api/og?type=article&title=Grok+Voice+API+Tutorial%3A+How+to+Build+a+Realtime+AI+Voice+Agent+for+Customer+Support&cat=AI+Engineering&rt=10+min+read&au=Varun+Raj+Manoharan&dt=2026-07-25"
url: "https://foundrysoft.co/blog/grok-voice-agent-api-realtime-build"
---

AI Engineering 2026-07-25 10 min read

# Grok Voice API Tutorial: How to Build a Realtime AI Voice Agent for Customer Support

A hands-on Grok Voice Agent API tutorial: connect over WebSocket, configure voice and turn detection, wire up function calling for live order lookups, and build a production customer support voice agent in Python.

![Varun Raj Manoharan](https://foundrysoft.co/images/about/founder.webp)

Varun Raj Manoharan

Grok xAI Voice Agents Realtime WebSocket Python

Every AI voice agent I built before this year was a plumbing project: a speech-to-text stream feeding an LLM feeding a text-to-speech stream, with three vendors, three latency budgets, and a mixing desk of failure modes where they meet. The model never heard the customer sigh. It read a transcript of the sigh.

xAI's Grok Voice Agent API, which got a major upgrade this month alongside the Voice Agent Builder beta, takes the other architecture: one speech-to-speech model, one WebSocket, audio in and audio out. I rebuilt a client's customer support voice agent on it to see whether the single-path promise holds up. It does, and the interesting work moves from plumbing to conversation design. This tutorial walks through the build.

### Connecting to the Grok Voice Agent API

Everything happens over a WebSocket at `wss://api.x.ai/v1/realtime`, with the model selected by query parameter. Use `grok-voice-latest` unless you have a reason to pin a version. Server-side you authenticate with a Bearer key; for browser or mobile clients, mint an ephemeral token instead of shipping your API key to the client.

The first event you send is a `session.update` that configures the whole agent:

Python

Copy

```python
import asyncio, json, websockets

async def run():
    async with websockets.connect(
        "wss://api.x.ai/v1/realtime?model=grok-voice-latest",
        additional_headers={"Authorization": f"Bearer {XAI_API_KEY}"},
    ) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "ara",
                "instructions": SUPPORT_AGENT_PROMPT,
                "turn_detection": {
                    "type": "server_vad",
                    "silence_duration_ms": 600,
                },
                "audio": {
                    "input": {"format": {"type": "audio/pcm", "rate": 24000}},
                    "output": {"format": {"type": "audio/pcm", "rate": 24000}},
                },
                "tools": [ORDER_LOOKUP_TOOL],
            },
        }))
        await asyncio.gather(pump_mic(ws), play_responses(ws))
```

With `server_vad` on, turn-taking becomes the server's problem. You stream microphone audio with `input_audio_buffer.append` and the API decides when the caller has finished speaking. The `silence_duration_ms` knob matters more than anything else in the config; 600ms feels responsive for support calls, while the default is tuned more conservatively. A manual mode exists too (`turn_detection: null` plus explicit commit events) for push-to-talk interfaces.

### Function calling in a voice agent

Our support agent looks up orders mid-conversation. Tools are declared in the session config with JSON schemas, the same mental model as any chat API, plus built-ins for `web_search`, `x_search`, `file_search`, and remote MCP servers. When the model calls your function, you get a `response.function_call_arguments.done` event. You execute, return the output as a conversation item, and ask the model to resume:

Python

Copy

```python
async def handle_event(ws, event):
    if event["type"] == "response.function_call_arguments.done":
        result = await lookup_order(json.loads(event["arguments"]))
        await ws.send(json.dumps({
            "type": "conversation.item.create",
            "item": {
                "type": "function_call_output",
                "call_id": event["call_id"],
                "output": json.dumps(result),
            },
        }))
        await ws.send(json.dumps({"type": "response.create"}))
```

Two details we learned the hard way. When the model fires parallel tool calls, resolve all of them before sending a single `response.create`; sending one per result makes the model start talking with half its facts. And wait for the current audio playback to finish before `response.create`, or the agent talks over itself. In a voice UI these aren't style points. They're the difference between "assistant" and "malfunction."

### force_message: deterministic speech for compliance scripts

My favorite feature in the whole API is the least glamorous one. Support lines have scripted utterances that legal requires verbatim: recording disclosures, identity-verification language. Asking an LLM to say something exactly is famously a coin flip. The `force_message` item type bypasses the model entirely and pushes scripted text straight to speech synthesis:

Python

Copy

```python
await ws.send(json.dumps({
    "type": "conversation.item.create",
    "item": {
        "type": "force_message",
        "role": "assistant",
        "interruptible": False,
        "content": [{"type": "output_text", "text":
            "This call may be recorded for quality and training purposes."}],
    },
}))
```

It completes as a full turn on its own, so don't follow it with `response.create`. Setting `interruptible: False` means the caller can't barge past the disclosure. Per-response instruction overrides give you the same targeted control in the other direction: a `response.create` can carry one-turn instructions, like answering a single turn in Spanish. You get deterministic control exactly where you need it and model judgment everywhere else.

### Session resumption for dropped connections

WebSockets drop, and by default the conversation history drops with them, which for a phone-length call is disqualifying. Session resumption fixes it. Set `resumption: {"enabled": true}` in your session config, capture the `conversation.id` from the server's `conversation.created` event, and on reconnect append `?conversation_id={id}` to the URL. Cached turns replay as `conversation.item.created` events and the agent picks up where it left off. History survives 30 minutes of inactivity, enough for any hold music ever composed. You must opt in on both the original session and the resuming one. We learned that by not doing it.

### Grok Voice API pricing and where this fits

Grok Voice runs at **$0.05 per minute of agent audio**. A support call averaging six minutes costs thirty cents in voice-model spend. Compare that with the fully loaded cost of a human answering the same tier-one "where is my order" call and the math ends quickly, which is exactly why the design bar goes up rather than down. A bad voice agent at scale is cheap frustration, industrially produced.

The single-path architecture is what makes the quality ceiling reachable. The model hears hesitation, interruption, and tone, all things a transcript deletes, and its responses come back with natural prosody that the stitched three-API stack never managed. If you tried the pipeline approach in 2024 and shelved it, the thing you shelved no longer exists. We have a phone number on this prototype now; that build is its own post.

#### Related reading

[Claude Opus 5 Batch API Tutorial: AI Contract Data Extraction at Scale, with Real Costs

A step-by-step tutorial for bulk AI document extraction with the Claude Opus 5 Batch API: structured outputs with JSON schemas, 50% batch pricing, the 300K output beta, and the real cost of processing 40,000 vendor contracts.

Claude Opus 5 Anthropic Batch API

](https://foundrysoft.co/blog/claude-opus-5-batch-api-contract-extraction)[Claude Opus 5 Effort Parameter Guide: How to Reduce Claude API Costs Without Switching Models

How to use the Claude Opus 5 effort parameter (low, medium, high, xhigh, max) to cut Claude API costs. Real per-request cost numbers, working Python code, and why we retired our Haiku/Sonnet/Opus routing layer.

Claude Opus 5 Anthropic Claude API

](https://foundrysoft.co/blog/claude-opus-5-effort-parameter-cost-routing)[How to Build a Long-Running AI Agent with the Claude Opus 5 API: An Overnight Build Log

A hands-on guide to building autonomous AI agents on the Claude Opus 5 API: the agent loop in Python, mid-conversation system messages, prompt caching costs, and what an overnight dependency-upgrade run actually cost.

Claude Opus 5 Anthropic Claude API

](https://foundrysoft.co/blog/claude-opus-5-overnight-long-horizon-agent)

#### Next Article

[

How to Build a Multilingual AI Voice Agent with Grok Voice: Hindi, Spanish, and 20+ Languages

](https://foundrysoft.co/blog/grok-voice-multilingual-support-agent)

Available for new projects

## Let's build something great.

Have a project in mind? We are an elite software and AI development studio ready to bring your ideas to production. Let's talk about your roadmap.

Start a Project [See our work](https://foundrysoft.co/work)

```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "FoundrySoft",
  "url": "https://foundrysoft.co",
  "logo": "https://foundrysoft.co/logo.svg",
  "description": "FoundrySoft builds production-grade software and AI systems for US companies, from an India-based team of senior engineers.",
  "sameAs": [
    "https://github.com/foundrysofthq",
    "https://www.linkedin.com/company/foundrysoft",
    "https://www.instagram.com/foundrysoft/"
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "FoundrySoft",
  "url": "https://foundrysoft.co"
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "Grok Voice API Tutorial: How to Build a Realtime AI Voice Agent for Customer Support",
  "description": "A hands-on Grok Voice Agent API tutorial: connect over WebSocket, configure voice and turn detection, wire up function calling for live order lookups, and build a production customer support voice agent in Python.",
  "url": "https://foundrysoft.co/blog/grok-voice-agent-api-realtime-build",
  "mainEntityOfPage": "https://foundrysoft.co/blog/grok-voice-agent-api-realtime-build",
  "image": [
    "https://foundrysoft.co/api/og?type=article&title=Grok+Voice+API+Tutorial%3A+How+to+Build+a+Realtime+AI+Voice+Agent+for+Customer+Support&cat=AI+Engineering&rt=10+min+read&au=Varun+Raj+Manoharan&dt=2026-07-25"
  ],
  "datePublished": "2026-07-25",
  "dateModified": "2026-07-25",
  "keywords": "Grok, xAI, Voice Agents, Realtime, WebSocket, Python",
  "author": {
    "@type": "Person",
    "name": "Varun Raj Manoharan"
  },
  "publisher": {
    "@type": "Organization",
    "name": "FoundrySoft",
    "logo": {
      "@type": "ImageObject",
      "url": "https://foundrysoft.co/logo.svg"
    }
  }
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://foundrysoft.co/"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://foundrysoft.co/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "Grok Voice API Tutorial: How to Build a Realtime AI Voice Agent for Customer Support",
      "item": "https://foundrysoft.co/blog/grok-voice-agent-api-realtime-build"
    }
  ]
}
```