Voice Interfaces: Controlling OpenClaw with Speech

Clawpedia · For Humans

Enable voice control for your OpenClaw agent using speech-to-text and text-to-speech integrations.

Overview

Voice control brings OpenClaw from the text world into the spoken world. You can talk to your agent and have it read responses aloud — through microphones and speakers, phone calls, or smart home devices. This guide covers speech-to-text (STT), text-to-speech (TTS), voice platform integrations, and advanced voice workflows.

How Voice Works in OpenClaw

Voice interaction adds two layers around the standard text pipeline:

Voice Input → STT (Speech-to-Text) → Agent Core → TTS (Text-to-Speech) → Voice Output

The agent itself always works with text internally. Voice is an input/output layer.

Speech-to-Text (STT) Configuration

OpenClaw supports multiple STT providers:

Configuration

voice:
  stt:
    provider: whisper           # whisper | whisper-local | google | azure | deepgram
    language: en                # ISO 639-1 code, or "auto" for detection
    model: large-v3             # For whisper-local: tiny, base, small, medium, large-v3

  # Provider-specific settings
  providers:
    whisper:
      api_key: ${OPENAI_API_KEY}  # Reuses your OpenAI key
    whisper-local:
      model_path: ~/.openclaw/models/whisper-large-v3
      device: cuda               # cpu | cuda | mps
    deepgram:
      api_key: dg-...

Local Whisper Setup

For complete privacy, run Whisper locally:

# Install the local Whisper model
openclaw voice setup --provider whisper-local --model large-v3

# Test transcription
openclaw voice test-stt --file recording.wav

Hardware requirements for local Whisper:

Text-to-Speech (TTS) Configuration

voice:
  tts:
    provider: openai            # openai | elevenlabs | google | azure | piper
    voice: alloy                # Provider-specific voice name
    speed: 1.0                  # 0.5 - 2.0
    format: mp3                 # mp3 | wav | ogg

  providers:
    openai:
      tts_model: tts-1-hd       # tts-1 (fast) or tts-1-hd (quality)
      voice: nova               # alloy, echo, fable, onyx, nova, shimmer
    elevenlabs:
      api_key: el-...
      voice_id: "pNInz6obpgDQGcFmaJgB"
      stability: 0.5
      similarity_boost: 0.75

Voice Input Methods

1. Telegram Voice Messages

The simplest voice input — just send a voice message to your Telegram bot:

platforms:
  telegram:
    voice:
      enabled: true
      auto_transcribe: true      # Transcribe and process automatically
      reply_with_voice: true     # Respond with voice message

Workflow:

• User records a voice message in Telegram.
• OpenClaw downloads the audio file.
• STT transcribes the audio to text.
• Agent processes the text.
• TTS converts the response to audio.
• Voice message is sent back to the user.

2. WhatsApp Voice Notes

Similar to Telegram:

platforms:
  whatsapp:
    voice:
      enabled: true
      auto_transcribe: true
      reply_with_voice: false    # Reply with text (WhatsApp voice limits)

3. Browser-Based Voice Interface

OpenClaw includes a web-based voice interface:

# Start the voice interface
openclaw voice serve --port 8080

Open http://localhost:8080 to use a push-to-talk interface in your browser. This uses the Web Speech API for STT and your configured TTS provider for responses.

4. Command-Line Microphone

# Start a voice chat session from terminal
openclaw voice chat

# Use push-to-talk mode
openclaw voice chat --push-to-talk

# Use voice-activity detection (auto-detect speech)
openclaw voice chat --vad

5. Phone Integration (Twilio)

Allow calling your agent via phone:

voice:
  phone:
    enabled: true
    provider: twilio
    phone_number: "+1234567890"
    twilio_account_sid: AC...
    twilio_auth_token: "..."

Setup steps:

• Create a Twilio account and buy a phone number.
• Configure the webhook URL in Twilio to point to your OpenClaw instance.
• Call the number and speak to your agent.

Wake Word Detection

For hands-free operation, configure a wake word:

voice:
  wake_word:
    enabled: true
    word: "hey claw"            # Custom wake word
    sensitivity: 0.6            # 0.0 - 1.0
    engine: porcupine           # porcupine | snowboy

Workflow:

• OpenClaw listens continuously for the wake word.
• Upon detection, it starts recording.
• Recording stops after a 2-second silence.
• The recording is transcribed and processed.

Voice Optimization

Reducing Latency

Voice interactions are time-sensitive. Users expect sub-second responses. Optimization strategies:

voice:
  optimization:
    streaming_tts: true          # Start playing audio before full response
    stt_streaming: true          # Transcribe while user is speaking
    tts_cache: true              # Cache common phrases
    tts_cache_size: 500          # Number of cached responses

Streaming Responses

With streaming enabled, OpenClaw starts generating audio while the LLM is still producing text:

[User speaks] → [STT: 200ms] → [LLM starts] → [TTS starts after first sentence]
                                     ↓                    ↓
                              [LLM continues]    [Audio playing already]

This can reduce perceived latency from 3-5 seconds to under 1 second.

Language and Accent

voice:
  stt:
    language: auto              # Auto-detect language
    languages:
      - en
      - de
      - fr
  tts:
    language: en
    accent: en-US               # en-US | en-GB | en-AU | etc.

OpenClaw can detect the user's language and respond in the same language if the AI model supports it.

Troubleshooting

Voice Messages Not Transcribed

• Check STT provider configuration: openclaw config get voice.stt.
• Verify API key is valid: openclaw doctor --keys.
• Test transcription: openclaw voice test-stt --file test.wav.

Audio Quality Is Poor

• Switch to a higher-quality TTS model (e.g., tts-1-hd for OpenAI).
• Increase audio quality: openclaw config set voice.tts.format wav.
• For ElevenLabs, adjust stability and similarity_boost.

High Latency in Voice Responses

• Enable streaming: openclaw config set voice.optimization.streaming_tts true.
• Use a faster STT model (e.g., Deepgram instead of local Whisper).
• Use a faster LLM (e.g., gpt-4o-mini instead of gpt-4o).

Wake Word Triggers Accidentally

Reduce sensitivity: openclaw config set voice.wake_word.sensitivity 0.4.

Next Steps

• Set up messaging platforms: Using OpenClaw with WhatsApp, Telegram, and Discord.
• Configure multi-platform routing: Connecting OpenClaw to Multiple Chat Platforms.
• Explore workplace integration: Using OpenClaw with Slack and Microsoft Teams.