Documentation

VOICE_PROVIDERS

docs/features/VOICE_PROVIDERS.md

Voice Providers Configuration

Overview

Petunia supports multiple voice providers with intelligent fallback and automatic provider selection based on available API keys.

Primary Provider: Cartesia (STT/TTS)

Cartesia is Petunia's primary voice provider for voice I/O, offering:

  • Cartesia Sonic 3.0: Ultra-realistic text-to-speech (TTS)
  • Cartesia Ink: Speech-to-text (STT) transcription

Required Environment Variables

# Required for Cartesia to work
CARTESIA_API_KEY=sk_car_xxxxxxxxxxxxx

# Optional - override default voice ID
CARTESIA_VOICE_ID=6ccbfb76-1fc6-48f7-b71d-91ac6298247b

# Optional - override default provider selection
DEFAULT_VOICE_PROVIDER=cartesia

Fallback Providers

ElevenLabs

ElevenLabs is used as a fallback when Cartesia is unavailable.

# Required for ElevenLabs fallback
ELEVENLABS_API_KEY=xxxxxxxxxxxxx
# OR
ELEVEN_API_KEY=xxxxxxxxxxxxx

# Optional - override default voice ID
NEXT_PUBLIC_ELEVENLABS_VOICE_ID=EXAVITQu4vr4xnSDxMaL

Retell (Optional / Non-onboarding)

Retell is not used for onboarding. We keep it only for optional, client-specific integrations.

Automatic Provider Selection (Telephony vs Voice I/O)

The system automatically selects the best available telephony provider based on what’s configured:

  1. Twilio (if Twilio env vars are set)
  2. RingCentral (if RingCentral env vars/keys are set)
  3. ElevenLabs (TTS fallback / preview)
  4. Retell (optional, non-onboarding)

Provider Types

  • cartesia: TTS generation using Cartesia Sonic
  • cartesia-ink: Speech-to-text using Cartesia Ink
  • elevenlabs: TTS and conversational AI
  • retell: Phone call capabilities

Usage

API Routes

// Unified voice API - automatically selects best provider
POST /api/communications/voice/unified
{
  "provider": "auto", // or "twilio", "ringcentral", "retell", "elevenlabs"
  "phoneNumber": "+1234567890",
  "text": "Hello, this is Petunia...",
  "contactId": "contact-123"
}

// Cartesia-specific routes
POST /api/voice/cartesia/tts
POST /api/voice/cartesia/ink

Code Usage

import { getUnifiedVoiceService } from '@/lib/services';

const voiceService = await getUnifiedVoiceService();

// Initiate call - automatically uses best available provider
const response = await voiceService.initiateCall({
  phoneNumber: '+1234567890',
  text: 'Hello from Petunia',
  contactId: 'contact-123',
});

Error Handling

The system includes automatic fallback:

  • If Cartesia fails, automatically tries ElevenLabs
  • If primary provider unavailable, uses first available fallback
  • Clear error messages logged when all providers fail

Troubleshooting

"Voice provider not found"

  • Check that at least one API key is set
  • Verify API keys are correct in environment variables
  • Check logs for initialization errors

"Cartesia API key not configured"

  • Set CARTESIA_API_KEY in your environment
  • System will automatically fallback to other providers if available

Onboarding-Specific Contract (IMPORTANT)

Browser Voice (Talk through your computer)

  • STT: Cartesia Ink (POST /api/voice/cartesia/ink)
  • Agent brain: backend route (POST /api/voice/onboarding) which uses our onboarding prompt from lib/config/petunia-personality.ts
  • TTS: Cartesia Sonic-3 (POST /api/voice/cartesia/tts)

Phone Onboarding (Call by phone)

  • Tier A (ship today): turn-based conversational loop using Twilio Voice webhook (/api/webhooks/twilio/voice) + signed <Play> streaming TTS (/api/webhooks/twilio/voice/tts).
  • Tier B (real-time duplex): Twilio Media Streams (WebSocket duplex audio) via realtime-server /twilio-media with streaming STT/TTS and interruption (clear). This is behind env flags.

Key point: Petunia’s agent lives on our backend (LLM + memory + tools). Telephony is transport.

Tier B Env Flags

TWILIO_VOICE_MODE=stream
TWILIO_MEDIA_WS_URL=wss://YOUR_PUBLIC_REALTIME_SERVER_HOST
TWILIO_STREAM_STT=cartesia
TWILIO_STREAM_BRAIN=anthropic
TWILIO_STREAM_TTS=elevenlabs

"No voice provider API keys found"

  • Set at least one of: CARTESIA_API_KEY, ELEVENLABS_API_KEY, or RETELL_API_KEY
  • System will log warnings but attempt to use Cartesia (will fail without key)

Vercel Deployment

All required dependencies are in dependencies (not devDependencies):

  • eslint - Required for Next.js builds
  • @typescript-eslint/parser - Required for ESLint
  • @types/swagger-ui-react - Required for API docs
  • swagger-ui-react - Required for API docs

Make sure these are set in Vercel environment variables:

  • CARTESIA_API_KEY (recommended)
  • ELEVENLABS_API_KEY or ELEVEN_API_KEY (fallback)
  • RETELL_API_KEY (optional, for phone calls)