Kyndreddocs
DashboardGitHub

SDK Reference

A provider-agnostic voice SDK. Program against one interface, swap voice backends without changing your UI code.

Why an SDK?

Voice AI moves fast. Today we use ElevenLabs ConvAI. Tomorrow we might use a custom pipeline with Claude + ElevenLabs TTS directly. The CompanionVoiceProvider interface lets you switch providers with one config change.

The interface

interface CompanionVoiceProvider {
  name: string;
  startSession(config: SessionConfig): Promise<VoiceSession>;
}

interface VoiceSession {
  sendMessage(text: string): void;
  setMicMuted(muted: boolean): void;
  getInputFrequencyData(): Uint8Array | null;
  getOutputFrequencyData(): Uint8Array | null;
  endSession(): Promise<void>;
}

interface SessionConfig {
  agentId: string;
  textOnly?: boolean;
  systemPrompt?: string | null;
  language?: string;
  callbacks: {
    onConnect?: (data: { conversationId: string }) => void;
    onDisconnect?: () => void;
    onMessage?: (data: { message: string; source: 'user' | 'ai' }) => void;
    onModeChange?: (data: { mode: 'speaking' | 'listening' | null }) => void;
    onStatusChange?: (data: { status: VoiceStatus }) => void;
    onError?: (message: string, context?: unknown) => void;
  };
}
ts

Using it

import { getVoiceProvider } from '@/app/lib/voice';

// 1. Pick a provider
const provider = getVoiceProvider('elevenlabs-convai');

// 2. Get an agent ID from the Kyndred API
const res = await fetch('/api/embed/YOUR_TOKEN/voice-token', { method: 'POST' });
const { agent_id } = await res.json();

// 3. Start a session
const session = await provider.startSession({
  agentId: agent_id,
  callbacks: {
    onConnect: () => console.log('connected'),
    onMessage: ({ message, source }) => console.log(source, ':', message),
    onModeChange: ({ mode }) => console.log('mode:', mode),
    onDisconnect: () => console.log('disconnected'),
    onError: (msg) => console.error(msg),
  },
});

// Control the session
session.sendMessage('Hello');
session.setMicMuted(true);
await session.endSession();
ts

Visualization

Both frequency data accessors return Uint8Array from an AnalyserNode. Perfect for building audio visualizations (reactive rings, waveforms, etc.).

// In an animation frame loop:
function draw() {
  const freqData = mode === 'speaking'
    ? session.getOutputFrequencyData()
    : session.getInputFrequencyData();

  if (freqData) {
    // freqData[0..n] = amplitude per frequency bin (0-255)
    const avgAmplitude = freqData.reduce((a, b) => a + b, 0) / freqData.length;
    drawReactiveRing(avgAmplitude / 255);
  }
  requestAnimationFrame(draw);
}
tsx

Available providers

NameStatusBackend
elevenlabs-convaiAvailableElevenLabs ConvAI (STT + LLM + TTS)
kyndredComing soonCustom pipeline: Deepgram + Claude + ElevenLabs TTS

Adding a custom provider

Implement the CompanionVoiceProvider interface and register it:

// my-custom-provider.ts
export class MyCustomProvider implements CompanionVoiceProvider {
  readonly name = 'my-custom';
  async startSession(config: SessionConfig): Promise<VoiceSession> {
    // your implementation
  }
}

// register in app/lib/voice/index.ts
const providers: Record<string, () => CompanionVoiceProvider> = {
  'elevenlabs-convai': () => new ElevenLabsConvAIProvider(),
  'my-custom': () => new MyCustomProvider(),
};
ts

Related