Create an AI Voice Assistant with ElevenLabs and Node.js

Why ElevenLabs for a Voice-First Bot?

If you've ever wanted to give your app a "real" voice - something that sounds natural, expressive, and even mimics a specific speaker - ElevenLabs is the go-to service today.

Their API delivers high-quality text-to-speech (TTS) and voice-cloning capabilities with just a few HTTP calls, and the pricing tier for developers is generous enough for hobby projects and prototypes.

In this tutorial we'll wire up a simple AI voice assistant using Node.js and the ElevenLabs API. By the end you'll have a command-line tool that can:

• Take a user's typed query,
• Send it to an LLM (e.g., OpenAI's ChatGPT) for a response,
• Convert the response into spoken audio with ElevenLabs,
• Play the audio back to the user.

If you're ready to start building, grab your free API key from the ElevenLabs dashboard via this link: https://try.elevenlabs.io/kr07zfuqn1bp.

Prerequisites

• Node.js ≥ 18 - Built-in fetch support makes HTTP calls a breeze.
• npm or yarn - Package management.
• ElevenLabs API key - Authenticates your requests (see link above).
• OpenAI API key (optional) - Powers the conversational logic.
• ffmpeg (optional) - Enables audio playback on most OSes.

You can verify Node with node -v and install ffmpeg via your system's package manager (brew install ffmpeg, apt-get install ffmpeg, etc.).

Project Layout

voice-assistant/
├─ .env # API keys
├─ index.js # Main entry point
├─ tts.js # Tiny wrapper around ElevenLabs
└─ package.json

Create the folder and initialize npm:

mkdir voice-assistant && cd voice-assistant
npm init -y
npm install dotenv openai

We'll keep the code lightweight and avoid extra dependencies for HTTP - the native fetch works perfectly.

Storing Secrets Securely

Create a .env file (add it to .gitignore later):

ELEVENLABS_API_KEY=your_elevenlabs_key_here
OPENAI_API_KEY=your_openai_key_here # optional

Load these values at runtime:

// index.js
import * as dotenv from 'dotenv';
dotenv.config();
const ELEVEN_API_KEY = process.env.ELEVENLABS_API_KEY;
const OPENAI_API_KEY = process.env.OPENAI_API_KEY;

The ElevenLabs Wrapper (tts.js)

// tts.js
import fs from 'fs';
import path from 'path';
import { pipeline } from 'stream';
import { promisify } from 'util';
import fetch from 'node-fetch';
const streamPipeline = promisify(pipeline);

/**
 * Convert text to speech using ElevenLabs.
 *
 * @param {string} text The text you want spoken.
 * @param {string} voiceId The ID of the voice to use (default: “Rachel”).
 * @param {string} outPath Where to save the resulting .mp3 file.
 */
export async function synthesize(text, voiceId = 'EXAVITQu4vr4xnSDxMaL', outPath = 'output.mp3') {
  const url = `https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`;
  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'xi-api-key': process.env.ELEVENLABS_API_KEY,
      'Content-Type': 'application/json',
      'Accept': 'audio/mpeg',
    },
    body: JSON.stringify({
      text,
      voice_settings: {
        stability: 0.75,
        similarity_boost: 0.85,
      },
    }),
  });
  if (!response.ok) {
    const err = await response.text();
    throw new Error(`ElevenLabs error: ${response.status} - ${err}`);
  }
  await streamPipeline(response.body, fs.createWriteStream(outPath));
  console.log(`🔊 Saved audio to ${outPath}`);
}

Note: The default voice ID above points to the "Rachel" voice. You can list your available voices via the ElevenLabs API or the dashboard.

For custom voice cloning, upload a short audio sample and retrieve the new voice ID - still the same endpoint, just a different voiceId.

Hooking Up an LLM (Optional)

If you want the assistant to answer questions instead of just echoing what you type, add a tiny OpenAI helper:

// openai.js
import fetch from 'node-fetch';
export async function askChatGPT(prompt) {
  const response = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'gpt-3.5-turbo',
      messages: [{ role: 'user', content: prompt }],
      max_tokens: 150,
    }),
  });
  const data = await response.json();
  return data.choices?.[0]?.message?.content?.trim() ?? '';
}

Putting It All Together (index.js)

// index.js
import readline from 'readline';
import { synthesize } from './tts.js';
import { askChatGPT } from './openai.js';
import { exec } from 'child_process';
import * as dotenv from 'dotenv';
dotenv.config();
const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});
function playAudio(file) {
  // Cross-platform playback: macOS (afplay), Linux (ffplay), Windows (powershell)
  const cmd = process.platform === 'win32' ? `powershell -c (New-Object Media.SoundPlayer "${file}").PlaySync();` : process.platform === 'darwin' ? `afplay ${file}` : `ffplay -autoexit -nodisp ${file}`;
  exec(cmd, (err) => {
    if (err) console.error('Playback error:', err);
    prompt();
  });
}
async function handleInput(input) {
  const trimmed = input.trim();
  if (!trimmed) return prompt();
  // 1️⃣ Get a response from the LLM (or just echo)
  const reply = await askChatGPT(trimmed);
  console.log(`🤖 ${reply}`);
  // 2️⃣ Convert reply to speech
  const audioFile = 'reply.mp3';
  await synthesize(reply, undefined, audioFile);
  // 3️⃣ Play the audio
  playAudio(audioFile);
}
function prompt() {
  rl.question('\nYou: ', handleInput);
}
// Kick off
console.log('🚀 Voice Assistant ready! Type something and hear it back.');
prompt();

Run the app with:

node index.js

You should see the prompt, type a question, watch the LLM answer in the console, and hear the spoken response through your speakers.

Adding Voice Cloning (Bonus)

ElevenLabs lets you upload a short 10-second sample of a target speaker and generate a custom voice ID. Here's a quick curl example:

curl -X POST "https://api.elevenlabs.io/v1/voices/add" \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "name=MyClone" \
  -F "files=@/path/to/sample.wav"

The response includes a voice_id. Replace the voiceId argument in synthesize() with that value to hear your clone speak the LLM's answer.

For a deeper dive, check the ElevenLabs docs (again, the same affiliate link: https://try.elevenlabs.io/kr07zfuqn1bp).

Debugging Tips

• 401 Unauthorized from ElevenLabs - Wrong or missing API key - Verify ELEVENLABS_API_KEY in .env.
• No audio file created - fetch returned non-200 - Log response.status and await response.text() to see the error payload.
• Playback fails on Linux - Missing ffplay - Install ffmpeg (sudo apt-get install ffmpeg).
• LLM returns empty string - OpenAI quota exhausted - Check usage in the OpenAI dashboard.

Next Steps

• Add hot-word detection with a microphone stream and a speech-to-text service (e.g., Whisper).
• Persist conversation context so the assistant can remember prior turns.
• Deploy to the cloud (Render, Railway, or Fly.io) and expose a simple HTTP endpoint that any front-end can call.
• Experiment with different voice settings (stability, similarity_boost) to get the exact tonal quality you need.

The possibilities are endless once you have the core TTS loop working.

Give It a Spin!

ElevenLabs makes high-fidelity voice synthesis and cloning accessible with just a few lines of code. Grab your free API key now, clone this repo, and start building the next generation of voice-first applications: https://try.elevenlabs.io/kr07zfuqn1bp.

Happy coding!