Voice Chat

Add immersive voice communication to your multiplayer games with spatial audio, push-to-talk, voice channels, and privacy controls powered by WebRTC.

Voice chat enhances multiplayer experiences by enabling real-time communication between players. Web Engine provides a comprehensive voice chat system built on WebRTC with support for spatial audio, push-to-talk, voice channels, and advanced audio processing.

Voice Chat Features#

Spatial Voice

3D positional audio that attenuates with distance for immersive communication.

Push-to-Talk

Hold-to-speak controls with configurable keybindings for precise voice activation.

Voice Channels

Separate voice channels for teams, proximity chat, and global communication.

Audio Processing

Noise suppression, echo cancellation, and automatic gain control.

Privacy Controls

Mute, block, and volume controls for individual players.

Customizable

Configure audio quality, codec, bitrate, and processing settings.

Voice Chat Setup#

Voice chat uses WebRTC for peer-to-peer or server-relayed audio streaming. The setup involves requesting microphone permissions, creating audio streams, and establishing peer connections:

Basic Voice Chat Setup#

Initialize Voice Chat

typescript

import { VoiceChatManager } from '@web-engine-dev/core/network';

// Create voice chat manager
const voiceChat = new VoiceChatManager({
  spatialAudio: true,          // Enable 3D positional audio
  maxDistance: 50,             // Voice audible within 50m
  rolloffFactor: 1.0,          // Audio attenuation rate
  audioConstraints: {
    echoCancellation: true,
    noiseSuppression: true,
    autoGainControl: true
  }
});

// Request microphone permission
async function initializeVoice() {
  try {
    await voiceChat.requestMicrophonePermission();
    console.log('Microphone access granted');

    // Create local audio stream
    await voiceChat.createLocalStream();
    console.log('Local audio stream created');

    // Connect to voice server
    await voiceChat.connect(network.getRoomId());
    console.log('Connected to voice chat');

  } catch (error) {
    console.error('Voice chat initialization failed:', error);
    showMicrophonePermissionError();
  }
}

// Call on game start
initializeVoice();

Browser Permissions

Voice chat requires microphone permissions. Always request permissions with clear user prompts explaining why voice chat is being activated. Browsers may block automatic microphone access without user interaction.

WebRTC Peer Connections#

Voice chat establishes peer-to-peer connections using WebRTC. The signaling process is handled through the game server (Colyseus):

WebRTC Signaling Flow

typescript

import { NetworkManager, PacketType } from '@web-engine-dev/core/network';

class VoiceChatManager {
  private peerConnections = new Map<string, RTCPeerConnection>();
  private audioElements = new Map<string, HTMLAudioElement>();

  // Create peer connection for remote player
  async createPeerConnection(peerId: string): Promise<RTCPeerConnection> {
    const config: RTCConfiguration = {
      iceServers: [
        { urls: 'stun:stun.l.google.com:19302' },
        { urls: 'stun:stun1.l.google.com:19302' }
      ]
    };

    const pc = new RTCPeerConnection(config);

    // Add local audio tracks
    const localStream = this.localStream;
    if (localStream) {
      localStream.getTracks().forEach(track => {
        pc.addTrack(track, localStream);
      });
    }

    // Handle incoming audio tracks
    pc.ontrack = (event) => {
      console.log('Received remote audio track from', peerId);
      this.handleRemoteTrack(peerId, event.streams[0]);
    };

    // Handle ICE candidates
    pc.onicecandidate = (event) => {
      if (event.candidate) {
        // Send ICE candidate to peer via game server
        network.sendPacket({
          type: PacketType.ICE_CANDIDATE,
          senderId: network.getClientId(),
          timestamp: network.getServerTime(),
          targetPeerId: peerId,
          candidate: event.candidate
        });
      }
    };

    // Monitor connection state
    pc.onconnectionstatechange = () => {
      console.log(`Voice connection to ${peerId}: ${pc.connectionState}`);

      if (pc.connectionState === 'failed') {
        console.error('Voice connection failed, attempting ICE restart');
        pc.restartIce();
      }
    };

    this.peerConnections.set(peerId, pc);
    return pc;
  }

  // Create and send offer
  async createOffer(peerId: string): Promise<void> {
    const pc = await this.createPeerConnection(peerId);

    const offer = await pc.createOffer({
      offerToReceiveAudio: true,
      offerToReceiveVideo: false
    });

    await pc.setLocalDescription(offer);

    // Send offer to peer via game server
    network.sendPacket({
      type: PacketType.SDP_OFFER,
      senderId: network.getClientId(),
      timestamp: network.getServerTime(),
      targetPeerId: peerId,
      sdp: offer
    });
  }

  // Handle incoming offer
  async handleOffer(peerId: string, offer: RTCSessionDescriptionInit): Promise<void> {
    const pc = await this.createPeerConnection(peerId);

    await pc.setRemoteDescription(offer);

    const answer = await pc.createAnswer();
    await pc.setLocalDescription(answer);

    // Send answer to peer
    network.sendPacket({
      type: PacketType.SDP_ANSWER,
      senderId: network.getClientId(),
      timestamp: network.getServerTime(),
      targetPeerId: peerId,
      sdp: answer
    });
  }

  // Handle incoming answer
  async handleAnswer(peerId: string, answer: RTCSessionDescriptionInit): Promise<void> {
    const pc = this.peerConnections.get(peerId);
    if (pc) {
      await pc.setRemoteDescription(answer);
    }
  }

  // Handle ICE candidate
  async handleIceCandidate(peerId: string, candidate: RTCIceCandidateInit): Promise<void> {
    const pc = this.peerConnections.get(peerId);
    if (pc) {
      await pc.addIceCandidate(candidate);
    }
  }
}

// Listen for WebRTC signaling messages
network.on('sdpOffer', async (packet) => {
  await voiceChat.handleOffer(packet.senderId.toString(), packet.sdp);
});

network.on('sdpAnswer', async (packet) => {
  await voiceChat.handleAnswer(packet.senderId.toString(), packet.sdp);
});

network.on('iceCandidate', async (packet) => {
  await voiceChat.handleIceCandidate(packet.senderId.toString(), packet.candidate);
});

Spatial Voice Audio#

Spatial voice audio creates immersive 3D positional audio where voice volume and direction change based on player positions:

3D Positional Audio

typescript

import { SpatialAudioSystem } from '@web-engine-dev/core/audio';
import { Transform } from '@web-engine-dev/core';

// Create spatial audio system
const spatialAudio = new SpatialAudioSystem(audioContext);

// Configure distance model for voice
const voiceDistanceModel = {
  type: 'linear' as const,
  refDistance: 1,        // Full volume within 1m
  maxDistance: 50,       // Inaudible beyond 50m
  rolloffFactor: 1.0     // Linear falloff
};

// Set up spatial audio for remote player
function setupSpatialVoice(peerId: string, audioStream: MediaStream) {
  // Create audio element
  const audio = new Audio();
  audio.srcObject = audioStream;
  audio.play();

  // Create spatial audio source
  const source = audioContext.createMediaStreamSource(audioStream);
  const panner = audioContext.createPanner();

  // Configure 3D audio
  panner.panningModel = 'HRTF';
  panner.distanceModel = voiceDistanceModel.type;
  panner.refDistance = voiceDistanceModel.refDistance;
  panner.maxDistance = voiceDistanceModel.maxDistance;
  panner.rolloffFactor = voiceDistanceModel.rolloffFactor;

  // Connect audio graph
  source.connect(panner);
  panner.connect(audioContext.destination);

  // Store for updates
  spatialAudioSources.set(peerId, { audio, panner, source });
}

// Update spatial audio every frame
function updateSpatialVoice(world: IWorld, cameraEid: number) {
  // Update listener position from camera
  spatialAudio.updateListener(camera);

  // Update each remote player's voice position
  spatialAudioSources.forEach((audioSource, peerId) => {
    const eid = findEntityByPeerId(peerId);
    if (!eid) return;

    // Get player position
    const position = new Vector3(
      Transform.position[eid]![0],
      Transform.position[eid]![1],
      Transform.position[eid]![2]
    );

    // Update audio source position
    const panner = audioSource.panner;
    const time = audioContext.currentTime;

    if (panner.positionX) {
      panner.positionX.setValueAtTime(position.x, time);
      panner.positionY.setValueAtTime(position.y, time);
      panner.positionZ.setValueAtTime(position.z, time);
    } else {
      panner.setPosition(position.x, position.y, position.z);
    }
  });
}

// Call in render loop
function onRender(world: IWorld, cameraEid: number) {
  updateSpatialVoice(world, cameraEid);
}

Spatial Voice Distance

Tune maxDistance based on your game's scale. For close-quarters combat, use 20-30m. For open-world games, use 50-100m. This prevents voice chat from spanning the entire map while maintaining immersion.

Push-to-Talk (PTT)#

Push-to-Talk Implementation

typescript

class VoiceChatManager {
  private isPushToTalkEnabled = true;
  private isTalking = false;
  private pushToTalkKey = 'KeyV'; // V key

  // Enable/disable microphone based on PTT state
  setMicrophoneEnabled(enabled: boolean) {
    if (!this.localStream) return;

    this.localStream.getAudioTracks().forEach(track => {
      track.enabled = enabled;
    });

    this.isTalking = enabled;

    // Visual feedback
    if (enabled) {
      showTalkingIndicator();
    } else {
      hideTalkingIndicator();
    }
  }

  // Handle PTT key press
  onKeyDown(event: KeyboardEvent) {
    if (event.code === this.pushToTalkKey && this.isPushToTalkEnabled) {
      if (!this.isTalking) {
        this.setMicrophoneEnabled(true);
        console.log('Push-to-talk: Started talking');
      }
    }
  }

  // Handle PTT key release
  onKeyUp(event: KeyboardEvent) {
    if (event.code === this.pushToTalkKey && this.isPushToTalkEnabled) {
      if (this.isTalking) {
        this.setMicrophoneEnabled(false);
        console.log('Push-to-talk: Stopped talking');
      }
    }
  }

  // Toggle PTT mode
  setPushToTalkEnabled(enabled: boolean) {
    this.isPushToTalkEnabled = enabled;

    if (!enabled) {
      // Always-on mode
      this.setMicrophoneEnabled(true);
    } else {
      // PTT mode (default off)
      this.setMicrophoneEnabled(false);
    }
  }

  // Configure PTT key
  setPushToTalkKey(keyCode: string) {
    this.pushToTalkKey = keyCode;
  }
}

// Set up keyboard listeners
window.addEventListener('keydown', (e) => voiceChat.onKeyDown(e));
window.addEventListener('keyup', (e) => voiceChat.onKeyUp(e));

// Settings UI
function renderVoiceSettings() {
  return (
    <div>
      <label>
        <input
          type="checkbox"
          checked={voiceChat.isPushToTalkEnabled}
          onChange={(e) => voiceChat.setPushToTalkEnabled(e.target.checked)}
        />
        Enable Push-to-Talk
      </label>

      <label>
        PTT Key:
        <input
          type="text"
          value={voiceChat.pushToTalkKey}
          onChange={(e) => voiceChat.setPushToTalkKey(e.target.value)}
        />
      </label>
    </div>
  );
}

Voice Channels#

Voice channels allow segregating voice communication by teams, proximity, or broadcast:

Channel Type	Description	Use Case
Global	All players in room can hear	Lobby chat, announcements
Team	Only team members can hear	Team coordination, strategy
Proximity	Players within distance can hear	Spatial chat, local interactions
Direct	One-to-one private communication	Private messages, whispers

Voice Channel System

typescript

enum VoiceChannel {
  GLOBAL = 'global',
  TEAM = 'team',
  PROXIMITY = 'proximity',
  DIRECT = 'direct'
}

class VoiceChannelManager {
  private currentChannel: VoiceChannel = VoiceChannel.PROXIMITY;
  private teamId: string | null = null;

  // Check if we can hear player in current channel
  canHearPlayer(peerId: string, playerData: PlayerData): boolean {
    switch (this.currentChannel) {
      case VoiceChannel.GLOBAL:
        // Hear all players
        return true;

      case VoiceChannel.TEAM:
        // Only hear teammates
        return playerData.teamId === this.teamId;

      case VoiceChannel.PROXIMITY:
        // Only hear nearby players (handled by spatial audio)
        const distance = getDistanceToPlayer(peerId);
        return distance <= spatialAudioConfig.maxDistance;

      case VoiceChannel.DIRECT:
        // Only hear specific player (1v1 chat)
        return peerId === this.directChatTarget;

      default:
        return false;
    }
  }

  // Switch voice channel
  setChannel(channel: VoiceChannel) {
    console.log(`Switching to ${channel} voice channel`);
    this.currentChannel = channel;

    // Update audio routing
    this.updateAudioConnections();

    // UI feedback
    showChannelIndicator(channel);
  }

  // Update which peers we're connected to
  updateAudioConnections() {
    const allPeers = getAllConnectedPeers();

    allPeers.forEach(peerId => {
      const playerData = getPlayerData(peerId);
      const shouldHear = this.canHearPlayer(peerId, playerData);

      const audioSource = spatialAudioSources.get(peerId);
      if (audioSource) {
        // Mute/unmute based on channel rules
        audioSource.audio.muted = !shouldHear;
      }
    });
  }
}

// UI for channel switching
function renderChannelSelector() {
  return (
    <div className="voice-channels">
      <button onClick={() => channelManager.setChannel(VoiceChannel.GLOBAL)}>
        Global
      </button>
      <button onClick={() => channelManager.setChannel(VoiceChannel.TEAM)}>
        Team
      </button>
      <button onClick={() => channelManager.setChannel(VoiceChannel.PROXIMITY)}>
        Proximity
      </button>
    </div>
  );
}

// Hotkey for quick channel switching
window.addEventListener('keydown', (e) => {
  if (e.code === 'Digit1') channelManager.setChannel(VoiceChannel.GLOBAL);
  if (e.code === 'Digit2') channelManager.setChannel(VoiceChannel.TEAM);
  if (e.code === 'Digit3') channelManager.setChannel(VoiceChannel.PROXIMITY);
});

Audio Processing & Quality#

Audio Processing Configuration

typescript

// Configure audio constraints for high-quality voice
const audioConstraints: MediaTrackConstraints = {
  // Noise reduction
  noiseSuppression: true,
  echoCancellation: true,
  autoGainControl: true,

  // Quality settings
  sampleRate: 48000,        // 48kHz sample rate
  channelCount: 1,          // Mono for voice
  latency: 0.01,            // Low latency (10ms)

  // Optional: advanced constraints
  sampleSize: 16,           // 16-bit audio
};

// Create high-quality audio stream
async function createHighQualityStream() {
  const stream = await navigator.mediaDevices.getUserMedia({
    audio: audioConstraints,
    video: false
  });

  // Apply additional processing
  const audioContext = new AudioContext({ sampleRate: 48000 });
  const source = audioContext.createMediaStreamSource(stream);

  // Optional: Add noise gate to reduce background noise
  const noiseGate = createNoiseGate(audioContext, {
    threshold: -50,    // dB threshold
    attack: 0.01,      // Fast attack
    release: 0.1       // Moderate release
  });

  source.connect(noiseGate);
  noiseGate.connect(audioContext.destination);

  return stream;
}

// Voice Activity Detection (VAD)
class VoiceActivityDetector {
  private analyser: AnalyserNode;
  private dataArray: Uint8Array;
  private threshold = 30; // Adjust based on environment

  constructor(audioContext: AudioContext, stream: MediaStream) {
    this.analyser = audioContext.createAnalyser();
    this.analyser.fftSize = 512;
    this.dataArray = new Uint8Array(this.analyser.frequencyBinCount);

    const source = audioContext.createMediaStreamSource(stream);
    source.connect(this.analyser);
  }

  // Check if user is speaking
  isVoiceActive(): boolean {
    this.analyser.getByteFrequencyData(this.dataArray);

    // Calculate average volume
    const average = this.dataArray.reduce((a, b) => a + b) / this.dataArray.length;

    return average > this.threshold;
  }

  // Auto-mute when not speaking (for PTT alternative)
  update() {
    const isActive = this.isVoiceActive();

    if (isActive) {
      showTalkingIndicator();
    } else {
      hideTalkingIndicator();
    }

    return isActive;
  }
}

// Use VAD for voice-activated transmission
const vad = new VoiceActivityDetector(audioContext, localStream);

setInterval(() => {
  const isSpeaking = vad.update();

  // Auto-enable mic when speaking detected
  if (voiceActivatedMode && isSpeaking !== voiceChat.isTalking) {
    voiceChat.setMicrophoneEnabled(isSpeaking);
  }
}, 100); // Check every 100ms

Privacy Controls#

Player Voice Controls

typescript

class PlayerVoiceControls {
  private mutedPlayers = new Set<string>();
  private blockedPlayers = new Set<string>();
  private volumeLevels = new Map<string, number>(); // 0-1

  // Mute specific player
  mutePlayer(peerId: string) {
    this.mutedPlayers.add(peerId);

    const audioSource = spatialAudioSources.get(peerId);
    if (audioSource) {
      audioSource.audio.muted = true;
    }

    console.log(`Muted player: ${peerId}`);
  }

  // Unmute specific player
  unmutePlayer(peerId: string) {
    this.mutedPlayers.delete(peerId);

    const audioSource = spatialAudioSources.get(peerId);
    if (audioSource && !this.blockedPlayers.has(peerId)) {
      audioSource.audio.muted = false;
    }

    console.log(`Unmuted player: ${peerId}`);
  }

  // Block player (prevents all communication)
  blockPlayer(peerId: string) {
    this.blockedPlayers.add(peerId);
    this.mutedPlayers.add(peerId);

    // Disconnect voice connection
    const pc = voiceChat.peerConnections.get(peerId);
    if (pc) {
      pc.close();
      voiceChat.peerConnections.delete(peerId);
    }

    // Remove audio source
    const audioSource = spatialAudioSources.get(peerId);
    if (audioSource) {
      audioSource.audio.pause();
      audioSource.source.disconnect();
      spatialAudioSources.delete(peerId);
    }

    console.log(`Blocked player: ${peerId}`);
  }

  // Set individual player volume
  setPlayerVolume(peerId: string, volume: number) {
    // Clamp to 0-1 range
    volume = Math.max(0, Math.min(1, volume));
    this.volumeLevels.set(peerId, volume);

    const audioSource = spatialAudioSources.get(peerId);
    if (audioSource) {
      audioSource.audio.volume = volume;
    }

    console.log(`Set ${peerId} volume to ${(volume * 100).toFixed(0)}%`);
  }

  // Get player volume
  getPlayerVolume(peerId: string): number {
    return this.volumeLevels.get(peerId) ?? 1.0;
  }

  // Check if player is muted
  isPlayerMuted(peerId: string): boolean {
    return this.mutedPlayers.has(peerId);
  }

  // Mute all players
  muteAll() {
    spatialAudioSources.forEach((_, peerId) => {
      this.mutePlayer(peerId);
    });
  }

  // Unmute all players
  unmuteAll() {
    const muted = Array.from(this.mutedPlayers);
    muted.forEach(peerId => this.unmutePlayer(peerId));
  }
}

// UI for player voice controls
function renderPlayerVoiceControl(peerId: string, playerName: string) {
  const controls = new PlayerVoiceControls();
  const isMuted = controls.isPlayerMuted(peerId);
  const volume = controls.getPlayerVolume(peerId);

  return (
    <div className="player-voice-control">
      <span>{playerName}</span>

      <button onClick={() => {
        if (isMuted) {
          controls.unmutePlayer(peerId);
        } else {
          controls.mutePlayer(peerId);
        }
      }}>
        {isMuted ? 'Unmute' : 'Mute'}
      </button>

      <input
        type="range"
        min="0"
        max="100"
        value={volume * 100}
        onChange={(e) => controls.setPlayerVolume(peerId, e.target.value / 100)}
      />

      <button onClick={() => controls.blockPlayer(peerId)}>
        Block
      </button>
    </div>
  );
}

Voice Chat Best Practices#

Request permissions clearly — Show UI explaining why microphone access is needed before requesting.
Default to muted — Start with microphone muted and require explicit PTT or toggle to talk.
Visual feedback — Show talking indicators for both local and remote players.
Provide mute controls — Allow players to mute individual players, teams, or all voice.
Volume controls — Let players adjust individual voice volumes to their preference.
Use spatial audio — Enable 3D positional audio for immersive proximity voice chat.
Optimize for mobile — Test voice chat on mobile devices with different network conditions.
Handle disconnections — Gracefully handle WebRTC connection failures and reconnection.
Respect privacy — Allow players to opt-out of voice chat entirely.
Monitor quality — Track connection quality metrics (RTT, packet loss, jitter).

Common Issues & Solutions#

Issue	Cause	Solution
No audio from remote player	Audio element not playing	Call audio.play() and check autoplay policies
Echo/feedback	Echo cancellation disabled	Enable echoCancellation in audio constraints
Connection fails	STUN/TURN server issues	Add TURN servers for NAT traversal
High latency	Server relay mode	Use peer-to-peer connections when possible
Distorted audio	Bitrate too low	Increase audio bitrate in codec settings
Spatial audio not working	Listener/source not updated	Update positions every frame

PreviousEntity SynchronizationSync game entities NextCustom MessagesSend custom data

Web Engine Docs

Preparing documentation

Documentation0%

Use the search bar to quickly find any topic

Content

Examples

Ready

Web Engine v0.1.0-alpha

Edit this page on GitHub

Last updated: 1/15/2026

Was this page helpful?