Skip to content

Dynamic Waveform Visualization

ashritha kunjeti requested to merge wave-form into develop

Overview

Converted the existing static waveform visualization system into a real-time dynamic waveform streaming and rendering pipeline using WebSockets and HTML5 Canvas.

The implementation progressively streams waveform amplitude data and renders animated waveform bars synchronized with audio playback.

Key Changes

1. Replaced Static Visualization Workflow

Previous Workflow

  • Generated complete static waveform images
  • Rendered waveform as a static visual output

Updated Workflow

  • Streams waveform data progressively using WebSockets
  • Dynamically renders waveform frames using HTML5 Canvas

Improvements

  • real-time visualization
  • progressive waveform loading
  • synchronized playback tracking
  • smoother rendering experience

2. Added Upload Validation & Security Handling

File Validation

Implemented validation for:

  • missing uploads
  • invalid filenames

Supported Formats

Allowed audio formats:

  • .wav
  • .mp3
  • .m4a
  • .flac

File Size Protection

  • Added 100MB upload limit
  • Returns 413 File too large for oversized uploads

Unique Audio Identification

  • Added UUID-based audio identifiers
  • Prevents filename collisions

Temporary Storage Handling

  • Stores uploaded audio in dedicated waveform temp directory

3. Implemented WebSocket-Based Streaming

Added persistent waveform streaming using:

/ws/waveform/{audio_id}

Features

  • streams waveform amplitude chunks progressively
  • maintains continuous communication during playback
  • removes dependency on static image generation

4. Optimized Audio Processing

Implemented waveform preprocessing using Librosa.

Features

  • extracts waveform amplitude data
  • safely downsamples audio
  • limits waveform size to ~200k points

Benefits

  • prevents browser crashes
  • reduces rendering overhead
  • improves memory efficiency

5. Added Chunked Streaming Architecture

Instead of sending the complete waveform at once:

  • waveform data is split into smaller chunks
  • chunks are streamed incrementally

Benefits

  • smoother rendering
  • reduced memory spikes
  • progressive loading effect

6. Added Controlled Streaming Delay

Implemented asynchronous pacing between chunks:

await asyncio.sleep(0.02)

Purpose

Creates:

  • live-streaming effect
  • smoother progressive visualization
  • controlled animation pacing

7. Replaced Static Rendering with HTML5 Canvas

Implemented dynamic Canvas-based rendering system.

Benefits

  • real-time drawing
  • smooth animation
  • customizable visual effects
  • improved rendering performance

8. Implemented Progressive Waveform Rendering

Flow

  • waveform chunks arrive incrementally
  • data appended progressively
  • canvas continuously redraws waveform

Result

Waveform fills dynamically from left to right during playback.

9. Added requestAnimationFrame Rendering Loop

Implemented:

requestAnimationFrame(drawWaveform)

Benefits

  • smoother animations
  • browser-optimized rendering
  • refresh-rate synchronized drawing

10. Added Custom Visualization Styling

Implemented:

  • pink-to-red gradients
  • glow/neon shadow effects
  • animated waveform bars
  • cinematic visualizer appearance

11. Improved Audio Player Interaction

Changes

  • enabled visible audio controls
  • repositioned audio player below waveform canvas

Result

Allows:

  • manual play/pause
  • reload support
  • synchronized interaction

12. Added Play-Triggered Streaming

Moved WebSocket initialization into audio player "play" event.

Problem Solved

Previously:

  • waveform streamed before playback interaction
  • autoplay restrictions caused synchronization mismatch

Result

Waveform streaming now starts only when playback begins.

13. Implemented Real-Time Playhead Synchronization

Added custom red playhead tracker.

Logic

Uses:

audio.currentTime / audio.duration

instead of chunk-count tracking.

Result

Playhead remains accurately synchronized with playback.

14. Added timeupdate Playback Synchronization

Implemented:

audioPlayer.addEventListener("timeupdate", ...)

Purpose

Continuously updates:

  • playhead position
  • waveform redraw timing

Result

Smooth playback synchronization.

15. Rendering & Performance Optimizations

Optimized visualization performance using:

  • waveform downsampling
  • chunked rendering
  • Canvas-based drawing
  • reduced rendering overhead

Final Result

Successfully transformed the waveform visualization system from a static rendering workflow into a real-time streaming visualization architecture using:

  • WebSockets
  • progressive chunked waveform streaming
  • HTML5 Canvas rendering
  • synchronized playback tracking
  • dynamic animated waveform rendering

*Closes #31

Edited by ashritha kunjeti

Merge request reports