Under the Hood

How MusicPulse Works

From raw data to beautiful charts — here is how we bring the world's music data together in one place, updated every hour.

1

Data Collection

Our Cloudflare Workers run on automated cron schedules — every hour for charts and every 2 hours for trending data. They fetch from public APIs and RSS feeds including Spotify Charts CSV endpoints, Apple Music RSS feeds, YouTube Data API, and the TikTok Creative Center. No private APIs or authentication tokens are required for any of these sources — everything is publicly accessible.

  • Spotify: Regional daily chart data via public JSON endpoints
  • Apple Music: Top 100 RSS feeds by country
  • YouTube: Most popular music videos via Data API
  • TikTok: Trending sounds from the Creative Center API
2

Normalization & Matching

Raw data arrives in different formats from each platform. Our processing pipeline normalizes everything into a unified schema — songs are matched across platforms using ISRC codes, titles, and artist names. Chart positions are tracked over time to calculate position changes, peak positions, and weeks on chart. Cross-platform scores are computed by weighting performance across all sources.

  • Songs matched by ISRC, title similarity, and artist name
  • Position changes calculated against previous day data
  • Cross-platform power scores weighted by platform reach
  • Trending velocity computed from 7-day sparkline data
3

Storage & Caching

Processed data is stored in Cloudflare D1 (SQLite at the edge) for fast queries and Cloudflare KV for cached page responses. When a scraper writes new data, it automatically invalidates the relevant KV cache keys, ensuring the website always shows the freshest data without sacrificing performance. Every query is optimized for sub-10ms response times at the edge.

  • D1 database for persistent chart, trending, and artist data
  • KV cache for pre-rendered page fragments
  • Automatic cache invalidation on data updates
  • Edge-deployed for global sub-10ms latency
4

Presentation & Discovery

The website is built with Next.js as a static export deployed on Cloudflare Pages. Charts, trending sections, and artist pages are pre-rendered for instant loading. Interactive elements like platform filters and region selectors are handled client-side for a smooth, app-like experience. The dark theme and gradient accents are designed for comfortable long browsing sessions.

  • Static export for instant page loads
  • Client-side interactivity for filters and selectors
  • Responsive design from mobile to ultrawide
  • Accessibility-first with semantic HTML