Python / FastAPI API scraping 3 music platforms (SoundCloud, Beatport, Bandcamp) and exposing the data via REST, then through an MCP server for native integration with AI agents.
Python/FastAPI API to scrape 3 music platforms (Soundcloud, Beatport, Bandcamp) and expose the data to n8n workflows, complemented by an MCP server (Model Context Protocol) that exposes the same features as native tools for AI agents (Claude Desktop, n8n MCP Server Trigger). Both interfaces coexist on the same business code.
Business goal: have a unified interface to automatically extract music metadata across several platforms, usable both by classic workflows (REST) and by AI agents (MCP).
My role: end-to-end design, development and deployment, autonomously.
Robust scrapers for 3 platforms: Soundcloud (via official API after obtaining developer access), Beatport (extraction of structured JSON data), Bandcamp (HTML parsing with filtering).
Technical challenges: rate-limit handling, mid-project pivot on Soundcloud (HTML scraping → official API), resilience to page-structure changes.
Solutions: async architecture with retry and backoff, separate handling of temporary errors (automatic retry) and permanent errors (escalation), parallelization of related requests.
Scrapers exposed via an authenticated REST API (classic n8n workflows, scripts, dashboards) and via an MCP server (native tools for AI agents: Claude Desktop, n8n MCP). Both interfaces share the same business code — no rewrite, no replacement.
Technical challenges: first MCP server implemented, several iterations to stabilize communication, sharing the business code between both interfaces without duplicating the scrapers' logic.
Solutions: each MCP tool maps to its REST endpoint counterpart, scrapers factored into a business layer reused by both interfaces, separate Docker images so REST and MCP can be deployed independently as needed.
Initial deployment via GitHub Actions + SSH, then migration to Dokploy to simplify orchestration and centralize the infrastructure with the other self-hosted projects. Automatic CI/CD on merge.