Stop Building Your Analytics Pipeline Like It's 2015
Adil Sher
Author
Last month, I spent three days debugging why our analytics dashboard was lagging. The culprit? Our analytics queries were hitting the production MySQL database directly. Every pivot table, every date range filter, every curious product manager's ad-hoc question was putting load on the same database powering our actual application. I remember sitting in the incident channel thinking: there has to be a better way to do this that doesn't require me to become a Kafka infrastructure expert.
That's when I stumbled onto this walkthrough about moving data from MySQL to ClickHouse using Iceberg and Polaris. And honestly, it changed how I'm thinking about the analytics problem for teams that don't have the luxury of hiring a data engineer.
The Real Problem Nobody Talks About
Everyone knows you shouldn't query production databases for analytics. But what most tutorials skip over is the operational cost. You can use ClickPipes if you're locked into ClickHouse Cloud. You can spin up Kafka and Debezium if you want to manage three more infrastructure components. You can pay Fivetran per GB if you don't mind the opaque pricing. Or you can try the solution this article walks through: open source replication with Iceberg tables that multiple engines can read.
What interests me here isn't the technical specificity, it's the philosophy. The author is suggesting a path where your data lives once, in a format multiple tools can actually understand. ClickHouse reads it. DuckDB reads it. Trino reads it. That's genuinely different from most CDC setups I've seen, where you end up with data silos and duplicate pipelines.
How This Actually Works
The setup uses OLake Go, an open source replication tool that reads MySQL's binary log and writes Apache Iceberg tables into a Polaris catalog. Polaris is essentially a metadata layer, it doesn't store your data, just pointers to it. The actual Parquet files land on MinIO (or any S3-compatible storage you own).
Here's what matters: OLake Go handles the initial snapshot of your tables in parallel, then continuously replays changes from the binlog. It's not streaming in the Kafka sense, it's transactional. You're building Iceberg tables, which means you get ACID guarantees and actual time travel support, not just "eventually consistent" data that might be slightly wrong.
The MySQL side requires some setup. You need row-based binary logging enabled with full row metadata. That binlog_row_metadata = FULL setting is crucial because it's what lets OLake Go figure out schema changes without breaking. And yes, you need a primary key on every table you want to replicate deletes from.
What I'd Actually Use This For
The walkthrough is thorough but I want to be honest about where I see this fitting in real work. If you're running a small team (under 20 engineers) and you need analytics separate from production, this is solid. You avoid managed service costs, you're not vendor-locked, and you can read the same tables from multiple query engines.
Where I'd hesitate: operational burden. You're responsible for Polaris uptime, MinIO storage, monitoring the replication job, handling binlog retention windows when jobs pause. That's not nothing. For a team of two backend engineers trying to ship features, that's probably too much.
But there's a middle ground. If you're already running infrastructure (Kubernetes, managed databases), bolting this on is maybe an afternoon of work. And having a shared data layer that ClickHouse and DuckDB can both read? That's genuinely valuable. It means your analytics engineer isn't fighting with tool limitations.
The Setup That Matters
The original article includes actual copy-paste commands, which I appreciate. But here's what I'd add: before running this in production, test your binlog retention strategy. The article mentions seven days, but that depends on your write volume. I've seen servers where seven days is 100GB of binlog. Know your numbers.
Also, that ANALYZE TABLE step they mention? Don't skip it. OLake Go uses table statistics to parallelize the initial sync. A table with no statistics will replicate serially and waste hours.
Where I'm Landing
This approach appeals to me because it's pragmatic. It's not "hire someone to manage Kafka" and it's not "pay for SaaS." It's "build something that works with the tools you probably already understand." The fact that Iceberg has become the lingua franca for data lakes makes this actually reasonable now.
My question for you: in your infrastructure, would this lower friction or add it? If you're already managing storage and databases, probably lower. If you're trying to stay serverless, maybe skip it.
Source: This post was inspired by "MySQL to ClickHouse with Binlog CDC: A Copy-Paste Walkthrough Through Polaris" by Dev.to. Read the original article