A Hetzner Box vs. a Databricks cluster
A version of a working argument, written for anyone who has ever looked at a cloud bill and wondered what they are paying for. The short version: most companies are renting a fleet of trucks to deliver an envelope.
A friend runs the data side of a Dutch logistics company, somewhere between a few thousand and a few hundred million in revenue. Every night, his team runs a job that does three things to 80 terabytes of historical shipment data. The job takes forty minutes. The bill for the computers that run the job is north of two hundred thousand euros a year. The actual work — reading the data, doing the sums, writing the result — takes about two minutes. The other thirty-eight are spent starting machines, asking them to agree to talk to each other, and waiting for them to wake up.
He told me, when I asked, that switching to a cheaper setup would never work. He said it three times, in three different ways: it would not work, the tool he would need is a toy, and besides, they have eighty terabytes.
He was wrong on all three counts, and the rest of this note is the case for why a lot of Dutch companies are in the same position. The argument is not about clever engineering. It is about scale, and the fact that the size of the tool should match the size of the job.
Most companies are paying for a fleet of trucks to deliver an envelope.
— the working argumentTwo ways to do the same job
Imagine you run a logistics company and every night you need to summarise the day's shipments into a few KPIs. The data lives on a shelf somewhere, the summary goes to a dashboard, and the work itself is simple. The question is what kind of computer you hire to do it.
A cluster: many computers, hired by the hour
This is how most large companies do it. You rent a small fleet of computers from a cloud provider — usually a service called Databricks or Spark (think of it as a name-brand version of the same thing). The fleet is on standby 24/7 because starting and stopping it is expensive. A manager program splits the work between the machines, and the machines shuffle data between each other to get the job done.
What you pay for
The whole fleet, always on
Plus a per-hour tax for the privilege of using the manager program. The tax is usually larger than the computer bill itself.
What you actually use
A few minutes of work, per night
The fleet sits idle the rest of the time. You are paying for a 24-hour rental to use a vehicle for forty minutes.
Bill, per year
Around €200,000
Realistic for a mid-sized Dutch company. Often more. This number is not unusual.
One powerful computer, rented by the month
This is the alternative. One big, well-built computer, rented by the month from a European host called Hetzner (based in Germany, GDPR-clean, much cheaper than the big American clouds). The computer has 128 GB of working memory, a fast solid-state drive, and a quiet data centre in a small town. A small piece of free software called DuckDB — built, incidentally, in Amsterdam — does the work, by itself, in a few minutes.
What you pay for
One machine, one month at a time
No manager program. No per-hour tax. No idle fleet. You pay for what is plugged in.
What you actually use
All of it, all the time
The machine does the work in minutes. The other 23 hours and 55 minutes, it waits. Cheap waiting is fine.
Bill, per year
Around €1,400
For a single machine, 128 GB memory, 4 TB of fast storage. Less than a single restaurant dinner per workday.
What the bill actually looks like
Both setups can do the same job. The work is the same work. The data is the same data. The output is the same output. The difference is on the invoice. Here is the honest accounting, with the prices I verified against the actual vendor websites this week, June 2026.
| What you are paying for | Setup A · The fleet | Setup B · The single machine |
|---|---|---|
| The basic computer(s) | 4 cloud machines, always on. About €1,400 a month, before any extras. | 1 dedicated machine, rented by the month. About €117 a month, all in. |
| The manager program | Databricks's per-hour tax. Usually 1–3× the cost of the machines themselves. | None. DuckDB is a free program, the size of a small photo, with no external dependencies. |
| The setup work | 100-line configuration files. Cluster tuning. Dependency debugging. A dedicated engineer to keep it healthy. | Six lines of Python. uv add duckdb. The same engineer uses the time they saved for something else. |
| How long the work actually takes | ~40 minutes, of which ~2 minutes is the work. The rest is starting the fleet, asking the machines to talk to each other, and tidying up. | ~3–4 minutes. No fleet to start. No inter-machine chatter. The work, start to finish. |
| Per-year cost, mid-size Dutch company | €200,000+ | €1,500–€3,000 |
The €200,000 figure is not a strawman. It is what mid-sized Dutch companies are actually paying. The €1,500 figure is also not a strawman — it is the published monthly price of a real machine you can rent today from a company that has been in business since the 1990s. Both numbers are verifiable. The only honest question is: do you need a fleet, or do you need a van?
When the fleet is the right call
Pretending the cheaper option always wins is a sales pitch. It does not. The honest version has a clear line. Below that line, the single machine wins on every dimension — speed, cost, simplicity. Above it, the fleet earns its keep. Here is where that line sits, based on the public benchmarks and the four independent tests the data community has published in the last twelve months.
| The job's size | What wins on speed | The honest read |
|---|---|---|
| Under ~12 GB of data, single run | Single machine Always. By a wide margin. Tested and re-tested by the data community. | If your job touches less than the contents of a long phone video, you are paying a fleet to do a van's work. Switch. |
| ~12 GB to ~50 GB, single run | Single machine Still wins on speed, cost, and operational sanity. No contest for most jobs. | This is the size of a small company's annual data. The fleet is the wrong shape. Switch. |
| ~50 GB to ~100 GB, single run | It depends The fleet starts to catch up. The single machine is still cheaper. Benchmark on your real data. | If the job is genuinely this big and runs every night, the decision is a judgment call. Run both, time both, pick. |
| Above ~100 GB, single run | The fleet Wins on speed, especially the premium versions (Databricks's "Photon" mode, Microsoft's "Native Execution" mode). | Now the fleet is doing work the single machine cannot. The premium fleet is genuinely faster. The cost gap is still real — pay it, or redesign the job to scan less data. |
| The work is "I have 100 TB in storage" | It depends How much of it does the job actually touch? | Having 100 TB in a warehouse tells you nothing about the job. A COUNT(*) WHERE country = 'NL' on a well-organised table is a tiny scan. The right question is "how much does this job read," not "how much is in the warehouse." |
Why this lands differently here
Three things make the Netherlands a slightly different case from, say, the United States. The first is that the engine itself is local. The second is that the cheaper host is local and serious. The third is that the labour cost of doing the wrong thing here is high.
DuckDB — the single-machine engine described above — is a product of DuckDB Labs, headquartered at Science Park 404 in Amsterdam, founded in 2021. Its main authors are at CWI, the Dutch national research institute for mathematics and computer science. Supporting the product is a phone call, not a Salesforce case.
since 2021
Hetzner is a German host with data centres in Germany and Finland, GDPR-compliant, in business since the 1990s. A 128 GB machine with a fast solid-state drive rents for €117 a month. There is no American cloud region in the Netherlands that matches that on price. The Dutch companies using it are not taking a risk on a no-name provider.
memory 128 GB
A senior data engineer in the Randstad costs €70,000–€90,000 a year in salary, or about €90,000–€120,000 with benefits. International surveys consistently put 20–30% of a data engineer's week on "fighting the cluster" — debugging shuffles, tuning memory, restarting failed jobs. At a loaded cost of €100K, that is €20,000–€30,000 a year per engineer. For a team of five, the hidden tax on the wrong tool is €100,000–€150,000 a year, on top of the cloud bill.
per year fully loaded
Three small things, one afternoon
If any of this is starting to look like your own situation, the test is short. Pick three of your smallest nightly jobs. Look at the actual data size of each. Then run one of them on a rented machine, time it, and see.
A measurable experiment, not a leap of faith
The point of the test is not to switch on the first day. The point is to know, with a number, what the gap actually is on your real data. Most teams that run this test find the gap is bigger than they expected, and the conversation afterwards is short.
Step 1
Pick three small jobs
Open your scheduler. Find the three smallest nightly batch jobs. Write down the input data size and the wall-clock time for each.
Step 2
Rent one machine for a week
€117 from Hetzner, or €140 from any equivalent. The whole setup takes under an hour. The free program (DuckDB) installs in one line.
Step 3
Run the same work, time it
Most jobs port in an afternoon. The version with the manager program is typically 100 lines. The version without it is typically 6. Time both. Look at the number.
The right tool is the one that fits the size of the job
A cluster of computers for a job that one computer can finish in four minutes is not wrong, exactly. It is the wrong shape. It is paying for a fleet of trucks to deliver a letter. In a market where a mid-size company's data bill is €100,000–€300,000 a year, the question worth asking is not "is the fleet faster than the van at delivering letters." The question is "do we still need a fleet at all, and if not, what is the 30% of our infrastructure budget that we are lighting on fire by not having asked that question six months ago."
The honest version of the rule: a job that fits on a single machine should run on a single machine, and the bill is the bill. A job that genuinely needs a fleet — real-time event streams, petabyte-scale science, the kind of work a national radio telescope does — should run on a fleet, and no single-machine story will save it. The art is knowing which kind of job you have, and not pretending you have the second kind because the first kind feels too simple to be the answer.
— end —