Vol. 01 · No. 03
— Field Notes —
June 2026
A Field Note on Cost

A Hetzner Box vs. a Databricks cluster

A version of a working argument, written for anyone who has ever looked at a cloud bill and wondered what they’re paying for. The short version is that most companies are renting a fleet of trucks to deliver an envelope.

A lot of companies run their nightly data jobs on a whole cluster of machines, and most of that time isn’t spent on the actual work. Reading the data, doing the sums and writing the result can be a small part of the run. The rest goes into starting machines, getting them to agree to talk to each other and waiting for them to wake up. I think a lot of Dutch companies are in that position, and the rest of this note is the case for why. The argument is mostly about scale, and the simple fact that the size of the tool should match the size of the job.

Most companies are paying for a fleet of trucks to deliver an envelope.

— the working argument
01 · The setup

Two ways to do the same job

Imagine you run a logistics company and every night you need to summarise the day's shipments into a few KPIs. The data lives on a shelf somewhere, the summary goes to a dashboard, and the work itself is simple. The question is what kind of computer you hire to do it.

Setup A · The fleet

A cluster of computers, hired by the hour

This is how most large companies do it. You rent a small fleet of computers from a cloud provider, usually through a service called Databricks or Spark (think of it as a name-brand version of the same thing). The fleet is on standby 24/7 because starting and stopping it is expensive. A manager program splits the work between the machines, and the machines shuffle data between each other to get the job done.

What you pay for

The whole fleet, always on

On top of that there’s a per-hour tax for the privilege of using the manager program, and the tax is usually bigger than the computer bill itself.

What you actually use

A few minutes of work, per night

The fleet sits idle the rest of the time, so you’re paying for a 24-hour rental to use a vehicle for forty minutes.

Bill, per year

Around €34,000–€67,000

A working figure for four always-on machines plus the per-hour tax, using the prices in this note. A bigger cluster costs more.

Figure 1 Setup A, the cluster. A fleet of computers hired by the hour, with a manager program that splits the work between them. Most of the time most of the fleet is idle, and the bill doesn’t care.
Setup B · The single machine

One powerful computer, rented by the month

This is the alternative, one big, well-built computer rented by the month from a European host called Hetzner (based in Germany, GDPR-clean and much cheaper than the big American clouds). The computer has 128 GB of working memory, a fast solid-state drive and a quiet data centre in a small town. A small piece of free software called DuckDB, built in Amsterdam by the way, does the work by itself in a few minutes.

What you pay for

One machine, one month at a time

No manager program, no per-hour tax and no idle fleet. You just pay for what’s plugged in.

What you actually use

All of it, all the time

The machine does the work in minutes, and for the other 23 hours and 55 minutes it waits. Cheap waiting is fine.

Bill, per year

Around €1,400

For a single machine with 128 GB of memory and 4 TB of fast storage. That’s less than one restaurant dinner per workday.

Figure 2 Setup B, the single machine. One computer, rented by the month. The free software that runs the work was built in Amsterdam, twenty minutes from most Dutch offices. The bill is a small fraction of the cluster’s, and the work finishes faster.
02 · The numbers

What the bill actually looks like

Both setups can do the same job, with the same work, the same data and the same output, so the only difference is on the invoice. Here’s the honest accounting, with the prices I checked against the actual vendor websites this week, in June 2026.

What you are paying for Setup A · The fleet Setup B · The single machine
The basic computer(s) 4 cloud machines, always on. About €1,400 a month, before any extras. 1 dedicated machine, rented by the month. About €117 a month, all in.
The manager program Databricks’s per-hour tax, usually 1–3× the cost of the machines themselves. None. DuckDB is a free program, the size of a small photo, with no external dependencies.
The setup work 100-line configuration files, cluster tuning, dependency debugging and a dedicated engineer to keep it healthy. Six lines of Python and uv add duckdb, and the same engineer uses the time they saved for something else.
How long the work actually takes ~40 minutes, of which ~2 minutes is the actual work. The rest is starting the fleet, getting the machines to talk to each other and tidying up. ~3–4 minutes, with no fleet to start and no chatter between machines. Just the work, start to finish.
Per-year cost (working figure) €34,000–€67,000 €1,500–€3,000
Figure 3 The honest accounting. Same work, same data and same output. The fleet costs up to ~45× more and is ~10× slower at the actual job, because most of its time goes into being a fleet.

The cluster figure is a working figure built from the prices above, four always-on machines plus the per-hour tax, so a bigger cluster costs more and a smaller one less. The €1,500 figure comes from the published monthly price of a real machine you can rent today from a company that has been in business since the 1990s. You can check both numbers yourself. So the only honest question left is whether you need a fleet or a van.

03 · The honest line

When the fleet is the right call

Pretending the cheaper option always wins would be a sales pitch, because it doesn’t. The honest version has a clear line. Below it, the single machine wins on speed, cost and simplicity, and above it the fleet earns its keep. Here’s where that line sits, based on the public benchmarks and the four independent tests the data community has published in the last twelve months.

The job's size What wins on speed The honest read
Under ~12 GB of data, single run Single machine Always, and by a wide margin. Tested and re-tested by the data community. If your job touches less than the contents of a long phone video, you’re paying a fleet to do a van’s work, so switch.
~12 GB to ~50 GB, single run Single machine Still wins on speed, cost and operational sanity, and for most jobs it’s no contest. This is the size of a small company’s annual data, so the fleet is the wrong shape. Switch.
~50 GB to ~100 GB, single run It depends The fleet starts to catch up, but the single machine is still cheaper. Benchmark on your real data. If the job is genuinely this big and runs every night, it’s a judgment call. Run both, time both and pick.
Above ~100 GB, single run The fleet Wins on speed, especially the premium versions (Databricks's "Photon" mode, Microsoft's "Native Execution" mode). Now the fleet is doing work the single machine can’t, and the premium fleet is genuinely faster. The cost gap is still real, so either pay it or redesign the job to scan less data.
The work is "I have 100 TB in storage" It depends on how much of it the job actually touches. Having 100 TB in a warehouse tells you nothing about the job. A COUNT(*) WHERE country = 'NL' on a well-organised table is a tiny scan. So the right question is how much this job reads, more than how much is in the warehouse.
Figure 4 Where the line sits. Below ~50 GB the single machine wins, above ~100 GB the fleet earns its keep, and in between you measure. The question to ask is how much this job reads, and “I have 100 TB” doesn’t answer that.
04 · The Dutch part

Why this lands differently here

There are three things that make the Netherlands a slightly different case from, say, the United States. The first is that the engine itself is local, and the second is that the cheaper host is local and serious. The third is that the labour cost of doing the wrong thing is high here.

A short list of reasons this is a Dutch story
A
The engine is built here
Built in Amsterdam

DuckDB, the single-machine engine described above, is a product of DuckDB Labs, founded in 2021 and headquartered at Science Park 404 in Amsterdam. Its main authors are at CWI, the Dutch national research institute for mathematics and computer science. So getting support is a phone call away, and you don’t have to open a Salesforce case.

fact DuckDB Labs B.V., Amsterdam
since 2021
B
The cheaper host is here
Hetzner · Germany

Hetzner is a German host with data centres in Germany and Finland, GDPR-compliant and in business since the 1990s. A 128 GB machine with a fast solid-state drive rents for €117 a month, and no American cloud region in the Netherlands comes close to that on price. The Dutch companies using it aren’t taking a risk on a no-name provider.

price €117/month
memory 128 GB
C
The labour cost is high
€90K–€120K loaded

A senior data engineer in the Randstad costs €70,000–€90,000 a year in salary, or about €90,000–€120,000 with benefits. International surveys consistently put 20–30% of a data engineer’s week on “fighting the cluster,” meaning debugging shuffles, tuning memory and restarting failed jobs. At a loaded cost of €100K, that’s €20,000–€30,000 a year per engineer. For a team of five, the hidden tax on the wrong tool is €100,000–€150,000 a year, on top of the cloud bill.

range €90K–€120K
per year fully loaded
Figure 5 Three reasons this is a Dutch story. The engine is local, the cheaper host is local and serious, and the hidden labour cost of using the wrong tool is unusually high here, so all three push in the same direction.
05 · What to do on Monday

Three small things, one afternoon

If any of this is starting to look like your own situation, the test is pretty short. Pick three of your smallest nightly jobs and look at the actual data size of each. Then run one of them on a rented machine, time it and see.

The afternoon test

A measurable experiment, no leap of faith needed

The point of the test is to know, with a number, what the gap actually is on your real data, so you don’t have to switch on the first day. Most teams that run this test find the gap is bigger than they expected, and the conversation afterwards is short.

Step 1

Pick three small jobs

Open your scheduler. Find the three smallest nightly batch jobs. Write down the input data size and the wall-clock time for each.

Step 2

Rent one machine for a week

€117 from Hetzner, or €140 from any equivalent. The whole setup takes under an hour, and the free program (DuckDB) installs in one line.

Step 3

Run the same work, time it

Most jobs port in an afternoon. The version with the manager program is typically 100 lines and the version without it is typically 6, so time both and look at the number.

Figure 6 The afternoon test. Three steps, no risk and real numbers. If the gap on your data is as big as the published benchmarks suggest, the conversation afterwards writes itself, and if it isn’t, you’ve learned something too.
Closing observation

The right tool is the one that fits the size of the job

A cluster of computers for a job that one computer can finish in four minutes still works, it’s just the wrong shape. It’s a bit like paying for a fleet of trucks to deliver a letter. In a market where a data bill like this can easily run to tens of thousands of euros a year, the question I’d ask is whether we still need a fleet at all. And if we don’t, I’d want to know which 30% of our infrastructure budget we’re lighting on fire because nobody asked that six months ago.

So the honest version of the rule is that a job that fits on a single machine should run on a single machine, and the bill is the bill. A job that genuinely needs a fleet, like real-time event streams, petabyte-scale science or the kind of work a national radio telescope does, should run on a fleet. No single-machine story will save it. The hard part is knowing which kind of job you have, without pretending you have the second kind because the first kind feels too simple to be the answer.

— end —

A Hetzner Box vs. a Databricks Cluster · A Field Note Set in Shadows Into Light Two & Cause

← Back to all posts