Over 200,000 Servers in One Place! Visiting Hetzner in Falkenstein (Germany)
der8auer EN6 HN recommenders · recommended in 6 months, 2021-2025
1.6M views · 5 years ago · documentary · dated
Overclocker der8auer tours Hetzner's Falkenstein data center in Germany, showing custom racks, own mainboards, passively cooled servers and ATX-style cases on shelves.
HN points to it whenever Hetzner comes up, as the best look inside their unusual, semi-custom and low-cost data center setup.
9:35Hetzner shows Special AM5 Board with 90° Rotated Socket
der8auer EN
44K views • 3 years ago
23:33Why Do Mainframes Still Exist? What's Inside One? 40TB, 200+ Cores, AI, and more!
Dave's Garage
1.2M views • 3 years ago
5:28Inside a Google data center
Google Workspace
22.6M views • 11 years ago
28:31Velocity 2010: James Hamilton, "Datacenter Infrastructure Innovation"
O'Reilly
9K views • 16 years ago
24:09A walk through of installing Synapse+Riot+Jitsi from scratch on Debian
Matrixdotorg
61K views • 6 years ago
6:40High Res Wifi Signal Mapping
CNLohr
551K views • 11 years ago
44:37Kubernetes for SysAdmins | Kelsey Hightower at PuppetConf | Talk & Demo
Perforce Puppet
153K views • 10 years ago
1:26:02Stanford Seminar - The Soul of a New Machine: Rethinking the Computer
Stanford Online
76K views • 6 years ago
2:02:53Oxide and Friends 7/3/2023 -- Shipping the first Oxide rack: Your questions answered!
Oxide Computer Company
4K views • 3 years ago
52:29"I See What You Mean" by Peter Alvaro
Strange Loop Conference
59K views • 11 years ago
1:01:50BahnMining - Pünktlichkeit ist eine Zier (David Kriesel)
media.ccc.de
3.3M views • 6 years ago
1:01:46GigCityElixir23 - Bryan Hunter
Groxio
10K views • 3 years ago
1:33:27AWS re:Invent 2016: Tuesday Night Live with James Hamilton
Amazon Web Services
133K views • 9 years ago
3:55Velocity 2011: Artur Bergman, "Artur on SSD's"
O'Reilly
30K views • 15 years ago
15:39introducing copyparty, the FOSS file server
tripflag
613K views • 1 year ago
9:59Moving online webserver using public transport
Sascha P
164K views • 16 years ago
35:31June 2021 FreeBSD Developer Summit: Raid-Z Expansion
FreeBSD
4K views • 5 years ago
26:49Inside a Huge PCB Factory
Strange Parts
3.7M views • 8 years ago
1:04:04Overwatch Gameplay Architecture and Netcode
GDC Festival of Gaming
196K views • 7 years ago
4:57Rich Hickey rants about HttpServletRequest in 1080p: Death By Specificity from Clojure Made Simple
Fred Overflow
43K views • 6 years ago
17 comments from Hacker News
Der8auer did some similar videos a few years ago, but in English: ▶ this video link The second is especially interesting as it demonstrates Hetzner's unique semi-custom hardware.
The best resource I know is a tour of one of their data centers. Around minute 6 you get some nice views on their current-gen racks, and later in the video you see the back side, followed by the assembly of the servers. ▶ this video
There's some videos out there showing off their custom motherboards, it's an odd mixture of standard-ish consumer ATX but stripped down to the absolute bare essentials and with some components rearranged for better airflow in racks chassis they also build themselves: link Another video showing the custom racks and other infrastructure: ▶ this video
Couldn't agree more, I think Hetzner is probably Europe's best bet on a hyperscaler. One of the more telling indicators IMO is their growing market share outside of the EU/DACH. To add on to the comments about Hetzner building their own custom hardware, they also custom built their own software stack. They rejected the hype that was OpenStack and worked diligently on their own hypervisor platform (that they are incredibly secretive about) and that appears to be paying off in spades for them. Most sovereign cloud plays end up being suffocated by the complexity, and incoherence, of the OpenStack ecosystem. It just becomes impossible to ship. I know Hetzner is working like crazy on a managed Kubernetes offering atm, I think when that lands it could be very interesting. For a fascinatingly different take on how to build a datacenter: ▶ this video
▶ this video Datacenters in my country usually had some rooms with tower servers 20 years ago here, well my first colo was for the tower server I brought in the large backpack:-). But density requirements, cold/hot aisles etc. prevailed and towers are generally considered inefficient for the datacenter purposes. And then you have Hetzner datacenter that probably all people running DCs I know would ridicule, but they would not be able to respond to fan replacement at the same time. I wonder how many rack server chassis are recycled each year because the manufacturer just won't let you reuse them with new motherboard, power supply due to new shape, design, ports placement etc.
They have ATX cases on shelves. You can take a look at this tour of their datacenter [1] for more insight. ATX cases are visible around the 3 minute mark. -- [1] ▶ this video
There's a great YouTube tour of one of their data centres here: ▶ this video 'Over 200,000 Servers in One Place! Visiting Hetzner in Falkenstein (Germany)' The channel is well worth a subscribe too.
For those curious about what it actually looks like in a Hetzner datacenter, der8auer made a video about their Falkenstein, Germany location: ▶ this video
I'm quite jealous of their closet then :) For anyone curious here's a tour of one of their data centers ▶ this video .
Here's a Hetzner DC: ▶ this video Here's a GCP/Google DC: link Not to speak of network/power redundancy, managed hardware (like not having to prove to your ISP that their memory or disk is broken!), lots of services besides just metal... HN is actually really in love with putting things on Hetzner, OVH and co. It works and it's cheap. But it wouldn't be a good experience when things break, and you better have really good backups and plans to fail over to another box when something bad[1] happens. [1]: link
I know this is a joke but if you'd like to see inside their data center, there's a nice video: ▶ this video
German YouTuber derbauer has a 20 minute tour video of Hetzner's Falkenstein datacentre: ▶ this video He also has a newer video showing the power distribution, cooling, and fire suppression in one of the Dr Oetker datacentres: link
I came across tour of Hetzner datacenter[1] and it looks like a completely unprofessional setup, like castle on stilts. Does not inspire confidence. [1] ▶ this video
Hetzner Falkenstein was toured recently by a YouTuber and I didn't spot any fire suppression systems in the video [1]. OVH SBG1 (which was partially destroyed by the fire that wiped out SBG2) used shipping containers which also didn't appear to have any fire suppression systems [2]. By contrast, the typical data centers people know which have fire suppression systems appear to have much of their key electrical equipment (and control systems) located in the same area or adjoining rooms of the same facility [3] [4] [5] [6]. In fact, there is even a video recording of a UPS failure [7] showing a lucky case where no one was injured and not too much damage was caused. The employee lingered in the room when they should have immediately left at the first sign of danger. Arc flashes are a scary possibility as shown in [8] and [9] because of the need to switch multiple megawatts of electricity through complex power systems that includes UPS battery banks and automatic diesel generators. There are video recordings demonstrating how fire suppression systems work [10] [11] and a description of how a data center would be designed to respond to a fire (including closing ventilation dampers) [12] [13]. I'm sure fire suppression systems are not cheap, but in the grand scheme of a data center full of millions of dollars of equipment (not to mention cost of customer downtime) surely it would make sense to install them. [1] ▶ this video [2] link [3] link [4] link [5] link [6] link [7] [LIN
Their buildings are passively cooled so this sounds quite plausible. Pretty fun video showing the realities of their setup ▶ this video
Not much to explain, you're absolutely right. Hetzner would have been a much wiser choice here, but advocating any cloud provider at this scale probably has its perks too or he wouldn't be burning his money. Then again, perks only go so long and at some point do come to an end, so this is why he may be writing about costs right now. Take a look at their datacenter in Germany: ▶ this video
There has been a nice YT video recently showing how Hetzner operates their DC and their servers: ▶ this video Was quite surprised that they use custom built server racks with their own custom-made mainboards delivered by OEMs. They even have their own long-term Fan testing rigs and custom HDD failure check setups.