build system

Google engineers describe Google's internal build system, a unified, language-agnostic system used across the company. It was presented as a 2012 NYC tech talk.

Daniel Dunbar of Apple presents llbuild, a framework for building build systems, at the 2016 LLVM Developers' Meeting. He argues that the traditional way C compilers integrate with build systems limits compile speed, and proposes a new architecture.

Rich Hickey's 2013 talk applies language-design thinking to whole systems, covering names, queues, data, failure, and the interfaces between programs. It ends with a call to action for building systems holistically.

An AWS re:Invent 2014 talk on the core distributed systems primitives behind AWS and Amazon.com, such as consensus, key-value stores and transactional logging. It describes how AWS builds large services from a few common building blocks.

Joran Dirk Greef's Systems Distributed '23 talk on TigerStyle, the engineering methodology behind TigerBeetle. It covers designing safer systems in less time through up-front design, assertions, and deterministic simulation.

FreeBSD developer Benno Rice traces the history of init and bootstrap systems at linux.conf.au 2019 to explain why systemd was seen as necessary and why it provokes such strong reactions. He also discusses why change is scary.

Benno Rice's BSDCan 2018 talk on why systemd exists and what it got right and wrong. It traces the history of Unix init and why Linux distributions adopted it, while asking what the BSDs should learn.

José Valim's ElixirConf US 2023 closing keynote on the foundations of the Elixir type system, covering set-theoretic types and gradual typing with strong arrows.

Joe Duffy's RustConf 2017 closing keynote on his experience at Microsoft building Midori, an operating system written in a Rust-like safe systems language. He covers the technical and cultural challenges of applying those lessons to Windows.

Bryan Cantrill's QCon SF 2019 talk asking whether operating systems should be rewritten in Rust. He draws on his systems experience to discuss C, memory safety and Rust's fit for system software.

Chris Houser's Clojure/conj talk on why Clojure's exceptions destroy information as the stack unwinds, unlike Common Lisp's condition system, which can resume from the error site. He surveys the existing libraries that approximate conditions, explains why they don't compose with each other, and considers what a better design would look like.

A panel from the Lang NEXT 2014 conference on systems programming in 2014 and beyond. Language designers including Bjarne Stroustrup and Rob Pike discuss the topic, including C++ header compile times.

Will Wilson of FoundationDB explains how the team tested their distributed database using deterministic simulation. By running the whole system in a simulator with injected faults and reproducible seeds, they could find rare concurrency and network bugs.

Timothy Roscoe of ETH Zurich delivers the joint USENIX ATC/OSDI 2021 keynote arguing that operating systems must reckon with modern hardware, which is a heterogeneous collection of processors, firmware and coprocessors rather than a simple machine. It shows how much runs below and beside Linux.

A HOPE X talk giving an in-depth explanation of how elevators work, from their history in hacker culture to using their control systems to bypass building security. It covers features and abilities of elevators that most people never see.

Jeff Bonwick and Bill Moore, the creators of ZFS, present 'ZFS: The Last Word in File Systems' at the 2008 SNIA Software Developers' Conference. They explain the design, including copy-on-write, checksumming and avoiding the write hole.

Aditya Siram introduces ATS, a statically typed ML-like language that compiles to C and offers type-safe pointer arithmetic and a full proof language. The Strange Loop talk shows off C interop, concurrency and verification features with non-trivial examples.

Aaron Hsu's FnConf18 talk asks whether APL needs a static type system. He argues that APL's expressiveness and rapid iteration already deliver the benefits usually credited to types.

A Microsoft Research talk on BetrFS, a file system built on B-epsilon trees, a write-optimized data structure. It covers WODS such as LSM-trees, how to design a file system around them, and benchmark wins over ext4 and xfs.

A Microsoft Build 2019 session in which the WSL team explains the new WSL 2 architecture, which runs a real Linux kernel alongside Windows on a hypervisor. It covers faster file performance, Linux file access from Windows and better application support.

Marshall Kirk McKusick gives a talk at EuroBSDCon tracing filesystem and storage development from 1979 to today, using the BSD Fast Filesystem as the focus. It covers early disk-geometry optimization through later contiguous layout and small-file performance work.

Richard Cook's short Velocity 2013 talk on resilience in complex adaptive systems, arguing systems operate near the edge of failure and that the surprise is how rarely they fail. He presents a model of how pressures push systems toward failure boundaries.

Optiver's Carl Cook explains how to build high-frequency trading systems in C++, where the critical path runs rarely but must be extremely fast. He covers techniques for low and predictable latency, including avoiding main memory and cache misses.

Ian Hubert's Blender Conference talk on world building in Blender, covering photogrammetry, camera projection and assembling complex environments from simple shapes at great speed.

Dr. Richard Cook, a physician and patient safety researcher, presents his ideas on how complex systems fail, at Velocity 2012. He argues for designing for resiliency rather than trying to exhaustively handle every failure mode.

A short edited excerpt of Terence McKenna arguing that culture acts as an operating system for the mind, set to imagery and music. It touches on psychedelics and how they change thinking.

Dave Chinner's linux.conf.au 2012 talk on XFS scalability, covering pain points with large numbers of files and complex metadata, and the work to fix XFS metadata performance.

Joe Armstrong's Strange Loop 2013 talk on building large scalable systems from small isolated parallel components. It covers detecting and correcting errors at run time, changing code while it runs, and how Erlang embodies these principles.

An AWS re:Invent 2018 session diving into the Nitro system, a set of hardware offload and security building blocks behind next-generation EC2 instances. It covers the design and architecture and how Nitro enables new instance types.

Bryan Cantrill's GOTO Chicago 2017 talk on debugging production systems under pressure, centered on the incident where an operator's mistyped command rebooted an entire Joyent datacenter. It covers human factors, automation and architecture.

A 2008 Google Tech Talk by the creator of Org-mode, introducing the Emacs subsystem for plain-text note-taking and project planning. It covers outlines, TODO states, tags, dates and priorities, and later shows using Org-mode as a spreadsheet.

Max Brunsfeld introduces Tree-sitter, a parsing system for developer tools that offers incremental parsing and robust error recovery through a uniform C API. The talk explains the design, based on LR extended to GLR parsing.

Dan Holohan, author of The Lost Art of Steam Heating, talks about the old steam heating systems he learned about from veteran tradesmen. He shares stories and practical lessons on how these systems work, including running steam at low temperatures.

Max Brunsfeld of GitHub presents Tree-sitter, a parsing system for programming tools that supports incremental, error-tolerant parsing. He demos its integration into GitHub.com and Atom for syntax highlighting and code navigation.

Bryan Hunter's GigCityElixir 2023 talk on HCA Healthcare's Waterpark system, built on Elixir/Erlang. It describes how the architecture is distributed and fault-tolerant, with a multi-datacenter cluster and runtime code swapping.

Andrew Tanenbaum's FOSDEM 2010 talk on MINIX 3, a microkernel OS built for high reliability and security. It shows how the system detects and recovers from device driver crashes, and addresses microkernel performance concerns.

The 1969 pitch film in which Saul Bass presents the Bell System logo redesign to AT&T, part of the largest corporate re-identity program in U.S. history.

A 1994 talk by systems thinker Russ Ackoff, who knew W. Edwards Deming, on continuous versus discontinuous improvement. It ends with an unexpected lesson.

The SILE author's FOSDEM 2015 talk introducing SILE, a TeX-inspired typesetting system with frame-based layout, Unicode, OpenType and a Lua-based extensible design.

Robert Moss's JuliaCon 2015 talk describes using Julia as a specification language for ACAS X, the next-generation airborne collision avoidance system that succeeds TCAS.