performance testing

Sandi Metz's RailsConf 2013 talk offers practical guidelines for deciding what to test and how, so tests stay cheap and stable. She argues for testing the public interface rather than implementation and using mocks appropriately.

Will Wilson explains autonomous testing: using deterministic simulation and fuzzing, building on FoundationDB's approach, to test software continuously. It sets out the ideas behind his later company Antithesis.

John Hughes, co-creator of QuickCheck, talks about property-based testing of hard systems. He shows production examples where it found very low-level bugs, including issues in the CAN bus standard.

Hillel Wayne's PyCon 2018 talk on testing beyond pytest and mypy. It introduces property-based testing with the Hypothesis library and design-by-contract with dpcontracts, and shows how to combine them with an existing test suite.

A Netflix performance engineer presents how Netflix tunes EC2 instances, covering instance type choice, Xen modes, SR-IOV and kernel tuning. The goal is near bare-metal performance in the cloud.

Chandler Carruth's CppCon 2014 talk on writing fast C++, explaining why data structure choice and memory layout matter more than clever algorithms. It covers why standard containers like unordered_map perform poorly on modern CPUs.

A Google Tech Talk on Yahoo's 14 rules for faster web pages and the YSlow tool. It covers front-end practices like avoiding @import and fixing ETags, with live analysis of popular sites.

Malte Skarupke's C++Now 2018 talk on hash table performance, explaining why std::unordered_map is slow and memory hungry and presenting faster designs, including his bytell hash map.

Emery Berger's CppCon 2020 plenary on why performance is hard on modern hardware after the end of Dennard scaling. It covers benchmarking pitfalls and introduces new profiling tools for finding real bottlenecks.

Alen Ladavac of Croteam talks about continuous integration and automated testing pipelines in games, with case studies from The Talos Principle and Serious Sam.

Ted Young of LightStep proposes trace testing, using distributed tracing data to unify testing and observability. The KubeCon-era talk argues for testing against trace data rather than treating tests and monitoring separately.

Casey Muratori shows how typical 'clean code' object-oriented patterns can cost large factors in performance. He benchmarks them against simpler data-oriented code.

Destin of Smarter Every Day goes behind the scenes at the 23andMe lab to show how genotyping works. He also discusses DNA testing and privacy.

Will Wilson of FoundationDB explains how the team tested their distributed database using deterministic simulation. By running the whole system in a simulator with injected faults and reproducible seeds, they could find rare concurrency and network bugs.

Emery Berger explains why performance is hard to measure and deliver on modern hardware, showing how code layout and alignment can swamp optimization effects. He presents Stabilizer for statistically sound evaluation and Coz, a causal profiler.

Gary Bernhardt's RubyConf 2012 talk on testing in isolation without mocking everything. It goes through behavior versus data, mutation versus immutability, and how a functional core with an imperative shell yields natural isolation and parallelism.

Kaze Emanuar explains how he rewrote and optimized the Super Mario 64 source code so it runs far faster on real N64 hardware. He covers performance bottlenecks and fixes in detail for his ROM hacks.

Casey Muratori's University of Twente guest lecture is a case study in how straightforward code can cut thousands of lines and tens of seconds of runtime down to a few dozen lines running in under a second. It uses concrete examples to show why software is often an order of magnitude slower than it should be.

Deviant Ollam, head of a physical penetration testing team, presents tactics for getting into buildings, from bypassing door levers with a wire to defeating bank doors. He argues that physical security is often surprisingly weak.

Alex Russell's Chrome Dev Summit 2016 talk on building web experiences that work on slow, flaky networks and low-power devices. It argues developers should focus on the performance of real-world mobile hardware rather than desktop machines.

Emery Berger argues that performance is hard to measure and deliver on modern systems, showing how code layout and link order can swamp optimization effects. He presents tools that evaluate performance more reliably.

Chandler Carruth walks through tuning C++ performance, from writing benchmarks to analyzing results on real CPUs and compilers. It focuses on small-scale problems such as loop kernels, data structures and algorithms, and on keeping compilers from optimizing benchmarks away.

Veritasium tells how Kodak film was fogged by radioactive fallout from the first atomic tests, letting the company detect nuclear testing before anyone announced it. The video also covers the deal to keep it quiet and how fallout contaminated packaging.

Cliff Click walks through a small performance example on a modern out-of-order CPU. He covers pipelining, ILP, CISC vs RISC, multicore, and why cache misses and the memory subsystem dominate performance.

A 2009 Google Tech Talk by Nicholas Zakas on the hidden performance pitfalls of JavaScript. It covers what is fast and slow in the language and how to avoid common bottlenecks.

MedCram's Dr. Roger Seheult summarizes a discussion with Dr. Michael Mina on cheap, paper-based at-home COVID-19 antigen tests. It explains why lower sensitivity can still work well for screening infectious people.

Raymond Hettinger's PyBay 2018 keynote on preventing, finding and fixing bugs under a time budget. He covers testing practice, type annotation tradeoffs and the limits of coverage metrics.

Digital Foundry investigates why Resident Evil Village on PC runs worse with Capcom's Denuvo DRM than a cracked version. They benchmark and confirm performance and frame-pacing differences.

GreatScott! tests ProLogium lithium ceramic batteries, a solid-state design. He measures their volumetric and gravimetric energy density, compares them to Li-ion and LiPo, and runs durability tests at the end.

Chandler Carruth's CppCon 2016 talk on designing hybrid data structures for performance. He covers cache locality, working-set size and allocation overhead, and why standard C++ containers fall short.

John Oliver's Last Week Tonight segment on standardized testing in American schools. It covers how tests drive school funding, the achievement gap, and the testing industry around companies like Pearson.

Gil Tene of Azul Systems explains common pitfalls in measuring latency and response time, including why averages mislead and the problem of coordinated omission. He introduces open source tools like HdrHistogram for better measurement and reporting.

Optiver's Carl Cook explains how to build high-frequency trading systems in C++, where the critical path runs rarely but must be extremely fast. He covers techniques for low and predictable latency, including avoiding main memory and cache misses.

J. B. Rainsberger argues that large integrated tests are a scam: they are slow, fail opaquely, and give a false sense of coverage. He advocates focused collaboration and contract tests at boundaries instead.

Misko Hevery's Clean Code Talk shows how to use object-oriented techniques such as polymorphism to remove repeated if and switch statements. The result is code that is easier to test, understand and maintain.

Casey Muratori responds on-stream to a dismissive Twitter post by demonstrating how slow Visual Studio's startup and debugger watch window are, comparing it to much older software and machines. It is a long rant about software performance and bloat.

Aaron Hsu shows how to manipulate and transform tree data in APL using arrays and inverted tables instead of pointers. He claims much smaller code, better performance and easier parallelisation.

Leptos creator Greg Johnston examines Rust/WebAssembly frontend frameworks against JavaScript frameworks using the JS framework benchmark. He argues that WASM is no longer too slow for DOM-heavy rendering.

Builder Matt Risinger freezes water-filled copper and PEX piping capped with SharkBite fittings to see which survives. The video compares how each pipe type handles freezing.

A Tiny Desk concert in which pianist Dan Tepfer plays with a Disklavier player piano driven by a program he wrote that responds to his playing. He performs four pieces from his Natural Machines project.