data structures

Edward Kmett's 2013 talk at Prezi in Budapest on succinct data structures, which store data in close to the information-theoretic minimum space while supporting fast queries. It ties together the research and walks through his Haskell succinct library.

Juan Pedro Bolivar Puente explains immutable and persistent data structures in C++ and how they support value semantics. He presents the Immer library, rrb-trees, and a prototype text editor that opens a huge file instantly.

Chandler Carruth's CppCon 2016 talk on designing hybrid data structures for performance. He covers cache locality, working-set size and allocation overhead, and why standard C++ containers fall short.

Sean Parent's CppCon 2015 talk on better use of standard containers and the goal of having no incidental data structures. It argues that proper containers can greatly simplify code.

Chandler Carruth's CppCon 2014 talk on writing fast C++, explaining why data structure choice and memory layout matter more than clever algorithms. It covers why standard containers like unordered_map perform poorly on modern CPUs.

Erik Demaine's MIT 6.851 Advanced Data Structures lecture introducing succinct data structures, covering rank, select and tries. It is part of the Spring 2012 course.

Lee Byron of Facebook explains how persistent immutable data structures work and how to use them in React apps with Immutable.js. He covers memoization benefits and avoiding coupling through shared mutable state.

The first lecture of Andy Pavlo's CMU 15-445 Intro to Database Systems (Fall 2019), covering course introduction and the relational model.

James Powell's nearly two-hour PyData talk builds up Python's data model, decorators, generators and context managers, coding live in Vim. It aims to take intermediate programmers toward real Python expertise.

Mike Acton of Insomniac Games argues that a program's purpose is transforming data, and shows how common C++ approaches work against that in performance-critical game development. He covers cache behavior, data layout and the limits of compilers.

A five-minute lightning demo of VisiData, a terminal tool for exploring and manipulating data. It uses two months of NYC 311 complaint data to drill down, plot points on a virtual map and save selected rows.

John Oliver's Last Week Tonight segment explains how data brokers collect and sell personal information, including a stunt in which his team bought data to identify members of Congress.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Rich Hickey, creator of Clojure and designer of Datomic, gives a JaxConf 2012 talk that takes apart traditional database architecture. He presents a different way to think about databases and explains the reasoning behind Datomic's fact-based design.

Benn Jordan investigates infrasound and low-frequency noise from data centers, including xAI's Colossus and a crypto mine, and reports his own experiments on the effects. The video covers the health and community impacts on nearby residents.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

A Business Insider investigation into the US boom in AI data centers, mapping where they are, who owns them, and how much power and water they consume. It looks at the costs borne by communities, with a focus on Virginia.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Eager Space examines the practicality of putting data centers in orbit, walking through three scenarios with cost calculations and the launch cost reductions that would be required.

John Ousterhout's Netdev 0x16 keynote arguing that nearly every aspect of TCP is wrong for the datacenter, from connections and streams to reliable delivery and congestion control. He presents the case for a replacement protocol.

Tantacrul examines sonification, the practice of turning data into sound or music, and critiques how scientists and musicians, including NASA and CERN projects, use it well and poorly.

Scott Manley examines why SpaceX, Google and others are looking at orbital data centers. He walks through the economics, heat rejection and solar power arguments, with links to the Lumen/Starcloud and Google Suncatcher papers.

Scott Manley explains the thermal balance of spacecraft and how a satellite could be configured to radiate away the heat from datacenter-scale computing in space. His rough numbers suggest it is feasible.

Raymond Hettinger explains Python's dataclasses module, showing the problem it solves and its key design decisions. He places it in a progression from tuples and dicts through named tuples and attrs.

Scott Manley analyzes the idea of putting data centers in orbit, prompted by Lumen Orbit's funding. He examines the claimed advantage of constant solar power against launch cost, cooling and mass.

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

Jon Gjengset's 2020 PhD dissertation presentation at MIT on partial state in dataflow-based materialized views, the idea behind the Noria database. The Q&A is cut out.

Linus Tech Tips documents a major RAID failure on its Whonnock server and the attempt to recover the lost data with a data recovery firm.

James Hamilton of AWS gives a Velocity 2010 keynote on datacenter infrastructure innovation, focused on power and cooling costs and a breakdown of where datacenter spending goes.

AWS's short promo for Snowmobile, a 45-foot shipping container pulled by a semi truck that moves up to 100PB of data to AWS.

An LGR Oddware episode looking at Sony's MDH-10 MD Data drive from 1995, which used MiniDisc for computer storage years before Hi-MD. The host tries out the drive with its original drivers and software.

Joe Kava, VP of Google's Data Center Operations, gives a short tour of a Google data center. He covers security, sustainability and the core infrastructure architecture.

A how-to video on datamoshing, the glitch-art technique of removing keyframes so that motion data smears across video. It walks through the tools needed, such as ffmpegx and Avidemux, though the author notes parts are outdated.

Python core developer Raymond Hettinger explains how Python's dictionary implementation evolved into the compact, ordered dict in Python 3.6. He uses pictures and small bits of pure Python, and covers key-sharing, compaction and versioning.

A 2007 Google Tech Talk in which Cliff Click of Azul Systems presents a lock-free concurrent hash table that scales to 768 CPUs. He explains why it is correct by reasoning in a different way than usual happens-before and memory-order arguments.

Simon Peyton Jones explains nested data parallelism and Data Parallel Haskell, an approach to exploiting multicore and distributed hardware while keeping a simple single-control-flow programming model.

Malte Skarupke's C++Now 2018 talk on hash table performance, explaining why std::unordered_map is slow and memory hungry and presenting faster designs, including his bytell hash map.

Kevin Fang comically retells GitLab's 2017 incident in which an engineer deleted the production database, based on the company's postmortems. It walks through the confusion and failed backups.

Fox Business interview in which Oracle co-founder Larry Ellison claims Oracle will beat Amazon at cloud databases. He argues the world's valuable data lives in Oracle and says Amazon struggled to move off it.

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.