data storage

An LGR Oddware episode looking at Sony's MDH-10 MD Data drive from 1995, which used MiniDisc for computer storage years before Hi-MD. The host tries out the drive with its original drivers and software.

The first lecture of Andy Pavlo's CMU 15-445 Intro to Database Systems (Fall 2019), covering course introduction and the relational model.

Bryan Cantrill of Joyent presents Manta, an internet-facing object store that lets you run compute directly on the data. The talk is peppered with ZFS history and Cantrill's trademark commentary on the industry.

Linus Tech Tips documents a major RAID failure on its Whonnock server and the attempt to recover the lost data with a data recovery firm.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Rich Hickey, creator of Clojure and designer of Datomic, gives a JaxConf 2012 talk that takes apart traditional database architecture. He presents a different way to think about databases and explains the reasoning behind Datomic's fact-based design.

Jon Gjengset's 2020 PhD dissertation presentation at MIT on partial state in dataflow-based materialized views, the idea behind the Noria database. The Q&A is cut out.

AWS's short promo for Snowmobile, a 45-foot shipping container pulled by a semi truck that moves up to 100PB of data to AWS.

A short IBM behind-the-scenes documentary on how researchers moved individual atoms to make the world's smallest movie. It introduces the scientists and their work on atomic memory and data storage.

Bryan Cantrill's Uptime 2017 talk on the engineering challenges of the stateful data path, including storage, firmware and hardware bugs. It uses the 'zebras vs. horses' diagnostic heuristic to explain why rare failures keep appearing.

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

Kevin Fang comically retells GitLab's 2017 incident in which an engineer deleted the production database, based on the company's postmortems. It walks through the confusion and failed backups.

Fox Business interview in which Oracle co-founder Larry Ellison claims Oracle will beat Amazon at cloud databases. He argues the world's valuable data lives in Oracle and says Amazon struggled to move off it.

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.

Michael Arntzenius's Strange Loop talk on Datafun, a functional query language that brings database-style querying into a general-purpose language. It discusses bridging database theory and programming languages.

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

Martin Kleppmann's Kafka Summit 2018 keynote asks whether Kafka can be considered a database. He covers event logs and related data-systems ideas.

Tom7 (suckerpinch) builds and evaluates a series of absurd hard drives, including storage made from ping packets across the internet. The video accompanies his paper and mixes deep technical hacking with comedic storytelling.

Mark Raasveldt of CWI presents DuckDB, an embeddable analytical database described as the SQLite for analytics, in a CMU Database Group talk. He explains why it was built, what it is for, and how it works.

SQLite creator Richard Hipp gives a history of SQLite and explains where it shines as a database at the edge of the network with no DBA. He covers long-term support, testing and how it is used in products.

Lukas Eder's Devoxx talk on how modern cost-based SQL optimizers rewrite and plan queries in ways developers would not expect. It argues that declarative SQL, used well, can run complex queries very fast.

A short product demo of Dabble DB, a web tool for importing, entering and manipulating structured data with little effort. It was built in Smalltalk and later acquired by Twitter.

A Google Tech Talk by Raymond Blum on how Google backs up massive amounts of data such as Gmail and Picasa. It covers scaling challenges, failure types, and approaches that did not work, including tape.

Zoz's DEF CON 23 talk explores practical, often explosive ways to destroy data and hard drives quickly. It tests methods such as thermite and other physical techniques for rapid destruction.

James Powell's nearly two-hour PyData talk builds up Python's data model, decorators, generators and context managers, coding live in Vim. It aims to take intermediate programmers toward real Python expertise.

Mike Acton of Insomniac Games argues that a program's purpose is transforming data, and shows how common C++ approaches work against that in performance-critical game development. He covers cache behavior, data layout and the limits of compilers.

Juan Pedro Bolivar Puente explains immutable and persistent data structures in C++ and how they support value semantics. He presents the Immer library, rrb-trees, and a prototype text editor that opens a huge file instantly.

A five-minute lightning demo of VisiData, a terminal tool for exploring and manipulating data. It uses two months of NYC 311 complaint data to drill down, plot points on a virtual map and save selected rows.

Martin Fowler's GOTO 2012 introduction to NoSQL databases: where they came from, their data models, how consistency thinking differs, and when to use them. He argues they won't make relational databases obsolete and introduces polyglot persistence.

John Oliver's Last Week Tonight segment explains how data brokers collect and sell personal information, including a stunt in which his team bought data to identify members of Congress.

A five-minute excerpt of Rich Hickey's Clojure Made Simple talk, with new slides, ranting about HttpServletRequest and specificity. He argues for using generic maps instead of many specific classes and getters.

Lee Byron of Facebook explains how persistent immutable data structures work and how to use them in React apps with Immutable.js. He covers memoization benefits and avoiding coupling through shared mutable state.

Practical Engineering explains pumped storage hydropower, which makes up most grid-scale electricity storage. It covers how it works and why it fits only certain geographies.

Benn Jordan investigates infrasound and low-frequency noise from data centers, including xAI's Colossus and a crypto mine, and reports his own experiments on the effects. The video covers the health and community impacts on nearby residents.

MIT's Donald Sadoway presents the liquid metal battery as a design for grid-scale stationary electricity storage, at an Imperial College London event in 2018. He covers the grid benefits of massive-scale storage.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

A Business Insider investigation into the US boom in AI data centers, mapping where they are, who owns them, and how much power and water they consume. It looks at the costs borne by communities, with a focus on Virginia.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Eager Space examines the practicality of putting data centers in orbit, walking through three scenarios with cost calculations and the launch cost reductions that would be required.

Avi Bryant demos Dabble DB, a web-based spreadsheet-like database for building apps without code. It shows how users can model and relate data through a smooth interface.