immutable data

Lee Byron of Facebook explains how persistent immutable data structures work and how to use them in React apps with Immutable.js. He covers memoization benefits and avoiding coupling through shared mutable state.

Juan Pedro Bolivar Puente explains immutable and persistent data structures in C++ and how they support value semantics. He presents the Immer library, rrb-trees, and a prototype text editor that opens a huge file instantly.

Rich Hickey, creator of Clojure and designer of Datomic, gives a JaxConf 2012 talk that takes apart traditional database architecture. He presents a different way to think about databases and explains the reasoning behind Datomic's fact-based design.

The first lecture of Andy Pavlo's CMU 15-445 Intro to Database Systems (Fall 2019), covering course introduction and the relational model.

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.

James Powell's nearly two-hour PyData talk builds up Python's data model, decorators, generators and context managers, coding live in Vim. It aims to take intermediate programmers toward real Python expertise.

Mike Acton of Insomniac Games argues that a program's purpose is transforming data, and shows how common C++ approaches work against that in performance-critical game development. He covers cache behavior, data layout and the limits of compilers.

A five-minute lightning demo of VisiData, a terminal tool for exploring and manipulating data. It uses two months of NYC 311 complaint data to drill down, plot points on a virtual map and save selected rows.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Tantacrul examines sonification, the practice of turning data into sound or music, and critiques how scientists and musicians, including NASA and CERN projects, use it well and poorly.

Raymond Hettinger explains Python's dataclasses module, showing the problem it solves and its key design decisions. He places it in a progression from tuples and dicts through named tuples and attrs.

Edward Kmett's 2013 talk at Prezi in Budapest on succinct data structures, which store data in close to the information-theoretic minimum space while supporting fast queries. It ties together the research and walks through his Haskell succinct library.

Linus Tech Tips documents a major RAID failure on its Whonnock server and the attempt to recover the lost data with a data recovery firm.

James Hamilton of AWS gives a Velocity 2010 keynote on datacenter infrastructure innovation, focused on power and cooling costs and a breakdown of where datacenter spending goes.

AWS's short promo for Snowmobile, a 45-foot shipping container pulled by a semi truck that moves up to 100PB of data to AWS.

An LGR Oddware episode looking at Sony's MDH-10 MD Data drive from 1995, which used MiniDisc for computer storage years before Hi-MD. The host tries out the drive with its original drivers and software.

Joe Kava, VP of Google's Data Center Operations, gives a short tour of a Google data center. He covers security, sustainability and the core infrastructure architecture.

Simon Peyton Jones explains nested data parallelism and Data Parallel Haskell, an approach to exploiting multicore and distributed hardware while keeping a simple single-control-flow programming model.

Fox Business interview in which Oracle co-founder Larry Ellison claims Oracle will beat Amazon at cloud databases. He argues the world's valuable data lives in Oracle and says Amazon struggled to move off it.

David Nolen presents Om Next, the redesign of the ClojureScript Om library, drawing on Relay, Netflix's JSONGraph and Datomic. He covers the tradeoffs in handling the client-server boundary over immutable data.

Alexander Nix of Cambridge Analytica presents at the 2016 Concordia Summit on using big data, psychographic profiling and behavioral microtargeting in elections.

A CppCon 2018 talk arguing that OOP has failed to deliver on reuse and performance, and presenting data-oriented design for C++ with practical worked examples from real systems.

Zoz's DEF CON 23 talk explores practical, often explosive ways to destroy data and hard drives quickly. It tests methods such as thermite and other physical techniques for rapid destruction.

Michael Arntzenius's Strange Loop talk on Datafun, a functional query language that brings database-style querying into a general-purpose language. It discusses bridging database theory and programming languages.

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

A live-coded demo by Jonathan Blow showing how to get object-system-style expressiveness in a data-oriented way, covering structure-of-arrays layouts and composition. It shows features of his Jai language that make programs fast.

Martin Kleppmann's Kafka Summit 2018 keynote asks whether Kafka can be considered a database. He covers event logs and related data-systems ideas.

Chandler Carruth's CppCon 2016 talk on designing hybrid data structures for performance. He covers cache locality, working-set size and allocation overhead, and why standard C++ containers fall short.

Steve Yegge's OSCON Data 2011 talk imagines what you could do with Google-scale storage and compute a decade later. He argues smart people should work on hard problems such as biology rather than cat pictures, and the talk ends with a moment many took as him quitting Google.

Rich Hickey's JaxConf 2012 keynote examines what values are and why immutable values matter as information systems grow more complex and data gets bigger. It contrasts places with values and builds the case for immutability.

Chandler Carruth's CppCon 2014 talk on writing fast C++, explaining why data structure choice and memory layout matter more than clever algorithms. It covers why standard containers like unordered_map perform poorly on modern CPUs.

SQLite creator Richard Hipp gives a history of SQLite and explains where it shines as a database at the edge of the network with no DBA. He covers long-term support, testing and how it is used in products.

Sean Parent's CppCon 2015 talk on better use of standard containers and the goal of having no incidental data structures. It argues that proper containers can greatly simplify code.

Erik Demaine's MIT 6.851 Advanced Data Structures lecture introducing succinct data structures, covering rank, select and tries. It is part of the Spring 2012 course.

Journalist Matthew Cole's Black Hat 2013 talk on the CIA's operational security failures in the 2003 Milan kidnapping. Phone and location metadata exposed the team and led to arrest warrants.

Lukas Eder's Devoxx talk on how modern cost-based SQL optimizers rewrite and plan queries in ways developers would not expect. It argues that declarative SQL, used well, can run complex queries very fast.

Raph Levien presents his new Rust GUI framework, built on data-oriented design and ideas from entity component systems, for the Xi editor. He weighs the approach against alternatives and discusses pros and cons.

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

Jaron Lanier delivers the 2018 Willy Brandt Lecture in Berlin on who civilization is for, covering data, AI, tech platforms and the economics of who benefits, followed by a Q&A with Germany's data protection commissioner Ulrich Kelber.