sqlite

D. Richard Hipp's PGCon 2014 keynote on SQLite's relationship to PostgreSQL. He explains that SQLite was originally written from PostgreSQL 6.5 documentation and that the two engines serve different classes of problems.

A 2006 Google TechTalk in which SQLite's author Richard Hipp introduces the embeddable, zero-configuration C database library. He covers its history, strengths and weaknesses, and where it beats client/server databases.

SQLite creator Richard Hipp gives a history of SQLite and explains where it shines as a database at the edge of the network with no DBA. He covers long-term support, testing and how it is used in products.

Will Wilson of FoundationDB explains how the team tested their distributed database using deterministic simulation. By running the whole system in a simulator with injected faults and reproducible seeds, they could find rare concurrency and network bugs.

Rich Hickey's JaxConf 2012 keynote examines what values are and why immutable values matter as information systems grow more complex and data gets bigger. It contrasts places with values and builds the case for immutability.

Rick Houlihan's AWS re:Invent 2018 deep dive on advanced DynamoDB design patterns. It covers index overloading, GSI sharding, composite keys, graph processing and transactions.

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.

Martin Kleppmann's GOTO Berlin 2016 talk on how collaborative editors, offline apps, and multi-datacenter databases handle concurrent updates. It covers conflict resolution approaches including operational transformation and CRDTs, with a worked look at text CRDTs.

Martin Fowler's GOTO 2012 introduction to NoSQL databases: where they came from, their data models, how consistency thinking differs, and when to use them. He argues they won't make relational databases obsolete and introduces polyglot persistence.

Bryan Cantrill's Uptime 2017 talk on the engineering challenges of the stateful data path, including storage, firmware and hardware bugs. It uses the 'zebras vs. horses' diagnostic heuristic to explain why rare failures keep appearing.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Peter Alvaro's Strange Loop 2015 talk argues that query languages let you state what you mean, and applies that to distributed programming. It covers Datalog, Dedalus and the Bloom lineage for reasoning about distributed systems.

Dan Robinson, then CTO of Heap, explains how Heap runs its product analytics backend on PostgreSQL and CitusDB. It covers conversion funnels, retention and cohort queries across tens of billions of events and 50 TB of data, plus Postgres and Kafka together and the indexing strategy.

Redis Labs and Tim Davis present RedisGraph, which uses GraphBLAS to treat graph queries in Cypher as sparse linear algebra for lower latency.

Konrad Malawski's Strange Loop 2023 talk shows how the FoundationDB team adopted Swift incrementally in a C++ codebase. It covers Swift's C++ interoperability, which let them move single functions or entire types over one at a time.

Lukas Eder's Devoxx talk on how modern cost-based SQL optimizers rewrite and plan queries in ways developers would not expect. It argues that declarative SQL, used well, can run complex queries very fast.

Markus Triska tours Prolog's most distinctive features, including its use as a database and query language, rule processing, constraint solving and compile-time rewriting. It also covers concise meta-interpreters.

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

Martin Kleppmann's Strange Loop talk on database transactions, covering why they were dropped by NoSQL, their reputation for being slow, and newer algorithms that make serializable transactions perform and scale. It touches on consistency, linearizability and the CAP theorem.

Dave Chinner's linux.conf.au 2012 talk on XFS scalability, covering pain points with large numbers of files and complex metadata, and the work to fix XFS metadata performance.

A Clojure/conj 2019 talk by Wilker Silva on how namespace-qualified keywords and context-free federated names can be used pragmatically for data modeling, drawn from his experience at Nubank. It builds toward a graph-based view of connecting data from many sources.

Cuong Do of YouTube describes how a handful of engineers scaled the site, covering hardware, software and database scalability challenges during its rapid growth. The talk was given at the 2007 Seattle Conference on Scalability.

Tom Caputi presents the design of native encryption for ZFS at the OpenZFS Developer Summit, including design decisions and how authentication uses checksum fields.

Michael Arntzenius's Strange Loop talk on Datafun, a functional query language that brings database-style querying into a general-purpose language. It discusses bridging database theory and programming languages.

A guest lecture Jon Gjengset gave at Two Sigma on using Rust for large, high-performance systems. It covers why Rust suits such projects, the Noria research database, a concurrent data structure used in it, and practical experience.

A 49-minute demo of Palantir Foundry by its chief architect. It walks through data integration, ML and AI workflows, analytics, and application building across the platform.

Markus Winand shows how SQL has evolved since SQL-92, comparing old-style solutions to modern features such as window functions and system versioning, and notes that many developers never learn them.

Matt Ahrens presents the FreeBSD Foundation sponsored RAID-Z Expansion project for ZFS at the June 2021 FreeBSD Developer Summit. It explains how a RAID-Z vdev can be grown by adding disks on the fly.

An AWS re:Invent 2018 session on how DynamoDB was built as a hyper-scale NoSQL database. It discloses underpinnings of the service and how it is run for over 100,000 customers.

Marshall Kirk McKusick gives a talk at EuroBSDCon tracing filesystem and storage development from 1979 to today, using the BSD Fast Filesystem as the focus. It covers early disk-geometry optimization through later contiguous layout and small-file performance work.

An AWS re:Invent 2019 session for advanced DynamoDB users on design patterns used by large customers, including Amazon retail. It covers GSI sharding, index overloading, graph processing, composite keys and transactional workflows.

A Clojure/conj 2014 talk by Lucas Cavalcanti and Edward Wible on four uses of Datomic. They cover HTTP caching, audit trails, mobile database sync and authorization, drawing on the immutable transaction log.

Greg Young, who coined CQRS, gives an in-depth GOTO 2014 talk on what event sourcing is, when it is valuable and when it is not, along with its trade-offs and common misconceptions.

Jon Gjengset's 2020 PhD dissertation presentation at MIT on partial state in dataflow-based materialized views, the idea behind the Noria database. The Q&A is cut out.

Greg Young, who coined CQRS, looks back on a decade of domain-driven design, CQRS and event sourcing at DDD Europe 2016. He discusses where event sourcing applies and how simple its core can be.

Martin Kleppmann's Local-First Conf talk reviewing where the local-first idea came from, where it stands five years after the essay, and what the community should work toward. He argues for generic sync infrastructure.

Ahmon Dancy of Franz Inc explains low-level programming in Common Lisp, including off-heap data structures used in AllegroGraph to manage memory outside the garbage collector.

Martin Fowler's GOTO Chicago 2017 talk on the different things people mean by event-driven architecture. He identifies four patterns, including event notification and event-carried state transfer.

Uber Engineering's Joseph Gilley explains why Uber uses hexagons for geospatial indexing and introduces H3, its open source hierarchical grid system. He covers use cases and how distortion is handled.

Linus Tech Tips documents a major RAID failure on its Whonnock server and the attempt to recover the lost data with a data recovery firm.