learning from data

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.

Raymond Hettinger explains Python's dataclasses module, showing the problem it solves and its key design decisions. He places it in a progression from tuples and dicts through named tuples and attrs.

Max Tegmark's MIT CBMM talk on why deep learning works so well. He argues that the laws of physics and the symmetries in physical data make it compressible into small neural networks, tying physics to network structure.

John Ousterhout's Netdev 0x16 keynote arguing that nearly every aspect of TCP is wrong for the datacenter, from connections and streams to reliable delivery and congestion control. He presents the case for a replacement protocol.

Geoff Hinton's 2010 Google Tech Talk on deep belief networks, learning layers of representation one at a time from unlabeled data, and extending them with multiplicative interactions.

James Powell's nearly two-hour PyData talk builds up Python's data model, decorators, generators and context managers, coding live in Vim. It aims to take intermediate programmers toward real Python expertise.

Mike Acton of Insomniac Games argues that a program's purpose is transforming data, and shows how common C++ approaches work against that in performance-critical game development. He covers cache behavior, data layout and the limits of compilers.

Juan Pedro Bolivar Puente explains immutable and persistent data structures in C++ and how they support value semantics. He presents the Immer library, rrb-trees, and a prototype text editor that opens a huge file instantly.

Valve's John McDonald presents at GDC 2018 on how Valve uses deep learning (VACnet) to detect cheaters in Counter-Strike: Global Offensive. He covers training on human-reviewed replays and how model output is confirmed by manual review.

Lee Byron of Facebook explains how persistent immutable data structures work and how to use them in React apps with Immutable.js. He covers memoization benefits and avoiding coupling through shared mutable state.

Rich Hickey, creator of Clojure and designer of Datomic, gives a JaxConf 2012 talk that takes apart traditional database architecture. He presents a different way to think about databases and explains the reasoning behind Datomic's fact-based design.

A GDC 2018 talk by Valve's John McDonald on VACNet, a deep learning system that analyzes CS:GO gameplay to detect cheaters. It covers how the model is trained on human-reviewed Overwatch cases and how its output feeds human moderation.

UC Berkeley EECS colloquium by OpenAI's John Schulman on reinforcement learning from human feedback. He covers why RL helps with truthfulness, hallucination, and the limits of behavior cloning.

Edward Kmett's 2013 talk at Prezi in Budapest on succinct data structures, which store data in close to the information-theoretic minimum space while supporting fast queries. It ties together the research and walks through his Haskell succinct library.

Jon Gjengset's 2020 PhD dissertation presentation at MIT on partial state in dataflow-based materialized views, the idea behind the Noria database. The Q&A is cut out.

A Google Tech Talk by Laurens van der Maaten, co-creator of t-SNE, on visualizing high-dimensional data by mapping it to two dimensions. It explains how the technique works and how it compares with other visualization approaches.

James Hamilton of AWS gives a Velocity 2010 keynote on datacenter infrastructure innovation, focused on power and cooling costs and a breakdown of where datacenter spending goes.

Ali Rahimi accepts the NIPS 2017 Test of Time Award and argues that machine learning has become alchemy, with practice running ahead of understanding.

A five-minute excerpt of Rich Hickey's Clojure Made Simple talk, with new slides, ranting about HttpServletRequest and specificity. He argues for using generic maps instead of many specific classes and getters.

Simon Peyton Jones explains nested data parallelism and Data Parallel Haskell, an approach to exploiting multicore and distributed hardware while keeping a simple single-control-flow programming model.

A CppCon 2018 talk arguing that OOP has failed to deliver on reuse and performance, and presenting data-oriented design for C++ with practical worked examples from real systems.

Bill Dally of NVIDIA gives a Hot Chips 2023 keynote on the challenges of building hardware for deep learning. He traces how GPUs have been adapted for machine learning, especially training.

Peter Norvig, then Director of Research at Google, talks at Startup School 2008 about leveraging large amounts of data and about technical challenges in Google projects such as image search. A central idea is that data can beat hand-written programming.

Mark Raasveldt of CWI presents DuckDB, an embeddable analytical database described as the SQLite for analytics, in a CMU Database Group talk. He explains why it was built, what it is for, and how it works.

Andrej Karpathy's Databricks talk on Software 2.0, the shift from human-written code to code produced by neural network training. It covers how developer work moves toward curating datasets, the advantages and challenges, and opportunities for new tooling.

Martin Kleppmann argues that databases are global shared mutable state and proposes treating them as an ever-growing log of immutable facts. Derived data such as caches, indexes and views is then built by stream processors like Apache Samza. The talk draws on Kafka and LinkedIn's experience with commit logs.

Chandler Carruth's CppCon 2014 talk on writing fast C++, explaining why data structure choice and memory layout matter more than clever algorithms. It covers why standard containers like unordered_map perform poorly on modern CPUs.

Veritasium's Derek Muller gives a Perimeter Institute livestream on how AI might change teaching and learning. He draws on past technology shifts and cognitive science to argue that learning requires effortful repetition.

Bryan Cantrill of Joyent presents Manta, an internet-facing object store that lets you run compute directly on the data. The talk is peppered with ZFS history and Cantrill's trademark commentary on the industry.

Jason Scott's DerbyCon 2012 talk on rescuing the Prince of Persia source code from old floppy disks, working with the game's author Jordan Mechner. It is a story of digital archaeology and recovering data from ancient media.

Andrew Kelley, creator of Zig, presents practical data-oriented design techniques for shrinking memory footprint and making programs cache friendly. He ends with a case study of how these ideas sped up the Zig compiler.

SQLite creator Richard Hipp gives a history of SQLite and explains where it shines as a database at the edge of the network with no DBA. He covers long-term support, testing and how it is used in products.

Mike Rowe, host of Dirty Jobs, shares stories from messy real-world jobs in this TED talk. He argues that hard work has been unjustly degraded in society.

Alexander Nix of Cambridge Analytica presents at the 2016 Concordia Summit on using big data, psychographic profiling and behavioral microtargeting in elections.

Luca Sas's ACCU 2021 talk on Modern C, a style of C and C-like C++ aligned with data-oriented design. It covers centralized memory management, zero-is-initialization, error handling and API design.

Zoz's DEF CON 23 talk explores practical, often explosive ways to destroy data and hard drives quickly. It tests methods such as thermite and other physical techniques for rapid destruction.

Raph Levien presents his new Rust GUI framework, built on data-oriented design and ideas from entity component systems, for the Xi editor. He weighs the approach against alternatives and discusses pros and cons.

Michael Arntzenius's Strange Loop talk on Datafun, a functional query language that brings database-style querying into a general-purpose language. It discusses bridging database theory and programming languages.

Cambridge Analytica CEO Alexander Nix keynotes the OMR 2017 marketing festival on big data and personality-based targeting in political campaigns. It was given before the company's scandal broke.