misconceptions

A five-minute lightning demo of VisiData, a terminal tool for exploring and manipulating data. It uses two months of NYC 311 complaint data to drill down, plot points on a virtual map and save selected rows.

John Kitchin shows how Emacs org-mode with Python lets researchers combine narrative, math, figures and executable code in one document. It exports to LaTeX, HTML and PDF, and he used it for papers and e-books.

Jeremy Howard defends Jupyter notebooks as a serious development environment, especially when paired with nbdev. He responds directly to Joel Grus's talk about why he dislikes notebooks, and contrasts REPL-style and notebook-style work.

A reupload of Joel Spolsky's talk teaching spreadsheet users the Excel features they are probably missing, such as keyboard navigation, named ranges, tables and lookups. The original disappeared and this copy comes from the Wayback Machine.

Database pioneer Michael Stonebraker argues that 'big data' is really at least four different problems: big volumes with small analytics, big analytics, velocity, and variety. He comments on where each market is heading.

Fons van der Plas introduces Pluto.jl, a reactive notebook for Julia that tracks references between cells and re-runs affected cells when code changes. The talk was given at JuliaCon 2020.

Dan Robinson, then CTO of Heap, explains how Heap runs its product analytics backend on PostgreSQL and CitusDB. It covers conversion funnels, retention and cohort queries across tens of billions of events and 50 TB of data, plus Postgres and Kafka together and the indexing strategy.

David Beazley's PyData Global 2021 keynote, an oral history of early scientific Python. He tells how Python ended up running on a Connection Machine supercomputer at Los Alamos.

David Kriesel's 36C3 talk, in an English translation, analyzes a year of scraped Deutsche Bahn data covering every stop of every long-distance train in Germany. It explores delays and how punctuality is actually measured.

Sandy Pentland of MIT presents his book on social physics, which uses big-data studies of how people interact to explain how ideas spread. He describes findings on collective intelligence and social influence.

Robert Erdmann, a materials scientist at the Rijksmuseum, gives a PyCon keynote on the ultra-high-resolution imaging of Rembrandt's Night Watch. He covers the automated gantry camera system, the Python software that drives it, and the large volumes of image data involved.

John Rauser's Strata 2014 keynote argues that programmers can learn statistics quickly by simulating random processes and shuffling data. He explains p-values through resampling rather than formulas.

Mark Raasveldt of CWI presents DuckDB, an embeddable analytical database described as the SQLite for analytics, in a CMU Database Group talk. He explains why it was built, what it is for, and how it works.

Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.

Michael Betancourt's lecture at MLSS Iceland 2014 on efficient Bayesian inference with Hamiltonian Monte Carlo. It covers MCMC concepts and motivation and the problems of sampling in high-dimensional distributions.

Peter Norvig's UBC distinguished lecture on how massive amounts of simple data, such as for language modeling, can yield understanding that hand-built models could not.

Jan Vitek of Northeastern University gives a meetup talk on the challenges of making the R language run fast. He explains what in R's semantics makes it hard to optimize and to build a JIT compiler for.

Jake Vanderplas's PyCon 2016 talk shows how sampling-based methods such as simulation, shuffling and bootstrapping make statistics intuitive. His pitch is that if you can write a for-loop, you can do statistics.

David Sanders gives the first part of a two-part introduction to the Julia programming language at SciPy 2014. It walks through the language's basics and its appeal for scientific computing.

Geoff Cumming uses a simulation to show how wildly the p value varies from one replication to the next because of sampling variability. He argues for estimation and confidence intervals over null hypothesis significance testing.

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

Nassim Taleb argues that evidence on terrorism risk presented in media is statistically naive, using extreme value theory without math to separate fat-tailed risks like Ebola and terrorism from thin-tailed ones.

A JuliaCon talk by a NASA Launch Services Program engineer on modeling spacecraft separation dynamics in Julia. He describes replacing a Simulink-based workflow for checking launch vehicle separation requirements with Julia and ModelingToolkit, with large speedups and much shorter code.

A Berkeley lecture by psychologist Christina Maslach, a leading burnout researcher, on what burnout is and what causes it. It covers common misunderstandings and the job and workplace factors involved.