cloud lab

The day 2 keynote at Google Cloud Next '17, announcing new GCP and G Suite products with demos and customer stories from Google leaders.

Cuong Do of YouTube describes how a handful of engineers scaled the site, covering hardware, software and database scalability challenges during its rapid growth. The talk was given at the 2007 Seattle Conference on Scalability.

The day one keynote of Google Cloud's 2016 GCP NEXT conference, with Sundar Pichai, Diane Greene, Eric Schmidt, Urs Holzle and customers. It includes a Cloud ML demo by Jeff Dean.

Apple security head Ivan Krstic gives a Black Hat 2016 talk on iOS security, covering HomeKit, Auto Unlock and iCloud Keychain. He explains key hierarchies and how data can stay safe even in adversarial clouds.

Eben Moglen speaks at NYU in 2010 about freedom in the cloud, arguing that centralized services and handing over personal data and server logs erode privacy and freedom. The talk lays out the motivation for FreedomBox, a personal server for your own data.

Kyle Drake of Neocities explains the bandwidth pricing pitfalls of the big cloud providers. He describes how to run high-volume, bandwidth-heavy infrastructure far more cheaply, including through co-location.

Dave Chinner's linux.conf.au 2012 talk on XFS scalability, covering pain points with large numbers of files and complex metadata, and the work to fix XFS metadata performance.

James Hamilton, AWS VP and distinguished engineer, gives an insider view of the innovations behind the AWS cloud at re:Invent 2014. He covers custom hardware, networking, regions and availability zones, and infrastructure at scale.

Anthony Marcar describes lessons from three years of using Clojure at WalmartLabs, including scaling a team and a large multi-project codebase in an enterprise setting.

Jonathan Blow talks at Gamelab 2018 about the design decisions behind Jai, the programming language he is building for game programmers. He covers low-friction development, quality of life and the effects of compile times.

Nicholas Zakas of Yahoo discusses architecture for large, modular JavaScript web applications. He lays out the theory behind building front ends that scale in complexity.

A PyCon 2012 talk by early YouTube engineer Mike Solomon on how YouTube scaled its architecture, which is mostly Python.

Martin Kleppmann's GOTO Berlin 2016 talk on how collaborative editors, offline apps, and multi-datacenter databases handle concurrent updates. It covers conflict resolution approaches including operational transformation and CRDTs, with a worked look at text CRDTs.

Brian Fitzpatrick and Ben Collins-Sussman argue that the 'genius programmer' ideal harms collaboration. They advocate trading ego for growth, sharing work early, and understanding how tools shape social behavior.

Redis Labs and Tim Davis present RedisGraph, which uses GraphBLAS to treat graph queries in Cypher as sparse linear algebra for lower latency.

Jean-Marc Jancovici gives an MIT Media Lab talk on whether technology can save us from climate change. He argues from a physical view of energy that economic growth, energy use and climate impact are tightly linked.

John Graham-Cumming of Cloudflare talks at dotScale 2015 about handling roughly 10 trillion log lines a month. The talk touches on Cloudflare's hardware, its logging stack and its use of Lua in Nginx.

Brian Krebs and Lance James present research on DDoS booter services at Black Hat 2013, including the SWATting Krebs suffered. HN knows it mainly for a later exchange between Krebs and Cloudflare's Matthew Prince.

R. Stanley Williams of HP Labs gives a keynote on the memristor, covering the physics, crossbar computation, and potential for memory and new computer architectures.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, explains how small contributions from many people online can digitize books. He then introduces Duolingo as a way to translate the web while people learn languages.

Martin Kleppmann's QCon London 2018 talk on keeping data consistent in distributed systems without a central leader. It surveys CRDT algorithms for syncing devices and collaborative editing over peer-to-peer networks.

John Carmack's roughly three-hour QuakeCon 2013 keynote, delivered from a few bullet points. Topics include his Haskell experiments with functional programming and static typing, game state, and cloud gaming.

Neil Fraser of Google presents Differential Synchronization, the algorithm behind MobWrite, for keeping multiple copies of a document in sync in real time. The talk argues the approach is robust, convergent and efficient.

Matt Godbolt tells the story of Compiler Explorer, from a hacky basement server to a cloud-scaled service handling tens of thousands of daily requests. He also shows its features and how it is built.

Ori Bernstein gives an overview of Plan 9 at a NYC*BUG meeting. He explains what the system is, how it is built, what has happened since Bell Labs, and what its design teaches, with a live demo.

Rob Pike gives an insider's account of the history of Unix and the development of key pieces of modern computing, including Bell Labs anecdotes.

A Netflix performance engineer presents how Netflix tunes EC2 instances, covering instance type choice, Xen modes, SR-IOV and kernel tuning. The goal is near bare-metal performance in the cloud.

Kelsey Hightower's opening keynote at KubeCon, delivered as a live-demo style talk about Kubernetes and cloud-native tooling.

FOSDEM 2020 talk from Collabora on pressure-vessel, an experimental mechanism that puts Steam games in containers on Linux. It covers using namespaces to give games their own runtime library stack independent of the host.

Rick Reed of WhatsApp explains at Erlang Factory 2014 how the service scaled to billions of messages with a very small engineering team. He covers the FreeBSD and Erlang stack, Mnesia usage, server counts and bottleneck work.

James Hamilton's Tuesday Night Live keynote at AWS re:Invent 2016, touring AWS infrastructure at scale including networking, power and custom silicon. It also features NASA JPL's Tom Soderstrom.

Eric Brandwine's AWS re:Invent 2015 session on how Amazon VPC networking works. It covers the problems the design solved, why traditional approaches do not scale, and the mechanisms used for customer isolation and VPC endpoints.

Dave Ackley gives a high-level introduction to robust-first computing, which prioritizes robustness over efficiency in computer architecture. A sorting demo illustrates the idea.

Martin Kleppmann surveys current research on CRDTs, the algorithms for merging concurrent edits across offline replicas. He covers the hard parts such as rich text, moves and merge results that are technically correct but poor.

Rick Reed of WhatsApp speaks at MeetBSD about running a messaging service on FreeBSD and Erlang at huge scale. He covers the stack, tuning and the challenges of millions of connections per host.

A Linear engineer talks about scaling the company's sync engine, which keeps client data local and synchronized. It follows an earlier talk on the same system.

An AWS re:Invent 2018 talk on how AWS limits the blast radius of failures. It covers design techniques such as cell-based architecture and shuffle sharding.

A 2006 Google TechTalk in which Luis von Ahn introduces human computation, using people's effort to solve problems like image recognition that computers cannot yet handle. It covers games with a purpose and related ideas such as CAPTCHAs.

Hans Rosling makes the case that the washing machine was the greatest invention of the industrial revolution. Using Gapminder graphics, he shows how electricity and economic growth free up time for reading and learning.

A seven-minute Google I/O 2014 talk by Simon Newton on how developers can make use of Google's global network. It describes the network's footprint and some of the services running on it.