scale model

Scott Wlaschin's NDC talk on using statically typed functional programming, F# in particular, to model a business domain. Types are used to represent the domain precisely and to encode business rules so that illegal states cannot be built, with the compiler acting as an instant unit test.

A PyData London 2018 talk on how JPMorgan scaled its Athena Python platform over ten years to about 35 million lines of code and thousands of developers. It covers infrastructure, tooling and culture for analytics and visualization at that scale.

Richard Feldman's elm-conf talk on designing models so that bugs from impossible application states cannot occur, using Elm's compiler and types. It demonstrates techniques for making invalid states unrepresentable.

Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.

Jay Parlar's Strange Loop talk shows how model checkers such as Alloy can find complex bugs in a system's design without running or reading any code. He walks through modeling architectures and checking every possible sequence of operations, including a real security bug and a UI feature-interaction bug.

Scott Wlaschin shows how statically typed functional languages, using F# examples, can model a business domain so that invalid states cannot be represented. The type system acts almost like an instant unit test at compile time.

A panel with Tony Hoare, Joe Armstrong and Carl Hewitt, hosted by Francesco Cesarini, on the history and future of concurrency. The three creators of CSP, Erlang and the actor model discuss their differing approaches.

Luke Wagner of Fastly explains what a WebAssembly component is, how it relates to familiar concepts, and what the Component Model enables. It emphasizes security, composability and modularity.

Simon Brown's Agile on the Beach 2019 talk presents the C4 model, a lightweight approach to diagramming software architecture with a shared vocabulary and simple notation.

James Hamilton, AWS VP and distinguished engineer, gives an insider view of the innovations behind the AWS cloud at re:Invent 2014. He covers custom hardware, networking, regions and availability zones, and infrastructure at scale.

Hyrum Wright shares lessons from five years of mass code migrations in Google's huge C++ codebase. He covers why to migrate, how hard it really is, and how Google updates core libraries and their users continually.

A JuliaCon talk by a NASA Launch Services Program engineer on modeling spacecraft separation dynamics in Julia. He describes replacing a Simulink-based workflow for checking launch vehicle separation requirements with Julia and ModelingToolkit, with large speedups and much shorter code.

A DEF CON 23 talk by Marc Rogers and Kevin Mahaffey walking through the architecture of the Tesla Model S, what Tesla got right and wrong, and how the interconnected systems can be hacked. It also covers the data the car collects.

John Graham-Cumming of Cloudflare talks at dotScale 2015 about handling roughly 10 trillion log lines a month. The talk touches on Cloudflare's hardware, its logging stack and its use of Lua in Nginx.

SpaceX engineers present at NVIDIA's GPU Technology Conference on a multi-physics code that simulates the full-scale Raptor methane engine. It covers GPU-based turbulent combustion modeling with adaptive grids and wavelet compression.

Google engineers describe Google's internal build system, a unified, language-agnostic system used across the company. It was presented as a 2012 NYC tech talk.

Rich Hickey's Clojure/Conj 2018 keynote argues that Maybe/Option types and rigid type signatures are a poor way to model data that changes over time. He builds the case around optionality, schema growth and compatibility, and presents the ideas behind Clojure spec as an alternative.

Zig creator Andrew Kelley's RedisConf 2020 talk on modeling data flow with asynchronous I/O in Zig. It explains how Zig's async/await avoids function coloring, and Loris Cro demonstrates OkRedis, a zero-allocation Redis client.

Bryan Cantrill's Scale By The Bay 2019 talk on software after Moore's Law. He covers the end of easy scaling and the move toward specialized, emerging architectures.

A Foresight Institute workshop video with two talks: Shawn Douglas on nanoscale instruments for visualizing small proteins, and Bret Victor on Dynamicland. Victor's part shows biomolecular design built in his Realtalk system with physical objects.

Psychologist Robert Epstein presents a Stanford EE380 colloquium on how internet algorithms such as search rankings can manipulate opinions, purchases and votes at massive scale. He also describes a new monitoring system for tracking such influence.

A dotScale 2014 talk by Robert Kennedy, a former healthcare.gov SRE who later worked at Google, about the engineering effort to rescue the troubled site. It explains the technical and organizational problems behind the failed launch.

Joe Armstrong's Strange Loop 2013 talk on building large scalable systems from small isolated parallel components. It covers detecting and correcting errors at run time, changing code while it runs, and how Erlang embodies these principles.

An AWS re:Invent 2018 session on how DynamoDB was built as a hyper-scale NoSQL database. It discloses underpinnings of the service and how it is run for over 100,000 customers.

An F8 2015 talk touring the tool-chain Facebook uses to let thousands of engineers build mobile apps, including its move to Mercurial, an Atom-based IDE and continuous integration.

Olivier Giroux's CppCon 2019 talk recounts the roughly ten-year effort to bring std::atomic and the C++ memory model to CUDA GPUs, through ten stories from that work.

Anthony Marcar describes lessons from three years of using Clojure at WalmartLabs, including scaling a team and a large multi-project codebase in an enterprise setting.

Rachel Potvin describes the scale of Google's codebase and its custom monolithic source repository. She explains the reasons for the monorepo model, the tooling that makes it workable, and its tradeoffs.

Sasa Juric explains what makes Erlang and Elixir suited to server-side systems, focusing on the BEAM concurrency model rather than syntax or ecosystem. He demonstrates processes, supervision, and fault tolerance.

Rick Houlihan's AWS re:Invent 2018 deep dive on advanced DynamoDB design patterns. It covers index overloading, GSI sharding, composite keys, graph processing and transactions.

Audio of Charlie Munger's 1995 Harvard speech on the psychology of human misjudgment, laying out a framework of cognitive biases and mental models for decision making. It covers factors like incentives, social proof and denial.

James Powell's nearly two-hour PyData talk builds up Python's data model, decorators, generators and context managers, coding live in Vim. It aims to take intermediate programmers toward real Python expertise.

Martin Fowler's GOTO 2012 introduction to NoSQL databases: where they came from, their data models, how consistency thinking differs, and when to use them. He argues they won't make relational databases obsolete and introduces polyglot persistence.

Herb Sutter's deep C++ and Beyond 2012 talk on atomics and the C++ memory model, covering why programmers chasing top performance need to understand hardware ordering. This is part one of two.

A five-minute excerpt of Rich Hickey's Clojure Made Simple talk, with new slides, ranting about HttpServletRequest and specificity. He argues for using generic maps instead of many specific classes and getters.

Bryan Cantrill of Oxide Computer Company gives a Stanford EE380 talk on rethinking the server. He argues that hyperscaler-style rack-scale machines with open firmware, RISC-V and Rust should be available to everyone, not just the giants.

Bryan Cantrill argues that programming languages and platforms reflect core values, and compares languages through that lens, with Rust as a main example. He draws on his experience with Solaris, DTrace and ZFS.

Dropbox engineer Kevin Modzelewski gives a Stanford colloquium on Dropbox's history and the technical problems it faced scaling, including detailed database schema decisions.

Luis von Ahn, creator of CAPTCHA and reCAPTCHA, explains how small contributions from many people online can digitize books. He then introduces Duolingo as a way to translate the web while people learn languages.

Greg Young's Code on the Beach 2014 talk on CQRS and event sourcing. He argues mature domains are built on facts that occurred rather than current state, and covers the business and technical benefits.