What I Wish I Had Known Before Scaling Uber to 1000 Services • Matt Ranney • GOTO 2016
GOTO Conferences3 HN recommenders · recommended in 3 months, 2016-2023
334K views · 10 years ago · talk · dated
Matt Ranney of Uber talks at GOTO Chicago 2016 about what Uber learned from growing past 1,000 production microservices. He covers regrets, sprawl and organizational effects.
Often brought up as an honest account of microservice sprawl, including the fact that nobody at Uber knew exactly how many services existed.
53:14Mastering Chaos - A Netflix Guide to Microservices
InfoQ
2.3M views • 9 years ago
45:27Avoiding Microservice Megadisasters - Jimmy Bogard
NDC Conferences
439K views • 9 years ago
1:22:45Building Software Systems At Google and Lessons Learned
Stanford
257K views • 15 years ago
52:41Seattle Conference on Scalability: YouTube Scalability
Google TechTalks
30K views • 19 years ago
1:06:08Scaling the Linear Sync Engine
Linear
37K views • 3 years ago
42:20"Stop Rate Limiting! Capacity Management Done Right" by Jon Moore
Strange Loop Conference
41K views • 9 years ago
56:19MINIX 3: a Modular, Self-Healing POSIX-compatible Operating System
FOSDEM
90K views • 16 years ago
1:10:23"Systems that run forever self-heal and scale" by Joe Armstrong (2013)
Strange Loop Conference
85K views • 5 years ago
47:51AWS re:Invent 2014: AWS Innovation at Scale with James Hamilton (SPOT301)
Amazon Web Services
70K views • 11 years ago
1:08:17How We've Scaled Dropbox
Stanford
372K views • 14 years ago
48:30Design Microservice Architectures the Right Way
InfoQ
741K views • 8 years ago
5:28Inside a Google data center
Google Workspace
22.6M views • 11 years ago
49:15Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp
Erlang Solutions
50K views • 12 years ago
54:14Rick Reed - WhatsApp: Half a billion unsuspecting FreeBSD users
iX
31K views • 11 years ago
1:33:27AWS re:Invent 2016: Tuesday Night Live with James Hamilton
Amazon Web Services
133K views • 9 years ago
38:43Scalability at YouTube
Next Day Video
51K views • 14 years ago
50:06The Many Meanings of Event-Driven Architecture • Martin Fowler • GOTO 2017
GOTO Conferences
677K views • 9 years ago
1:19:35Lecture 1: Introduction
MIT 6.824: Distributed Systems
921K views • 6 years ago
56:39Alan Kay - Programming and Scaling
Colin McDonnell
36K views • 11 years ago
1:26:02Stanford Seminar - The Soul of a New Machine: Rethinking the Computer
Stanford Online
76K views • 6 years ago
9 comments from Hacker News
Apparently they started at 1000 and went from there... "What I Wish I Had Known Before Scaling Uber to 1000 Services" - ▶ this video
At a goto; talk, an Uber engineer said they had so many microservices it was impossible to really count how many they had: ▶ this video
Having worked at different companies adopting both monorepo and the multiple repos approach, I find monorepo a better normalizer at scale in consolidating all "software" that runs the company. Just like what many commenters here have mentioned, the monorepo approach is a forcing function on keeping compatibility issues at bay. What you don't want is to end up in a situation where teams reinvent their own wheels instead of building on top of existing code, and at scale, I think the multiple repo approach tends to breed such codebase smell. [1] I'm sure 8000 repos is living hell for most organizations. [1] - ▶ this video
1) Do not attempt to live code. change your code via version control, build small videos, copy and paste... whatever you do... do not try and live code 2) Read "The presentation secrets of Steve Jobs" ... its not really about Steve Jobs so much as its about how to be a great presenter. 3) building or writing a talk is significantly less important than the delivery... which means that more of your time should be spent on how to deliver the speech (recording I find is the best way to practice) 4) Pick a single thing and do it well, do not attempt to put too much into a talk or tackle something massively complex. 5) Make sure that everyone is walking away with something they can actually use. This doesn't require it be a hard skill... "Soft" or philosophical utilities are just as good 6) Have Fun and be funny... Speakers have a tendency to want to show off the size of their intellect, and the level their competency- having a speech be entertaining is way way more important. 7) DO NOT READ FROM YOUR SLIDES. know your shit. slides are supplementary Extra Credit. Here's a collection of some fantastic technical speeches: * link * link * link * link * link * ▶ this video
I think as technologists we have a natural preference for clean technological solutions that don't involve politics or messy interpersonal relationship issues. Failing to confront that whole human sphere of life leads to negative consequences everywhere, and the talk does a great job of pointing out some of the macro social effects. On a similar note but a more localized scale I'm reminded of this talk on one of the downsides of moving to microservices being that people can avoid difficult conversations ▶ this video
Little drunk so bear with me :) Rich Hickey: Clojure concurrency[ link ] Keynote - Rust: confident, productive systems programming by Aaron Turon [ link ] RustConf 2016 - A Modern Editor Built in Rust by Raph Levien [ link ] Microservices at Netflix Scale: Principles, Tradeoffs & Lessons Learned[ link ] What I Wish I Had Known Before Scaling Uber to 1000 Services[ ▶ this video ] Dynamic Code Optimization and the NVIDIA Denver Processor [ link ] * if you can get anything on Apple's Ax series that's even better
Total armchair observer viewpoint - In reading Susan's blog, there are a lot of sexual harassment claims but there are many claims of cloak and dagger behavior. This points to a dysfunctional organization that lacks trust. And in that, I couldn't help but think of Conway's law [1] and the video from the goto; conference [2] about Uber's infrastructure. "We have so many services in production we don't even know how many there are." This may reflect a few different trust issues. One of which includes "Not invented here" thus building services that are duplicative. The goal of which is to show value and get promoted. Given the blog, the communication and reporting structure is also broken. Which, results in >1000 services in production and it being unclear how many there are and what they all do. Most likely, this is how the teams are structured as well. [1] - link [2] - ▶ this video
You're going to talk about Uber? Watch this video: [1]. He talks about every regret regarding their infrastructure. At first you kind of nod your head along with "yeah, it's not that easy", but as he keeps talking you can't help but wonder why the company has absolutely no internal technical direction whatsoever . They employ over 2000 engineers (just devs, excluding support staff, etc.). They use micro-services designed to the point where teams can't reuse pre-existing services. They are rewriting the same functionality over and over, and over and over again, as separate teams don't even have a clue what has already been written. Pick a piece of functionality you need. There are straight up 20 different APIs that do the exact same thing , but good luck figuring out which one you should be using - oh wait, why bother when you can just write the 21st version of the same thing. They are at the point where they have no clue what their repositories contain. In the video, he can't even provide an accurate number of deployed micro-services. THEY CANNOT EVEN ASSESS WHAT THEY ARE RUNNING IN PRODUCTION . They're literally drowning, and cannot recover. The point of the article is wrong. It's not about what can be done in one weekend, it's what can be done in 12-24 months by a competent team. Yes, Uber operates internationally which comes with a very crazy set of challenges. 2000 engineers worth of challenge? Not even remotely close. You should need fewer than 100 people who really know
Joking aside I enjoyed that video ( ▶ this video - for those who didn't see it), the presenter was pretty honest and way more open than I expected. I'm just so curious how those services break down - if it's genuinely on the "lstrip as a service" level or whatever, whether that's "1,000 services" figure was actually a relatively simple API with 10 versions at various levels of deprecation, whether he meant it was 1,000 different service endpoints (250 services each with POST, GET, PUT, DELETE) or what. I want to know more basically :)