HackerTube
HomeLongsLikedHistoryWatch laterCategoriesPlaylistsSettings

Scalability at YouTube

Next Day Video1 HN recommender · recommended in 1 month, 2012-2024
YouTube
51K views · 14 years ago · talk · dated

A PyCon 2012 talk by early YouTube engineer Mike Solomon on how YouTube scaled its architecture, which is mostly Python.

HN mostly cites it as evidence that YouTube was built on Python and as background on how it scaled.

Distributed systemsPython, Ruby & scripting languagesSoftware architecture & design#youtube#python#scalability#pycon#webarchitecture

6 comments from Hacker News

@teitoklien2024 · in “Ask HN: Video streaming is expensive yet YouTube "seems" to do it for free. How?”

They improve their stack on multiple fronts 1 - They use their own custom hardware encoders now optimised for their scale (previously they’d use a ton of intel cpu for encoding) This is often not possible for smaller startups to have their own custom hardware, but one could setup their own Xilinx FPGA based hardware accelerated encoding servers, Xillinx Alveo Boards are popular for media acceleration, they support both ffmpeg and gstreamer for acceleration needs. Nvidia Nvenc encoding is decent too but mostly for the most popular encoding types, and honestly an fpga based accelerator is probably better after cpu unless you have gpus lying around. 2 - Much lower bandwith costs, a lot of startups build their streaming applications on top of AWS, but that’s insanely expensive due to the insanely expensive price of AWS itself As a streaming service you could immediately cut down costs by shifting to ovhcloud, hetzner etc, they often have dedicated gameserver like offerings which are beneficial for streaming usecases too. That itself should allow you to bring down costs by 10x or even more (you have to choose the plan correctly on other tho, ovh, hetzner etc say you have unlimited bandwith on some packages but there is always a catch or they throttle you later, need to be careful about the offering you choose but otherwise a lot better) 3 - Having colocation hosts with your own servers setup and each video encoded at different sizes for different media devices also help only sendi

CitesView on HN
@Leftium2024 · in “Ask HN: Can I create a mobile and Web App using a Python/Python Framework?”

For reference, YouTube runs on Python[1,2,3]: > 1. Python and Django: YouTube’s backend is predominantly written in Python, offering a balance of performance and readability. > 2. Google Cloud Platform... > 3. Java and C++: YouTube also utilizes Java and C++ for specific backend services, as they provide better performance for certain tasks. --- A long time ago, I looked into these Python frameworks: - link - link [1]: link [2]: link [3]: ▶ this video

CitesView on HN
@Jugurtha2021 · in “Ask HN: When to switch startup backend from Python to Golang?”

Scalability at YouTube: ▶ this video The Python is the less likely think that will suffocate you. What did you suffer from in past projects with each of these languages?

CitesView on HN
@tbatterii2013 · in “Security incident update”

>Facebook and YouTube were both created later, and in PHP youtube was always done in python as far as I understand. ▶ this video But yeah, back in 2002-3 coldfusion was an option.

CitesView on HN
@gitarr2012 · in “We need more PyCon US 2013 submissions”

YouTube (owned by Google) is mainly Python. Here a talk from Pycon 2012, the technical part starts around 10 min. in: ▶ this video

RecommendsView on HN
@gitarr2012 · in “Poll: What's Your Favorite Programming Language?”

The real beauty of Python is the fact that it is easy for beginning programmers, yes, but it has also a lot to offer for experienced coders concerning all the advanced things they might need. It's no accident that services like Dropbox[1] or YouTube[2], both arguably very complex systems, are relying on Python as their language of choice. [1] link [2] ▶ this video

CitesView on HN
Seattle Conference on Scalability: YouTube Scalability52:41

Seattle Conference on Scalability: YouTube Scalability

Google TechTalks
30K views • 19 years ago
The Clean Architecture in Python49:54

The Clean Architecture in Python

Next Day Video
114K views • 12 years ago
Building Software Systems At Google and Lessons Learned1:22:45

Building Software Systems At Google and Lessons Learned

Stanford
257K views • 15 years ago
Scaling the Linear Sync Engine1:06:08

Scaling the Linear Sync Engine

Linear
37K views • 3 years ago
DjangoCon 2008 Keynote: Cal Henderson1:02:44

DjangoCon 2008 Keynote: Cal Henderson

Google for Developers
125K views • 18 years ago
Cory Benfield - Building Protocol Libraries The Right Way - PyCon 201631:38

Cory Benfield - Building Protocol Libraries The Right Way - PyCon 2016

PyCon 2016
15K views • 10 years ago
Mastering Chaos - A Netflix Guide to Microservices53:14

Mastering Chaos - A Netflix Guide to Microservices

InfoQ
2.3M views • 9 years ago
Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp49:15

Erlang Factory 2014 - That's 'Billion' with a 'B': Scaling to the Next Level at WhatsApp

Erlang Solutions
50K views • 12 years ago
Google Developers Day US - Python Design Patterns44:03

Google Developers Day US - Python Design Patterns

GoogleDeveloperDay
90K views • 19 years ago
Raymond Hettinger, Keynote on Concurrency, PyBay 20171:13:53

Raymond Hettinger, Keynote on Concurrency, PyBay 2017

SF Python
161K views • 9 years ago
Stop Writing Classes27:29

Stop Writing Classes

Next Day Video
962K views • 14 years ago
How To Design A Good API and Why it Matters1:00:19

How To Design A Good API and Why it Matters

Google TechTalks
453K views • 19 years ago
"Systems that run forever self-heal and scale" by Joe Armstrong (2013)1:10:23

"Systems that run forever self-heal and scale" by Joe Armstrong (2013)

Strange Loop Conference
85K views • 5 years ago
Python at Massive Scale - Stephen Simmons, Neil Slinger44:45

Python at Massive Scale - Stephen Simmons, Neil Slinger

PyData
10K views • 8 years ago
Build Your Own Async2:12:42

Build Your Own Async

David Beazley
60K views • 7 years ago
Understanding the Python GIL46:10

Understanding the Python GIL

David Beazley
127K views • 14 years ago
Nicholas Zakas: Scalable JavaScript Application Architecture52:23

Nicholas Zakas: Scalable JavaScript Application Architecture

YUI Library
68K views • 15 years ago
How We've Scaled Dropbox1:08:17

How We've Scaled Dropbox

Stanford
372K views • 14 years ago
Alan Kay - Programming and Scaling56:39

Alan Kay - Programming and Scaling

Colin McDonnell
36K views • 11 years ago
React Helsinki February 20201:16:07

React Helsinki February 2020

Fraktio
42K views • 6 years ago