HackerTube
HomeLongsLikedHistoryWatch laterCategoriesPlaylistsSettings

Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13

New Relic2 HN recommenders · recommended in 2 months, 2014-2022
YouTube
18K views · 13 years ago · talk · dated

Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.

HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.

Databases & storageOperating systems & kernelsData science & statistics#bryancantrill#manta#unix#map-reduce#objectstorage

3 comments from Hacker News

@btschaegg2022 · in “Ask HN: I’m New to Computer Science and Engineering. Any Advice?”

1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming languages, environments and paradigms, but that will likely apply to anything non-technical, as well). I gained this perspective from dealing with people who didn't really do that, but also from knowing that I personally also did this way too late (and from the benefits it brought me nonetheless). If you're only familiar with Windows, try Linux. Try BSD. Try OS X, if you want to. Try working in the terminal. Familiarize yourself with different editors (especially Vim and Emacs, maybe also newer ones like Kakoune). Learn bash, have a look at AWK and sed. Maybe learn Perl. Try out functional programming, logic programming (like Prolog) and so on and so forth. If you're feeling adventurous, play with niche systems like Haiku[1] or TempleOS[2], just to gain new insights on how things could work. Maybe try the ACME editor[3]. [1]: link [2]: link [3]: link Of course, there might be courses that will nudge towards some of those tools. If you visit some of those: Cool! But if there are no such courses (I didn't encounter many of them myself), try to explore these things nonetheless. Ev

CitesView on HN
@notpeter2015 · in “Command-line tools can be faster than your Hadoop cluster”

This article echoes a talk Bryan Cantrill gave two years ago: ▶ this video It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m screencast: link

Strongly recommendsView on HN
@erhardm2014 · in “Why pro-systemd and anti-systemd people will never get along”

They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: ▶ this video

RecommendsView on HN
Bryan Cantrill from Joyent on Manta: internet-facing object storage facility that features compute1:31:05

Bryan Cantrill from Joyent on Manta: internet-facing object storage facility that features compute

AI Council
25K views • 12 years ago
Unix and Google7:16

Unix and Google

FOAM
9K views • 10 years ago
Bryan Cantrill on ARC: A Self-Tuning, Low Overhead Replacement Cache [PWL SF] 10/20171:47:35

Bryan Cantrill on ARC: A Self-Tuning, Low Overhead Replacement Cache [PWL SF] 10/2017

PapersWeLove
13K views • 9 years ago
Michael Stonebraker | Big Data is (at least) Four Different Problems55:53

Michael Stonebraker | Big Data is (at least) Four Different Problems

Michigan Engineering
30K views • 10 years ago
Rich Hickey: Deconstructing the Database1:06:24

Rich Hickey: Deconstructing the Database

InfoQ
106K views • 14 years ago
Introduction to NoSQL • Martin Fowler • GOTO 201254:52

Introduction to NoSQL • Martin Fowler • GOTO 2012

GOTO Conferences
1M views • 13 years ago
Thesis: Partial State in Dataflow-Based Materialized Views45:50

Thesis: Partial State in Dataflow-Based Materialized Views

Jon Gjengset
17K views • 6 years ago
Bloat: How and Why UNIX Grew Up (and Out) - Rusty Russell,Matt Evans42:39

Bloat: How and Why UNIX Grew Up (and Out) - Rusty Russell,Matt Evans

Linux.conf.au 2012 -- Ballarat, Australia
21K views • 14 years ago
BetrFS: A Right-Optimized Write-Optimized File System1:00:31

BetrFS: A Right-Optimized Write-Optimized File System

Microsoft Research
3K views • 10 years ago
Lucas Cavalcanti & Edward Wible - Exploring four hidden superpowers of Datomic40:42

Lucas Cavalcanti & Edward Wible - Exploring four hidden superpowers of Datomic

ClojureTV
29K views • 11 years ago
"Datafun: a functional query language" by Michael Arntzenius42:31

"Datafun: a functional query language" by Michael Arntzenius

Strange Loop Conference
9K views • 9 years ago
Stanford Seminar - The Soul of a New Machine: Rethinking the Computer1:26:02

Stanford Seminar - The Soul of a New Machine: Rethinking the Computer

Stanford Online
76K views • 6 years ago
"Turning the database inside out with Apache Samza" by Martin Kleppmann47:43

"Turning the database inside out with Apache Samza" by Martin Kleppmann

Strange Loop Conference
205K views • 12 years ago
The Unix Chainsaw - Gary Bernhardt31:12

The Unix Chainsaw - Gary Bernhardt

timmonta
25K views • 11 years ago
The Value of Values with Rich Hickey31:44

The Value of Values with Rich Hickey

InfoQ
145K views • 14 years ago
Rust at speed — building a fast concurrent database52:54

Rust at speed — building a fast concurrent database

Jon Gjengset
225K views • 7 years ago
Scale By The Bay 2018: Bryan Cantrill, Rust and Other Interesting Things37:07

Scale By The Bay 2018: Bryan Cantrill, Rust and Other Interesting Things

FunctionalTV
45K views • 7 years ago
"I See What You Mean" by Peter Alvaro52:29

"I See What You Mean" by Peter Alvaro

Strange Loop Conference
59K views • 11 years ago
Benno Rice: The Tragedy of systemd -- BSDCan 201832:30

Benno Rice: The Tragedy of systemd -- BSDCan 2018

BSDCan
25K views • 8 years ago
How We've Scaled Dropbox1:08:17

How We've Scaled Dropbox

Stanford
372K views • 14 years ago