Bringing the Unix Philosophy to Big Data by Bryan Cantrill: FutureStack13
New Relic2 HN recommenders · recommended in 2 months, 2014-2022
18K views · 13 years ago · talk · dated
Bryan Cantrill presents Manta, an object store with in-situ compute that lets tools like grep, awk, and sed run map-reduce style over huge data. He frames it as bringing the Unix philosophy to big data.
HN recommends it as an elegant argument for simple Unix tools replacing heavyweight Hadoop-style pipelines.
1:31:05Bryan Cantrill from Joyent on Manta: internet-facing object storage facility that features compute
AI Council
25K views • 12 years ago
7:16Unix and Google
FOAM
9K views • 10 years ago
1:47:35Bryan Cantrill on ARC: A Self-Tuning, Low Overhead Replacement Cache [PWL SF] 10/2017
PapersWeLove
13K views • 9 years ago
55:53Michael Stonebraker | Big Data is (at least) Four Different Problems
Michigan Engineering
30K views • 10 years ago
1:06:24Rich Hickey: Deconstructing the Database
InfoQ
106K views • 14 years ago
54:52Introduction to NoSQL • Martin Fowler • GOTO 2012
GOTO Conferences
1M views • 13 years ago
45:50Thesis: Partial State in Dataflow-Based Materialized Views
Jon Gjengset
17K views • 6 years ago
42:39Bloat: How and Why UNIX Grew Up (and Out) - Rusty Russell,Matt Evans
Linux.conf.au 2012 -- Ballarat, Australia
21K views • 14 years ago
1:00:31BetrFS: A Right-Optimized Write-Optimized File System
Microsoft Research
3K views • 10 years ago
40:42Lucas Cavalcanti & Edward Wible - Exploring four hidden superpowers of Datomic
ClojureTV
29K views • 11 years ago
42:31"Datafun: a functional query language" by Michael Arntzenius
Strange Loop Conference
9K views • 9 years ago
1:26:02Stanford Seminar - The Soul of a New Machine: Rethinking the Computer
Stanford Online
76K views • 6 years ago
47:43"Turning the database inside out with Apache Samza" by Martin Kleppmann
Strange Loop Conference
205K views • 12 years ago
31:12The Unix Chainsaw - Gary Bernhardt
timmonta
25K views • 11 years ago
31:44The Value of Values with Rich Hickey
InfoQ
145K views • 14 years ago
52:54Rust at speed — building a fast concurrent database
Jon Gjengset
225K views • 7 years ago
37:07Scale By The Bay 2018: Bryan Cantrill, Rust and Other Interesting Things
FunctionalTV
45K views • 7 years ago
52:29"I See What You Mean" by Peter Alvaro
Strange Loop Conference
59K views • 11 years ago
32:30Benno Rice: The Tragedy of systemd -- BSDCan 2018
BSDCan
25K views • 8 years ago
1:08:17How We've Scaled Dropbox
Stanford
372K views • 14 years ago
3 comments from Hacker News
1.) Be curious. Stay curious. Experiment, play with things. Your time during university is likely the best time for this and it will likely get harder later on when you're on the job and there's much more emphasis on finished work than on learning experiences. Also, I'm quite sure that it will be beneficial for you if you familiarize yourself with different ways of thinking (obviously programming languages, environments and paradigms, but that will likely apply to anything non-technical, as well). I gained this perspective from dealing with people who didn't really do that, but also from knowing that I personally also did this way too late (and from the benefits it brought me nonetheless). If you're only familiar with Windows, try Linux. Try BSD. Try OS X, if you want to. Try working in the terminal. Familiarize yourself with different editors (especially Vim and Emacs, maybe also newer ones like Kakoune). Learn bash, have a look at AWK and sed. Maybe learn Perl. Try out functional programming, logic programming (like Prolog) and so on and so forth. If you're feeling adventurous, play with niche systems like Haiku[1] or TempleOS[2], just to gain new insights on how things could work. Maybe try the ACME editor[3]. [1]: link [2]: link [3]: link Of course, there might be courses that will nudge towards some of those tools. If you visit some of those: Cool! But if there are no such courses (I didn't encounter many of them myself), try to explore these things nonetheless. Ev
This article echoes a talk Bryan Cantrill gave two years ago: ▶ this video It's about how Joyent took the concept of a UNIX pipeline as a true powertool and built a distributed version atop an object filesystem with some little map/reduce syntactic sugar to replace Hadoop jobs with pipelines. The Bryan Cantrill talk is definitely worth your time, but you can get an understanding of Manta with their 3m screencast: link
They're good points, but from the philosophical point The Unix Way , systemd leaves some things to desire. Is there a modern Unix way? I don't see a lot of exploring in this area/side, but I see a lot of this is the way it's gonna be from now on on the other side. I'm not on either side, but I appreciate the way some complexity is dealt with simple little things. So elegantly solving problems: ▶ this video