Differential Synchronization
Google TechTalks3 HN recommenders · recommended in 3 months, 2012-2020
52K views · 17 years ago · talk · evergreen
Neil Fraser of Google presents Differential Synchronization, the algorithm behind MobWrite, for keeping multiple copies of a document in sync in real time. The talk argues the approach is robust, convergent and efficient.
HN points to it as a simple diff-and-patch alternative to operational transforms and CRDTs for collaborative editing.
43:39CRDTs and the Quest for Distributed Consistency
InfoQ
68K views • 8 years ago
1:10:10CRDTs: The Hard Parts
Martin Kleppmann
90K views • 6 years ago
51:36Conflict Resolution for Eventual Consistency • Martin Kleppmann • GOTO 2016
GOTO Conferences
46K views • 9 years ago
34:52Liberating Distributed Consensus - Heidi Howard
Reactive Summit
7K views • 7 years ago
51:27Google I/O 2012 - Go Concurrency Patterns
Google for Developers
894K views • 14 years ago
1:06:08Scaling the Linear Sync Engine
Linear
37K views • 3 years ago
58:17Raft lecture (Raft user study)
Diego Ongaro
87K views • 13 years ago
1:06:45Let's #TalkConcurrency Panel Discussion with Sir Tony Hoare, Joe Armstrong, and Carl Hewitt
Erlang Solutions
13K views • 7 years ago
1:16:07React Helsinki February 2020
Fraktio
42K views • 6 years ago
48:45Matrix 2.0 — How we're making Matrix go voom!
Matrixdotorg
16K views • 3 years ago
35:15Solid Ground // Saša Jurić
EMPEX Conference
14K views • 9 years ago
1:10:23"Systems that run forever self-heal and scale" by Joe Armstrong (2013)
Strange Loop Conference
85K views • 5 years ago
29:46The past, present, and future of local-first - Martin Kleppmann (Local-First Conf)
Local-First Conf
17K views • 2 years ago
1:06:34Paxos lecture (Raft user study)
Diego Ongaro
114K views • 13 years ago
1:19:35Lecture 1: Introduction
MIT 6.824: Distributed Systems
921K views • 6 years ago
58:36Advanced Topics in Programming Languages: A Lock-Free Hash Table
GoogleTalksArchive
29K views • 14 years ago
30:06"Tackling Concurrency Bugs with TLA+" by Hillel Wayne
Strange Loop Conference
24K views • 9 years ago
45:11Kilim: Fast, lightweight, cheap message passing in Java.
Google TechTalks
33K views • 18 years ago
34:00User-level threads....... with threads. - Paul Turner - Google
Linux Plumbers Conference
41K views • 13 years ago
41:06"Unison: a new distributed programming language" by Paul Chiusano
Strange Loop Conference
45K views • 7 years ago
4 comments from Hacker News
I'm working on a project with some offline data synchronization needs, but haven't started implementation yet. I've been following CRDTs with interest. I also saw many of the same downsides mentioned in the OP, e.g. bloat (which apparently are being addressed remarkably well). Beyond OT, another approach I've run across that looks very promising is Differential Synchronization[1] by Neil Fraser. While it also relies on a centralized server, it allows for servers to be chained in such a way that seems to address many of the downsides of OT. I wonder why I rarely ever see Differential Synchronization mentioned here on HN? Is it due to lack of awareness or because of use-case fit issues or some fatal flaw I haven't seen? Or something else? [1] ▶ this video
Differential synchronization does not require a fixed polling interval. You can have real time updates as each party types, in fact the algorithm prefers this setup as it is easier to process a lot of small changes rapidly. Neil goes into further detail in this google talk about why this is the case: ▶ this video . The trick is sync after someone changes the document while including a short debounce so that you aren't overloading the system with superfluous edits.
I've not tried it, but you might also be interested in an approach called differential synchronization[1]. Basically, it's just a specific way of passing diffs around to guarantee convergence, though (as far as I can tell) it doesn't guarantee much else (as in do you lose data if a patch can't be applied?). The really nice bits, though, are that it is a simple algorithm and that it can be applied to anything that you have diff/patch algorithms for. [1] ▶ this video
One real-world use case is differential synchronization. Here's a Google Tech Talk on the subject: ▶ this video This is for real-time collaborative editing or replication of structured data (unlike Operational Transformation, which is optimized for text). Sending patches back and forth is at the core of the protocol.