Nice article on The Log

"Each working data pipeline is designed like a log; each broken data pipeline is broken in its own way."—Count Leo Tolstoy (translation by Jay Kreps)

Here's an excellent manifesto about "the data log".  You might be familiar with this concept in the guise of database journals, or with event sequences in distributed systems.

Jay starts from there and writes (pretty comprehensively!) about how this idea is used in several modern systems, and talks about his experience at linkedin using this approach.  Lots of interesting references at the end.  I was happy to see John Ousterhout mentioned there!

The Log: What every software engineer should know about real-time data's unifying abstraction

blogodex = {"idx" : ["logging", "Jay Kreps", "Kafka", "fundamentals", "scalability"]};