Kafka Training Deck

Covering:

• Introducing Kafka: history, Kafka at LinkedIn, Kafka adoption in the industry, why Kafka
• Kafka core concepts: topics, partitions, replicas, producers, consumers, brokers
• Operating Kafka: architecture, hardware specs, deploying, monitoring, performance and scalability tuning
• Developing Kafka apps: writing to Kafka, reading from Kafka, testing, serialization, compression, example apps (with kafka-storm-starter)
• Playing with Kafka using Wirbelsturm

http://www.michael-noll.com/blog/2014/08/18/apache-kafka-training-deck-and-tutorial/

blogodex = {"idx" : ["kafka", "tutorials"]};

Nice article on The Log

"Each working data pipeline is designed like a log; each broken data pipeline is broken in its own way."—Count Leo Tolstoy (translation by Jay Kreps)

Here's an excellent manifesto about "the data log".  You might be familiar with this concept in the guise of database journals, or with event sequences in distributed systems.

Jay starts from there and writes (pretty comprehensively!) about how this idea is used in several modern systems, and talks about his experience at linkedin using this approach.  Lots of interesting references at the end.  I was happy to see John Ousterhout mentioned there!

The Log: What every software engineer should know about real-time data's unifying abstraction

blogodex = {"idx" : ["logging", "Jay Kreps", "Kafka", "fundamentals", "scalability"]};