5ae49a3332
|
||
|---|---|---|
| gradle/wrapper | ||
| src | ||
| .gitignore | ||
| HACKING.adoc | ||
| LICENSE.md | ||
| README.adoc | ||
| build.gradle | ||
| gradle.properties | ||
| gradlew | ||
| gradlew.bat | ||
README.adoc
Beetle
Beetle is a somewhat higher level Java API on top of the client libraries distributed distributed with Apache Kafka. The goal of this library is not to replace the use of those libraries, but to wrap the library in a more easy to use package.
System Requirements
-
JDK7 or later
Hacking
This project uses Gradle so building and testing should be as easy as executing:
% ./gradlew
Design/Notes/Thoughts
|
Note
|
Right now this section is very much just a brain-dump/work-in-progress |
What is fundamentally missing from the upstream Kafka clients is an evented view on the world. Despite Zookeeper and Kafka’s models effectively being event-driven, implementing a lower-level SimpleConsumer utilizes busy-loops and rather disjointed logic for reconnects and error handling. The higher-level consumer API is also awkward to use as far as receiving messages (using an iterator) and handling parallel operations (stuffing a thread pool somewhere for receiving).
A high-level Kafka consumer API maps rather nicely to the RxJava usage model of Observers and Subscribers, e.g.g
/* Using Java 8 Lambda syntax for conciseness' sake */
LocateBrokers.zookeeper("localhost:2181")
.map(broker -> TopicSubscription("some-topic"))
/* assuming a custom consume() operator exists in Beetle */
.consume(message -> doSomethingWithMessage(message))
.map(message -> message.commitOffset());Similar Projects
-
kafka-rx: Scala-based client which provides a push alternative to kafka’s pull-based stream