2014-06-02 20:34:55 +00:00
# Hermann
2014-10-14 22:31:25 +00:00
[![Gitter chat ](https://badges.gitter.im/lookout/Hermann.png )](https://gitter.im/lookout/Hermann) [![Build Status ](https://travis-ci.org/lookout/Hermann.svg?branch=master )](https://travis-ci.org/lookout/Hermann)
2014-09-02 21:01:55 +00:00
2014-08-18 18:31:09 +00:00
A Ruby gem implementing a Kafka Publisher and Consumer
2014-10-10 21:32:41 +00:00
On MRI (C-based Ruby), this library wraps the [librdkafka
library](https://github.com/edenhill/librdkafka) which is implemented in C.
On JRuby this library [declares jar
dependencies](https://github.com/mkristian/jar-dependencies/wiki/declare-jars-inside-gemspec)
inside the `.gemspec` to express dependencies on the Java-based Kafka library
provided by the Kafka project. Tools like
[jbundler ](https://github.com/mkristian/jbundler ) will handle these
declarations correctly.
2014-06-02 20:34:55 +00:00
### Usage
2014-08-18 20:48:56 +00:00
Usage is modelled on the
[kafka-rb gem ](https://github.com/acrosa/kafka-rb ) and is fairly
straightforward.
2014-06-02 20:34:55 +00:00
2014-06-16 21:49:51 +00:00
- Kafka 0.8 is supported.
2014-10-08 17:40:40 +00:00
- Ruby 1.9.3, 2.1.1 and JRuby are tested against
2014-08-18 18:31:09 +00:00
- This is an early alpha version of the gem, so expect bumps in the
road.
2014-06-02 20:34:55 +00:00
2014-10-10 21:32:41 +00:00
### Producer
#### Zookeeper discovery (JRuby-only)
2014-10-30 17:02:49 +00:00
Discover Kafka brokers through zookeeper. Looks at ```/brokers/ids``` in Zookeeper to find the list of brokers.
```ruby
require 'hermann/producer'
require 'hermann/discovery/zookeeper'
broker_ids_array = Hermann::Discovery::Zookeeper.new('localhost:2181').get_brokers
producer = Hermann::Producer.new('topic', broker_ids_array)
promise = producer.push('hello world') # send message to kafka
promise.value # forces the Concurrent::Promise to finish excuting (#value!)
promise.state # the state of the promise
2014-10-10 21:32:41 +00:00
```
#### MRI only
2014-10-30 17:02:49 +00:00
```ruby
require 'hermann/producer'
2014-10-10 21:32:41 +00:00
2014-10-30 17:02:49 +00:00
p = Hermann::Producer.new('topic', ['localhost:6667']) # arguments topic, list of brokers
f = p.push('hello world from mri')
f.state
p.tick_reactor
f.state
2014-10-10 21:32:41 +00:00
```
2014-10-10 22:26:19 +00:00
2014-10-30 17:12:05 +00:00
### Consumer
Messages can be consumed by calling the consume method and passing a block to handle the yielded messages. The consume method blocks, so take care to handle that functionality appropriately (i.e. use Concurrent::Promise, Thread, etc).
#### (JRuby-only)
```ruby
require 'hermann'
require 'hermann/consumer'
require 'hermann_jars'
zookeepers = "localhost:2181"
groupId = "group1"
topic = 'topic'
new_topic = 'other_topic'
the_consumer = Hermann::Consumer.new(topic, groupId, zookeepers)
the_consumer.consume(new_topic) do |msg| # can change topic with optional argument to .consume
puts "Recv: #{msg}"
end
```
2014-10-10 22:26:19 +00:00
#### How to convert from using jruby-kafka
* Gemfile
* remove jruby-kafka
* add ```gem "hermann"```
* ```bundle install```
* Jarfile
* removed unecessary jars from your Jarfile (i.e. kafka, log4j)
* jar dependencies are automatically included with Hermann
* ```jbundle install```
2014-10-30 17:13:35 +00:00
* Test out one of the Producer/Consumer examples above
2014-10-27 21:07:02 +00:00
### Integration Testing
* Download Kafka
* Start Zookeeper
* set port 2181
* Start Kafka
* Set properties file ```zookeeper.connect=localhost:2181```
2014-10-30 17:13:35 +00:00
* ```bundle exec jruby -S rspec spec/integration```
2014-10-27 21:07:02 +00:00