9bfd7ad2e6
This ensures that we're getting async values out of librdkafka into the calling
Ruby thread. Currently errors aren't being brought up properly, but we're getting there
Example:
[14:47:30] tyler:Hermann git:(issues/11-producer-feedback*) $ pry -I lib -r 'hermann/producer'
[1] pry(main)> p = Hermann::Producer.new('topic', 'kafka0.REDACTED.com:6667')
=> #<Hermann::Producer:0x00000803b3b450
@brokers="kafka0.REDACTED.com:6667",
@children=[],
@internal=#<Hermann::Lib::Producer:0x00000803b3b3d8>,
@topic="topic">
[2] pry(main)> r = p.push('hello world!')
=> #<Hermann::Result:0x00000803b8cb20
@producer=
#<Hermann::Producer:0x00000803b3b450
@brokers="kafka0.REDACTED.com:6667",
@children=[#<Hermann::Result:0x00000803b8cb20 ...>],
@internal=#<Hermann::Lib::Producer:0x00000803b3b3d8>,
@topic="topic">,
@reason=nil,
@state=:unfulfilled,
@value=nil>
[3] pry(main)> r.state
=> :unfulfilled
[4] pry(main)> r.value
ticking rdkafka reactor
ticked
=> "hello world!"
[5] pry(main)> r.state
=> :fulfilled
[6] pry(main)> r.rejected?
=> false
[7] pry(main)>
[14:47:56] tyler:Hermann git:(issues/11-producer-feedback*) $
Fixes #11
|
||
|---|---|---|
| ext/hermann | ||
| lib | ||
| scripts | ||
| spec | ||
| .gitignore | ||
| .travis.yml | ||
| Gemfile | ||
| Gemfile.lock | ||
| LICENSE.md | ||
| README.md | ||
| Rakefile | ||
| hermann.gemspec | ||
README.md
Hermann
A Ruby gem implementing a Kafka Publisher and Consumer
This library wraps the librdkafka library which is implemented in C. This library must be installed before we can use the Hermann gem.
Usage
Usage is modelled on the kafka-rb gem and is fairly straightforward.
- Kafka 0.8 is supported.
- Ruby 1.8.7 and Ruby 2.1.1 have been tested, but JRuby and versions
= 1.8 should work as long as the librdkafka library is installed.
- This is an early alpha version of the gem, so expect bumps in the road.
Note: The current implementation needs work to ensure thread safety.
Both Consumers and Producers now hold their own references to their Kafka state, and should be able to coexist nicely.
The brokers list and partition specification for Consumers are now working, as well. Producers will detect the number of partitions in a topic and randomly spread their messages over the set of partitions.
Consumers currently do not remember their "last message", nor do they yet coordinate partition (re)assigment within consumer groups.
Consumer
require 'hermann'
# Initialize requires topic, brokers list, and partition number
c = Hermann::Consumer.new( "device_actions", "localhost:9092", 0 )
c.consume() do
|msg| puts("Received: #{msg}")
end
Producer
require 'hermann'
p = Hermann::Producer.new( "device_actions", "localhost:9092" )
messages = [ "Locate", "Scream", "Wipe", "Degauss" ]
messages.each do
|msg| p.push(msg)
end
Questions?
Stan Campbell can be reached at stan.campbell3 at( @ ) gmail.com