reiseburo
/
hermann
mirror of https://github.com/reiseburo/hermann
1
0
Fork 0
Code Issues Releases Wiki Activity
77 Commits 9 Branches 20 Tags 604 KiB
Commit Graph

77 Commits

Author SHA1 Message Date
Stan Campbell f0254cdefe Add Hermann signal handler into chain to allow us to send SIGINT, etc. up the chain. 2014-09-05 12:16:37 -07:00
Stan Campbell 3a6c02f023 Remove exit(..) calls and replace by raising a runtime exception to Ruby. 2014-09-05 10:07:43 -07:00
Stan Campbell ab4f4cca5d Fix Consumer behavior to properly handle passed blocks. 2014-09-05 09:50:52 -07:00
Stan Campbell 5cf9f352e1 Merge pull request #23 from rtyler/issues/11-producer-feedback 
Tie Hermann::Result to the librdkafka reactor
 2014-09-04 15:09:44 -07:00
R. Tyler Croy 9bfd7ad2e6 Tie Hermann::Result#value to the underlying reactor to bring values up to Ruby 
This ensures that we're getting async values out of librdkafka into the calling
Ruby thread. Currently errors aren't being brought up properly, but we're getting there

Example:

    [14:47:30] tyler:Hermann git:(issues/11-producer-feedback*) $ pry -I lib -r 'hermann/producer'
    [1] pry(main)> p = Hermann::Producer.new('topic', 'kafka0.REDACTED.com:6667')
    => #<Hermann::Producer:0x00000803b3b450
    @brokers="kafka0.REDACTED.com:6667",
    @children=[],
    @internal=#<Hermann::Lib::Producer:0x00000803b3b3d8>,
    @topic="topic">
    [2] pry(main)> r = p.push('hello world!')
    => #<Hermann::Result:0x00000803b8cb20
    @producer=
    #<Hermann::Producer:0x00000803b3b450
    @brokers="kafka0.REDACTED.com:6667",
    @children=[#<Hermann::Result:0x00000803b8cb20 ...>],
    @internal=#<Hermann::Lib::Producer:0x00000803b3b3d8>,
    @topic="topic">,
    @reason=nil,
    @state=:unfulfilled,
    @value=nil>
    [3] pry(main)> r.state
    => :unfulfilled
    [4] pry(main)> r.value
    ticking rdkafka reactor
    ticked
    => "hello world!"
    [5] pry(main)> r.state
    => :fulfilled
    [6] pry(main)> r.rejected?
    => false
    [7] pry(main)>
    [14:47:56] tyler:Hermann git:(issues/11-producer-feedback*) $

Fixes #11
 2014-09-04 14:51:41 -07:00
R. Tyler Croy ed36300265 Pass a pointer to a Hermann::Result into the message delivery callback 
Provided that /something/ is ticking the librdkafka reactor underneath, the
Hermann::Result will be updated with the message delivery status when that's
completed.

What's still missing is producer level error calbacks

Fixes #11
Fixes #15
Fixes #17
 2014-09-04 14:31:29 -07:00
R. Tyler Croy 3a3df235a7 Properly pass the block to Consumer#consume into the C layer 2014-09-04 14:29:54 -07:00
R. Tyler Croy dc9a057eaa Add a simple script to validate big message behavior 
References #21
 2014-09-04 14:29:29 -07:00
Stan Campbell 34193dd142 Merge pull request #22 from rtyler/issues/21-buffer-overrun 
Avoid double-ocpying and a buffer overrun when pushing a Ruby string
 2014-09-04 13:50:33 -07:00
Stan Campbell 3a48bd6f64 Merge pull request #20 from rtyler/issues/11-producer-feedback 
An initial pass at exposing the async nature of librdkafka upwards
 2014-09-04 13:50:14 -07:00
R. Tyler Croy 7085385d7e Avoid double-ocpying and a buffer overrun when pushing a Ruby string into Kafka 
This approach to shuffling the message buffer into librdkafka relies on librdkafka to safely
copy the buffer for it's own uses from the Ruby string's C pointer.

This has been tested with buffers 2x the size of the SIGSEGV mentioned in #21

Fixes #21
Fixes #13
 2014-09-04 13:03:01 -07:00
R. Tyler Croy 74dfd19399 Remove inadvertantly added changes to msg_delivered function 2014-09-04 11:55:19 -07:00
Stan Campbell e71ab6cd14 Update the requires for the sample scripts. 2014-09-04 10:52:27 -07:00
Stan Campbell 8aff9226b7 Add example scripts testing producer and consumer functionality. 2014-09-04 10:27:47 -07:00
R. Tyler Croy ccc7d20d15 An initial pass at exposign the asynchronous nature of librdkafka up into Ruby 
This should be enough to get some initial review of the API going. Hooking up a
Hermann::Result class to the Hermann::Producer in Ruby is in this commit but
actually driving the librdkafka reactor underneath is not yet hooked up.

Fixes #11
Fixes #15
Fixes #18
 2014-09-03 14:19:09 -07:00
R. Tyler Croy 3081848bde Add the gitter chat link to the readme 2014-09-02 14:01:55 -07:00
Stan Campbell 9477cd10d3 Merge pull request #4 from rtyler/bro-do-you-even 
Add some basic sanity checks for Hermann gem
 2014-09-02 13:35:31 -07:00
R. Tyler Croy 0eb18479b6 Properly raise an exception when the consumer is given an empty topic 2014-08-30 15:30:11 -07:00
R. Tyler Croy 655cc4463a Fix almost all of the compiler warnings 
Some are intentionally left out of the fixing because I believe there's a
failing test case that needs to be added
 2014-08-30 15:30:11 -07:00
R. Tyler Croy f5e0c9384d Raise an exception instead of attempting to write to an empty topic name 2014-08-30 13:21:33 -07:00
R. Tyler Croy 3effe046f4 Raise an exception instead of exiting process on bad brokers for Herman::Consumer 2014-08-30 13:21:10 -07:00
R. Tyler Croy 2f5e9ae4e3 Make a plethora of whitespace and style consistency fixes to C code 2014-08-30 13:06:19 -07:00
R. Tyler Croy f3bf216b66 Raise a RuntimeError when a bad partition is given to a consumer 
Fixes #7
 2014-08-30 12:54:53 -07:00
R. Tyler Croy d8166b7fe4 Raise a runtime error when a bad broker is specified 
Fixes #6
 2014-08-30 12:51:03 -07:00
R. Tyler Croy 3a6a54def6 C code should be formatted with tabs not spaces 2014-08-30 08:40:58 -07:00
R. Tyler Croy e8a9ebe7a4 Depend on build instead of just compile in the default task 2014-08-29 18:37:41 -07:00
R. Tyler Croy e241f086ee Restructure the Hermann gem extension build process to use rake-compiler 
This is a little bit better and cleaner IMO than the hand-crafted rake tasks.

Would you like to know more? <https://github.com/luislavena/rake-compiler>
 2014-08-29 18:37:10 -07:00
R. Tyler Croy 9a24317c7f Major rework of the gemspec 
This change:

 - removes the platform specific binding (Ruby will run extconf.rb on platforms)
 - adds myself to the authors
 - Pulls version from the gem itself
 - includes all the files in lib that are added to the git tree
 2014-08-29 13:52:36 -07:00
R. Tyler Croy 3efa2c481b Avoid packaging the gem with a spurious object file 2014-08-29 13:47:49 -07:00
R. Tyler Croy 47914ad5e5 Add a simple Rake default task to build and test 2014-08-29 11:00:35 -07:00
R. Tyler Croy 4ca84f4ca9 Comment out tests which exit the Ruby interpreter until they can be fixed 2014-08-29 08:41:57 -07:00
R. Tyler Croy 603aba612a Implement enough of Herman::Consumer#consume to reproduce an exit(1) 2014-08-29 08:41:13 -07:00
R. Tyler Croy 941adaef3b Add test case that exits the ruby interpreter (to be fixed real soon) 2014-08-28 21:18:27 -07:00
R. Tyler Croy 164d87132b Default to documentation format 2014-08-28 21:17:44 -07:00
R. Tyler Croy 2f194b4352 Nuke ruby-version 2014-08-28 21:15:43 -07:00
R. Tyler Croy ff300e4175 Start refactoring unnecessary logic out of the C-core, starting with Producer 2014-08-28 17:14:38 -07:00
R. Tyler Croy 5e34b82498 Remove unused empty test case 2014-08-28 16:21:00 -07:00
R. Tyler Croy 96ff104061 Add some basic test cases to make sure that at least the library is getting loaded 2014-08-28 16:20:46 -07:00
R. Tyler Croy fbdf9b6055 Add rspec to the project for testing basic Hermann functionality 2014-08-28 16:13:12 -07:00
Stan Campbell d51050c3a6 Merge remote-tracking branch 'upstream/wip/PART-3886' 
Conflicts:
	hermann.iml
 2014-08-19 14:18:55 -07:00
Stan Campbell ae0b964934 Turn off TRACE and DEBUG by default. 
JIRA: PART-3886
 2014-08-19 13:37:22 -07:00
Stan Campbell 1185d54ff1 Add bulletproofing around the management of config->rk and config->rkt kafka handles. 
Avoid dereferencing or freeing null handles for Consumer and Producer classes.

JIRA: PART-3886
 2014-08-19 13:32:32 -07:00
Abhi Yerra 5af39f06ba Moved license to LICENSE.md and reformatted markdown. Closes #1 2014-08-18 13:48:56 -07:00
Abhi Yerra 8b056ed861 Remove unneeded files 2014-08-18 13:46:22 -07:00
Abhi Yerra 83daef2497 Updated README with correct formatting and added License 2014-08-18 13:45:11 -07:00
Abhi Yerra 9d4e06c394 Remove unnecessary code in gemspec 2014-08-18 13:42:14 -07:00
Abhi Yerra 1336deb53a Added travis.yml 2014-08-18 13:37:46 -07:00
Stan Campbell ba99648e0d Enable the 'x' perm on the scripts. 
JIRA: REL-1051
 2014-07-30 22:00:09 -07:00
Stan Campbell e7754c7e66 Add CI scripts for artifactory. 
JIRA: REL-1051
 2014-07-30 21:58:04 -07:00
Stan Campbell a18272e901 Remove Makefile and mkmf.log as they're generated by extconf.rb and mkmf. 2014-07-25 12:52:39 -07:00