reiseburo
/
verspaetung
mirror of https://github.com/reiseburo/verspaetung
1
0
Fork 0
Code Issues Releases Wiki Activity
71 Commits 2 Branches 9 Tags 355 KiB
Commit Graph

15 Commits

Author SHA1 Message Date
R. Tyler Croy e18ac81468
Introduce codenarc and make the project adhere to some coding standards 2015-09-01 15:50:31 -07:00
R. Tyler Croy b6bbcf92a4
Don't reconnect if we have invalid data in our brokerConsumerMap, just warn about it 
Fixes #30
 2015-08-14 15:11:16 -07:00
R. Tyler Croy be33333bfa
Bail out and try to force a reconnect if our broker's consumer is nulled 
Still need to explore how we're getting into this state, but this is a start to
handling that scenario more gracefully

References #130
 2015-08-14 10:50:45 -07:00
R. Tyler Croy 5df6083e6d
Log a single error line for KafkaException exceptions when polling 
Fixes #31
 2015-08-14 10:13:17 -07:00
R. Tyler Croy f053d9b68c Catch and log exceptions coming from the dumpMetadata() call 
Fixes #23
 2015-03-22 11:08:40 -07:00
R. Tyler Croy 71347594ca Major refactor to support dropwizard-metrics as the means of outputting metrics 
This commit changes the structure of Verspaetung pretty dramatically to allow
for the registering of Gauges for the various offsets

With this change the KafkaPoller is pushing the latest offsets into a map and the
ZK consumer tree watchers are pushing consumer offsets into a separate map.

References #17
 2015-02-06 08:34:19 -08:00
R. Tyler Croy 7b13050ee2 Restructure KafkaPoller to make catching and logging exceptions easier 
I've not cleaned this up enough to make things easier to test, partially
because mocking the Kafka client APIs is a giant pain in the ass

Fixes #16
 2015-01-30 02:45:16 -08:00
R. Tyler Croy a444f49b46 Invoke the onDelta callbacks for zero deltas too 
It turns out that datadog doesn't indicate that the delta is shrinking unless
you report a zero value.
 2015-01-26 09:40:16 -08:00
R. Tyler Croy c15c25534c Remove unnecessary synchronized statement 2015-01-26 04:17:28 -08:00
R. Tyler Croy 49c2d7fc1d Add logback as a dependency and gut the printlns from the codebase 2015-01-26 04:11:27 -08:00
R. Tyler Croy ba2fa562ac Stupidly and blindly fire a bunch of crap into statsd on localhost 
This is not the final product by a long shot, just needed to figure out how
things would look inside datadog
 2015-01-20 13:23:01 -08:00
R. Tyler Croy 5fe4d8efaa Hack together the actual delta reporting, UNCLEAN CODE ALERT 
Still very much in the experimental phase, need to refactor KafkaPoller a lot
at this point and decide which code should live where and give it plenty of
tests
 2015-01-19 16:22:22 -08:00
R. Tyler Croy 193b147064 Exploratory testing, a veritable boatload of garbage and hacks 
This will at least create some stupid consumers and try to fetch the latest
offsets for a bunch of stupid topics.

The Kafka/Scala internals are so immensely frustrating
 2015-01-19 15:55:47 -08:00
R. Tyler Croy 8b792d72fd Properly use the passed in AbstractMap for the StandardTreeWatcher 2015-01-19 13:08:01 -08:00
R. Tyler Croy 7e593f1235 Introducing the BrokerTreeWatcher to track changes to the broker list in Zookeeper 
This commit includes a lot of work in progress kind of code. STill
experimenting with how to bind the events from the Zookeeper event-driven
system, in with the KafkaPoller busy-wait-loop system.
 2015-01-19 11:45:22 -08:00