reiseburo
/
verspaetung
mirror of https://github.com/reiseburo/verspaetung
1
0
Fork 0
Code Issues Releases Wiki Activity
46 Commits 2 Branches 9 Tags 355 KiB
Groovy 92.1%
Gradle 6.6%
AsciiDoc 1.3%
Go to file
R. Tyler Croy 71347594ca Major refactor to support dropwizard-metrics as the means of outputting metrics 
This commit changes the structure of Verspaetung pretty dramatically to allow
for the registering of Gauges for the various offsets

With this change the KafkaPoller is pushing the latest offsets into a map and the
ZK consumer tree watchers are pushing consumer offsets into a separate map.

References #17
 		2015-02-06 08:34:19 -08:00
gradle/wrapper
…
src Major refactor to support dropwizard-metrics as the means of outputting metrics 2015-02-06 08:34:19 -08:00
.gitignore
…
.travis.yml Add support for TravisCI 2015-01-19 11:46:57 -08:00
LICENSE.md Add the MIT license 2015-01-09 15:57:05 -08:00
README.md Add some notes about using Verspaetung 2015-01-28 03:33:52 -08:00
build.gradle Major refactor to support dropwizard-metrics as the means of outputting metrics 2015-02-06 08:34:19 -08:00
gradle.properties Add some simple command line overrides for ZK and statsd hosts 2015-01-26 06:15:35 -08:00
gradlew
…
gradlew.bat
…

README.md

Verspätung

BuildStatus

Download

Verspätung is a small utility which aims to help identify delay of Kafka consumers.

Verspätung monitors the topics and their latest offsets by talking to Kafka, it will also keep track of how far along consumers are by monitoring the offsets that they have committed to Zookeeper. Using both of these pieces of information, Verspätung computs the delta for each of the consumer groups and reports it to statsd.

Using

% java -jar verspaetung-0.1.4-all.jar --help
usage: verspaetung
 -H,--statsd-host <STATSD>   Hostname for a statsd instance (defaults to
                             localhost)
 -n,--dry-run                Disable reporting to a statsd host
 -p,--statsd-port <PORT>     Port for the statsd instance (defaults to
                             8125)
 -s,--storm                  Watch Storm KafkaSpout offsets (under
                             /kafka_spout)
 -z,--zookeeper <HOSTS>      Comma separated list of Zookeeper hosts (e.g.
                             localhost:2181)

Running Verspätung is rather easy, by default the daemon will monitor the standard Kafka high-level consumer offset path of /consumers and start reporting deltas automatically.

Hacking

  • Running tests: ./gradlew check
  • Running the app locally: ./gradlew run -PzookeeperHosts=localhost:2181
  • Building the app for distribution: ./gradlew shadowJar