R Tyler Croy
cfefbe14b2
|
||
|---|---|---|
| src | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE.txt | ||
| README.adoc | ||
| docker-compose.yml | ||
| example.log | ||
| hotdog.yml | ||
| tsung.xml | ||
README.adoc
Hotdog!
Hotdog is a syslog-to-Kafka forwarder which aims to get log entries into Apache Kafka as quickly as possible.
Hotdog 0.1.0
R Tyler Croy <rtyler@brokenco.de
Forward syslog over to Kafka with ease
USAGE:
hotdog [OPTIONS]
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
-c, --config <FILE> Sets a custom config file [default: hotdog.yml]
-t, --test <TEST_FILE> Test a log file against the configured rulesConfiguration
Hotdog is configured by the hotdog.yml file, which has a very fluid syntax at
the moment. The two main sections are the global and rules blocks.
Rules defined in the configuration can be tested against an example log file in order to verify that the right rules are matching the expected log inputs, for example:
❯ RUST_LOG=info ./target/debug/hotdog -t example.log
Line 1 matches on:
- ^hello\s+(?P<name>\w+)?
- .*
Line 2 matches on:
- .*
Line 3 matches on:
- .*
Line 4 matches on:
- ^\{(.*)?\}$
- .*Global
The global configuration block helps configure hotdog itself, namely the
port it listens on, etc.
Rules
Hotdog’s rules define how it should handle and route the syslog messages it receives.
Actions
Actions determine what hotdog should do with the given log line when it
receives it.
Some actions, such as Replace, can perform variable substitutions on
log line. The variables available are a combination of the built-in variables
listed below, and whatever named groups exist in the regex field of the
Rules.
|
Caution
|
Named groups will override any built-in variables at the time of substitution, so be careful you are not naming your groups anything which might overlap with the built-in variable names |
| Name | Description |
|---|---|
|
The original log line message sent along from the syslog sender. |
Forward
The forward action will imply the Stop action when used.
Merge
| Key | Value |
|---|---|
|
A YAMl map which will be merged with the JSON object deserialized from the matched log line. |
The merge action will only work when the log line is a JSON object. JSON
arrays, or other arbitrary strings will not merge properly, and cause all
subsequent actions for the given rule to be aborted.
Replace
| Key | Value |
|---|---|
|
A Handlebars-style template which can be used to output a modified message. |
The template may utilize the matched and built-in variables in
order to generate a modified message. The output is only available to
subsequent actions defined after the replace action. Subsequent rules in
the chain will not utilize this generated message.
Stop
Development
Hotdog is tested against the latest Rust stable. A simple cargo build should
compile a working hotdog binary for your platform.
On Linux systems it is easy to test with:
logger --server 127.0.0.1 -T -P 1514 "hello world"