dipstick/README.md

139 lines
4.9 KiB
Markdown
Raw Normal View History

2017-09-29 20:45:09 +00:00
# dipstick
2017-08-19 18:09:17 +00:00
2017-09-29 20:45:09 +00:00
A fast and modular metrics toolkit for all Rust applications.
2017-09-29 18:41:22 +00:00
Similar to popular logging frameworks, but with counters, markers, gauges and timers.
2017-06-29 21:52:37 +00:00
2017-09-29 18:41:22 +00:00
Out of the box, Dipstick _can_ aggregate, sample, cache and queue metrics (async).
If aggregated, statistics can be published on demand or on schedule.
2017-09-29 20:45:09 +00:00
Dipstick does not bind application code to a single metrics output implementation.
2017-12-06 22:05:47 +00:00
Outputs `to_log`, `to_stdout`, `to_graphite` and `to_statsd` are currently provided.
2017-09-29 20:45:09 +00:00
2017-09-29 18:41:22 +00:00
Dipstick builds on stable Rust with minimal dependencies.
```rust
2017-08-21 21:36:35 +00:00
use dipstick::*;
2017-09-29 18:41:22 +00:00
let app_metrics = metrics(to_log("metrics:"));
2017-09-28 22:17:16 +00:00
app_metrics.counter("my_counter").count(3);
```
2017-10-10 14:11:42 +00:00
Metrics can be sent to multiple outputs at the same time:
2017-09-28 22:17:16 +00:00
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
2017-09-28 22:17:16 +00:00
```
2017-09-29 20:45:09 +00:00
Since instruments are decoupled from the backend, outputs can be swapped easily.
2017-10-10 14:11:42 +00:00
Metrics can be aggregated and scheduled to be published periodically in the background:
2017-09-28 22:17:16 +00:00
```rust
use std::time::Duration;
let (to_aggregate, from_aggregate) = aggregate();
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
let app_metrics = metrics(to_aggregate);
```
2017-09-29 20:45:09 +00:00
Aggregation is performed locklessly and is very fast.
2017-09-29 18:41:22 +00:00
Count, sum, min, max and average are tracked where they make sense.
2017-10-10 14:11:42 +00:00
Published statistics can be selected with presets such as `all_stats` (see previous example),
`summary`, `average`.
2017-09-29 18:41:22 +00:00
2017-10-10 14:11:42 +00:00
For more control over published statistics, a custom filter can be provided:
2017-09-29 18:41:22 +00:00
```rust
let (_to_aggregate, from_aggregate) = aggregate();
2017-09-29 20:45:09 +00:00
publish(from_aggregate, to_log("my_custom_stats:"),
2017-10-10 14:11:42 +00:00
|metric_kind, metric_name, metric_score|
match metric_score {
HitCount(hit_count) => Some((Counter, vec![metric_name, ".per_thousand"], hit_count / 1000)),
_ => None
});
2017-09-29 20:45:09 +00:00
```
2017-09-05 04:24:43 +00:00
2017-10-10 14:11:42 +00:00
Metrics can be statistically sampled:
2017-09-28 22:17:16 +00:00
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
```
2017-09-29 20:45:09 +00:00
A fast random algorithm is used to pick samples.
Outputs can use sample rate to expand or format published data.
2017-10-10 14:11:42 +00:00
Metrics can be recorded asynchronously:
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics(async(48, to_stdout()));
2017-09-28 22:17:16 +00:00
```
2017-10-10 14:11:42 +00:00
The async queue uses a Rust channel and a standalone thread.
The current behavior is to block when full.
2017-09-28 22:17:16 +00:00
2017-10-10 14:11:42 +00:00
Metric definitions can be cached to make using _ad-hoc metrics_ faster:
2017-09-28 22:17:16 +00:00
```rust
let app_metrics = metrics(cache(512, to_log()));
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
```
2017-09-29 20:45:09 +00:00
2017-10-10 14:11:42 +00:00
The preferred way is to _predefine metrics_,
possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block:
2017-09-29 20:45:09 +00:00
```rust
#[macro_use] external crate lazy_static;
lazy_static! {
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
}
COUNTER_A.count(11);
```
2017-10-10 14:11:42 +00:00
Timers can be used multiple ways:
2017-09-28 22:17:16 +00:00
```rust
let timer = app_metrics.timer("my_timer");
time!(timer, {/* slow code here */} );
timer.time(|| {/* slow code here */} );
let start = timer.start();
/* slow code here */
timer.stop(start);
2017-09-28 22:17:16 +00:00
timer.interval_us(123_456);
2017-06-29 21:52:37 +00:00
```
2017-10-10 14:11:42 +00:00
Related metrics can share a namespace:
2017-09-28 22:17:16 +00:00
```rust
let db_metrics = app_metrics.with_prefix("database.");
let db_timer = db_metrics.timer("db_timer");
2017-09-29 20:45:09 +00:00
let db_counter = db_metrics.counter("db_counter");
2017-09-28 22:17:16 +00:00
```
2017-09-29 20:45:09 +00:00
2017-09-28 22:17:16 +00:00
## Design
Dipstick's design goals are to:
- support as many metrics backends as possible while favoring none
- support all types of applications, from embedded to servers
- promote metrics conventions that facilitate app monitoring and maintenance
- stay out of the way in the code and at runtime (ergonomic, fast, resilient)
2017-09-05 04:24:43 +00:00
## Performance
2017-09-28 22:17:16 +00:00
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
2017-09-29 20:45:09 +00:00
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
Defining a metric `cache()` reduces that cost for recurring metrics.
2017-09-05 04:24:43 +00:00
Run benchmarks with `cargo +nightly bench --features bench`.
2017-09-05 04:24:43 +00:00
2017-09-29 20:45:09 +00:00
## TODO
Although already usable, Dipstick is still under heavy development and makes no guarantees
of any kind at this point. See the following list for any potential caveats :
2017-08-21 21:34:49 +00:00
- META turn TODOs into GitHub issues
- generic publisher / sources
2017-08-21 21:20:31 +00:00
- feature flags
2017-09-28 22:17:16 +00:00
- time measurement units in metric kind (us, ms, etc.) for naming & scaling
- heartbeat metric on publish
2017-08-21 21:34:49 +00:00
- logger templates
2017-10-10 14:11:42 +00:00
- configurable aggregation (?)
2017-08-18 17:55:04 +00:00
- non-aggregating buffers
2017-09-25 10:43:04 +00:00
- framework glue (rocket, iron, gotham, indicatif, etc.)
- more tests & benchmarks
2017-08-21 21:34:49 +00:00
- complete doc / inline samples
2017-09-25 10:43:04 +00:00
- more example apps
2017-09-29 20:45:09 +00:00
- A cool logo
2017-09-25 10:43:04 +00:00
- method annotation processors `#[timer("name")]`
2017-09-29 20:45:09 +00:00
- fastsinks (M / &M) vs. safesinks (Arc<M>)
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
License: MIT/Apache-2.0
2017-10-10 14:11:42 +00:00
_this file was generated using [cargo readme](https://github.com/livioribeiro/cargo-readme)_