dipstick/README.md

151 lines
5.4 KiB
Markdown
Raw Normal View History

A quick, modular metrics toolkit for Rust applications; similar to popular logging frameworks,
but with counters, markers, gauges and timers.
2017-08-19 18:09:17 +00:00
2017-12-13 19:58:42 +00:00
Dipstick's main attraction is the ability to send metrics to multiple customized outputs.
For example, captured metrics can be written immediately to the log _and_
2017-12-13 19:58:42 +00:00
sent over the network after a period of aggregation.
Dipstick builds on stable Rust with minimal dependencies
and is published as a [crate.](https://crates.io/crates/dipstick)
2017-06-29 21:52:37 +00:00
## Features
2017-09-29 20:45:09 +00:00
- Send metrics to stdout, log, statsd or graphite (one or many)
- Synchronous, asynchronous or mixed operation
- Optional fast random statistical sampling
- Immediate propagation or local aggregation of metrics (count, sum, average, min/max)
- Periodic or programmatic publication of aggregated metrics
- Customizable output statistics and formatting
- Global or scoped (e.g. per request) metrics
- Per-application and per-output metric namespaces
- Predefined or ad-hoc metrics
2017-09-29 20:45:09 +00:00
## Cookbook
Dipstick is easy to add to your code:
```rust
2017-08-21 21:36:35 +00:00
use dipstick::*;
let app_metrics = metrics(to_graphite("host.com:2003"));
2017-09-28 22:17:16 +00:00
app_metrics.counter("my_counter").count(3);
```
2017-10-10 14:11:42 +00:00
Metrics can be sent to multiple outputs at the same time:
2017-09-28 22:17:16 +00:00
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
2017-09-28 22:17:16 +00:00
```
2017-09-29 20:45:09 +00:00
Since instruments are decoupled from the backend, outputs can be swapped easily.
2017-10-10 14:11:42 +00:00
Metrics can be aggregated and scheduled to be published periodically in the background:
2017-09-28 22:17:16 +00:00
```rust
use std::time::Duration;
let (to_aggregate, from_aggregate) = aggregate();
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
let app_metrics = metrics(to_aggregate);
```
2017-09-29 20:45:09 +00:00
Aggregation is performed locklessly and is very fast.
2017-09-29 18:41:22 +00:00
Count, sum, min, max and average are tracked where they make sense.
2017-10-10 14:11:42 +00:00
Published statistics can be selected with presets such as `all_stats` (see previous example),
`summary`, `average`.
2017-09-29 18:41:22 +00:00
2017-10-10 14:11:42 +00:00
For more control over published statistics, a custom filter can be provided:
2017-09-29 18:41:22 +00:00
```rust
let (_to_aggregate, from_aggregate) = aggregate();
2017-09-29 20:45:09 +00:00
publish(from_aggregate, to_log("my_custom_stats:"),
2017-10-10 14:11:42 +00:00
|metric_kind, metric_name, metric_score|
match metric_score {
HitCount(hit_count) => Some((Counter, vec![metric_name, ".per_thousand"], hit_count / 1000)),
_ => None
});
2017-09-29 20:45:09 +00:00
```
2017-09-05 04:24:43 +00:00
2017-10-10 14:11:42 +00:00
Metrics can be statistically sampled:
2017-09-28 22:17:16 +00:00
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
```
2017-09-29 20:45:09 +00:00
A fast random algorithm is used to pick samples.
Outputs can use sample rate to expand or format published data.
2017-10-10 14:11:42 +00:00
Metrics can be recorded asynchronously:
```rust
2017-09-29 18:41:22 +00:00
let app_metrics = metrics(async(48, to_stdout()));
2017-09-28 22:17:16 +00:00
```
2017-10-10 14:11:42 +00:00
The async queue uses a Rust channel and a standalone thread.
The current behavior is to block when full.
2017-09-28 22:17:16 +00:00
2017-10-10 14:11:42 +00:00
Metric definitions can be cached to make using _ad-hoc metrics_ faster:
2017-09-28 22:17:16 +00:00
```rust
let app_metrics = metrics(cache(512, to_log()));
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
```
2017-09-29 20:45:09 +00:00
2017-10-10 14:11:42 +00:00
The preferred way is to _predefine metrics_,
possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block:
2017-09-29 20:45:09 +00:00
```rust
#[macro_use] external crate lazy_static;
lazy_static! {
pub static ref METRICS: GlobalMetrics<String> = metrics(to_stdout());
2017-09-29 20:45:09 +00:00
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
}
COUNTER_A.count(11);
```
2017-10-10 14:11:42 +00:00
Timers can be used multiple ways:
2017-09-28 22:17:16 +00:00
```rust
let timer = app_metrics.timer("my_timer");
time!(timer, {/* slow code here */} );
timer.time(|| {/* slow code here */} );
let start = timer.start();
/* slow code here */
timer.stop(start);
2017-09-28 22:17:16 +00:00
timer.interval_us(123_456);
2017-06-29 21:52:37 +00:00
```
2017-10-10 14:11:42 +00:00
Related metrics can share a namespace:
2017-09-28 22:17:16 +00:00
```rust
let db_metrics = app_metrics.with_prefix("database.");
let db_timer = db_metrics.timer("db_timer");
2017-09-29 20:45:09 +00:00
let db_counter = db_metrics.counter("db_counter");
2017-09-28 22:17:16 +00:00
```
2017-09-29 20:45:09 +00:00
2017-09-28 22:17:16 +00:00
## Design
Dipstick's design goals are to:
- support as many metrics backends as possible while favoring none
- support all types of applications, from embedded to servers
- promote metrics conventions that facilitate app monitoring and maintenance
- stay out of the way in the code and at runtime (ergonomic, fast, resilient)
2017-09-05 04:24:43 +00:00
## Performance
2017-09-28 22:17:16 +00:00
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
2017-09-29 20:45:09 +00:00
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
Defining a metric `cache()` reduces that cost for recurring metrics.
2017-09-05 04:24:43 +00:00
Run benchmarks with `cargo +nightly bench --features bench`.
2017-09-05 04:24:43 +00:00
2017-09-29 20:45:09 +00:00
## TODO
Although already usable, Dipstick is still under heavy development and makes no guarantees
of any kind at this point. See the following list for any potential caveats :
2017-08-21 21:34:49 +00:00
- META turn TODOs into GitHub issues
- generic publisher / sources
2017-08-21 21:20:31 +00:00
- feature flags
2017-09-28 22:17:16 +00:00
- time measurement units in metric kind (us, ms, etc.) for naming & scaling
- heartbeat metric on publish
2017-08-21 21:34:49 +00:00
- logger templates
2017-10-10 14:11:42 +00:00
- configurable aggregation (?)
2017-08-18 17:55:04 +00:00
- non-aggregating buffers
2017-09-25 10:43:04 +00:00
- framework glue (rocket, iron, gotham, indicatif, etc.)
- more tests & benchmarks
2017-08-21 21:34:49 +00:00
- complete doc / inline samples
2017-09-25 10:43:04 +00:00
- more example apps
2017-09-29 20:45:09 +00:00
- A cool logo
2017-09-25 10:43:04 +00:00
- method annotation processors `#[timer("name")]`
2017-09-29 20:45:09 +00:00
- fastsinks (M / &M) vs. safesinks (Arc<M>)
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
License: MIT/Apache-2.0
2017-10-10 14:11:42 +00:00
_this file was generated using [cargo readme](https://github.com/livioribeiro/cargo-readme)_