mirror of https://github.com/fralalonde/dipstick
Adopt cargo readme & cargo make
This commit is contained in:
parent
5608f21e93
commit
3897ec6d55
64
README.md
64
README.md
|
@ -1,15 +1,15 @@
|
|||
# Dipstick
|
||||
# dipstick
|
||||
|
||||
A fast and modular metrics toolkit for all Rust applications.
|
||||
A fast and modular metrics toolkit for all Rust applications.
|
||||
Similar to popular logging frameworks, but with counters, markers, gauges and timers.
|
||||
|
||||
Out of the box, Dipstick _can_ aggregate, sample, cache and queue metrics (async).
|
||||
If aggregated, statistics can be published on demand or on schedule.
|
||||
|
||||
Dipstick does not bind application code to a single metrics output implementation.
|
||||
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
|
||||
|
||||
Dipstick does not bind application code to a single metrics output implementation.
|
||||
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
|
||||
Adding a new module is easy and PRs are welcome :)
|
||||
|
||||
|
||||
Dipstick builds on stable Rust with minimal dependencies.
|
||||
|
||||
```rust
|
||||
|
@ -22,8 +22,8 @@ Metrics can be sent to multiple outputs at the same time.
|
|||
```rust
|
||||
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
|
||||
```
|
||||
Since instruments are decoupled from the backend, outputs can be swapped easily.
|
||||
|
||||
Since instruments are decoupled from the backend, outputs can be swapped easily.
|
||||
|
||||
Metrics can be aggregated and scheduled to be published periodically in the background.
|
||||
```rust
|
||||
use std::time::Duration;
|
||||
|
@ -31,38 +31,48 @@ let (to_aggregate, from_aggregate) = aggregate();
|
|||
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
|
||||
let app_metrics = metrics(to_aggregate);
|
||||
```
|
||||
Aggregation is performed locklessly and is very fast.
|
||||
Aggregation is performed locklessly and is very fast.
|
||||
Count, sum, min, max and average are tracked where they make sense.
|
||||
|
||||
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
|
||||
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
|
||||
```rust
|
||||
let (_to_aggregate, from_aggregate) = aggregate();
|
||||
publish(from_aggregate, to_log("my_custom_stats:"),
|
||||
publish(from_aggregate, to_log("my_custom_stats:"),
|
||||
|kind, name, score| match score {
|
||||
HitCount(hit) => Some((Counter, vec![name, ".per_thousand"], hit / 1000)),
|
||||
_ => None
|
||||
});
|
||||
```
|
||||
```
|
||||
|
||||
Metrics can be statistically sampled.
|
||||
```rust
|
||||
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
|
||||
```
|
||||
A fast random algorithm is used to pick samples.
|
||||
Outputs can use sample rate to expand or format published data.
|
||||
A fast random algorithm is used to pick samples.
|
||||
Outputs can use sample rate to expand or format published data.
|
||||
|
||||
Metrics can be recorded asynchronously.
|
||||
```rust
|
||||
let app_metrics = metrics(async(48, to_stdout()));
|
||||
```
|
||||
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
|
||||
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
|
||||
|
||||
Metric definitions can be cached to make using _ad-hoc metrics_ faster.
|
||||
```rust
|
||||
let app_metrics = metrics(cache(512, to_log()));
|
||||
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
|
||||
```
|
||||
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
|
||||
|
||||
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
|
||||
```rust
|
||||
#[macro_use] external crate lazy_static;
|
||||
|
||||
lazy_static! {
|
||||
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
|
||||
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
|
||||
}
|
||||
COUNTER_A.count(11);
|
||||
```
|
||||
|
||||
Timers can be used multiple ways.
|
||||
```rust
|
||||
|
@ -81,9 +91,11 @@ Related metrics can share a namespace.
|
|||
```rust
|
||||
let db_metrics = app_metrics.with_prefix("database.");
|
||||
let db_timer = db_metrics.timer("db_timer");
|
||||
let db_counter = db_metrics.counter("db_counter");
|
||||
let db_counter = db_metrics.counter("db_counter");
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Design
|
||||
Dipstick's design goals are to:
|
||||
- support as many metrics backends as possible while favoring none
|
||||
|
@ -93,13 +105,13 @@ Dipstick's design goals are to:
|
|||
|
||||
## Performance
|
||||
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
|
||||
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
|
||||
Defining a metric `cache()` reduces that cost for recurring metrics.
|
||||
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
|
||||
Defining a metric `cache()` reduces that cost for recurring metrics.
|
||||
|
||||
Run benchmarks with `cargo +nightly bench --features bench`.
|
||||
|
||||
## TODO
|
||||
Although already usable, Dipstick is still under heavy development and makes no guarantees
|
||||
## TODO
|
||||
Although already usable, Dipstick is still under heavy development and makes no guarantees
|
||||
of any kind at this point. See the following list for any potential caveats :
|
||||
- META turn TODOs into GitHub issues
|
||||
- generic publisher / sources
|
||||
|
@ -115,7 +127,11 @@ of any kind at this point. See the following list for any potential caveats :
|
|||
- more tests & benchmarks
|
||||
- complete doc / inline samples
|
||||
- more example apps
|
||||
- A cool logo
|
||||
- A cool logo
|
||||
- method annotation processors `#[timer("name")]`
|
||||
- fastsinks (M / &M) vs. safesinks (Arc<M>)
|
||||
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
|
||||
- fastsinks (M / &M) vs. safesinks (Arc<M>)
|
||||
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
|
||||
|
||||
License: MIT/Apache-2.0
|
||||
|
||||
_this file was generated using cargo readme_
|
||||
|
|
|
@ -0,0 +1,43 @@
|
|||
# {{crate}}
|
||||
|
||||
{{readme}}
|
||||
|
||||
## Design
|
||||
Dipstick's design goals are to:
|
||||
- support as many metrics backends as possible while favoring none
|
||||
- support all types of applications, from embedded to servers
|
||||
- promote metrics conventions that facilitate app monitoring and maintenance
|
||||
- stay out of the way in the code and at runtime (ergonomic, fast, resilient)
|
||||
|
||||
## Performance
|
||||
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
|
||||
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
|
||||
Defining a metric `cache()` reduces that cost for recurring metrics.
|
||||
|
||||
Run benchmarks with `cargo +nightly bench --features bench`.
|
||||
|
||||
## TODO
|
||||
Although already usable, Dipstick is still under heavy development and makes no guarantees
|
||||
of any kind at this point. See the following list for any potential caveats :
|
||||
- META turn TODOs into GitHub issues
|
||||
- generic publisher / sources
|
||||
- dispatch scopes
|
||||
- feature flags
|
||||
- derive stats
|
||||
- time measurement units in metric kind (us, ms, etc.) for naming & scaling
|
||||
- heartbeat metric on publish
|
||||
- logger templates
|
||||
- configurable aggregation
|
||||
- non-aggregating buffers
|
||||
- framework glue (rocket, iron, gotham, indicatif, etc.)
|
||||
- more tests & benchmarks
|
||||
- complete doc / inline samples
|
||||
- more example apps
|
||||
- A cool logo
|
||||
- method annotation processors `#[timer("name")]`
|
||||
- fastsinks (M / &M) vs. safesinks (Arc<M>)
|
||||
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
|
||||
|
||||
License: {{license}}
|
||||
|
||||
_this file was generated using cargo readme_
|
|
@ -13,9 +13,19 @@ fn main() {
|
|||
|
||||
let app_metrics = metrics((to_quick_aggregate, to_slow_aggregate));
|
||||
|
||||
publish_every(Duration::from_secs(3), from_quick_aggregate, to_stdout(), summary);
|
||||
publish_every(
|
||||
Duration::from_secs(3),
|
||||
from_quick_aggregate,
|
||||
to_stdout(),
|
||||
summary,
|
||||
);
|
||||
|
||||
publish_every(Duration::from_secs(10), from_slow_aggregate, to_stdout(), all_stats);
|
||||
publish_every(
|
||||
Duration::from_secs(10),
|
||||
from_slow_aggregate,
|
||||
to_stdout(),
|
||||
all_stats,
|
||||
);
|
||||
|
||||
let counter = app_metrics.counter("counter_a");
|
||||
loop {
|
||||
|
|
|
@ -1,12 +1,13 @@
|
|||
//! A sample application asynchronously printing metrics to stdout.
|
||||
|
||||
#[macro_use] extern crate dipstick;
|
||||
#[macro_use]
|
||||
extern crate dipstick;
|
||||
|
||||
use std::thread::sleep;
|
||||
use std::time::Duration;
|
||||
use dipstick::*;
|
||||
|
||||
use dipstick::core::{Sink, self};
|
||||
use dipstick::core::{self, Sink};
|
||||
|
||||
fn main() {
|
||||
|
||||
|
|
|
@ -1,7 +1,8 @@
|
|||
//! An app demonstrating the basics of the metrics front-end.
|
||||
//! Defines metrics of each kind and use them to print values to the console in multiple ways.
|
||||
|
||||
#[macro_use] extern crate dipstick;
|
||||
#[macro_use]
|
||||
extern crate dipstick;
|
||||
use std::thread::sleep;
|
||||
use std::time::Duration;
|
||||
use dipstick::*;
|
||||
|
|
|
@ -15,25 +15,39 @@ fn main() {
|
|||
// schedule aggregated metrics to be printed every 3 seconds
|
||||
let to_console = to_stdout();
|
||||
|
||||
publish_every(Duration::from_secs(3), from_aggregate, to_console, |kind, name, score|
|
||||
publish_every(Duration::from_secs(3), from_aggregate, to_console, |kind,
|
||||
name,
|
||||
score| {
|
||||
match kind {
|
||||
// do not export gauge scores
|
||||
Kind::Gauge => None,
|
||||
|
||||
_ => match score {
|
||||
// prepend and append to metric name
|
||||
ScoreType::HitCount(hit) => Some((Kind::Counter, vec!["name customized_with_prefix:", &name, " and a suffix: "], hit)),
|
||||
_ => {
|
||||
match score {
|
||||
// prepend and append to metric name
|
||||
ScoreType::HitCount(hit) => Some((
|
||||
Kind::Counter,
|
||||
vec![
|
||||
"name customized_with_prefix:",
|
||||
&name,
|
||||
" and a suffix: ",
|
||||
],
|
||||
hit,
|
||||
)),
|
||||
|
||||
// scaling the score value and appending unit to name
|
||||
ScoreType::SumOfValues(sum) => Some((kind, vec![&name, "_millisecond"], sum * 1000)),
|
||||
// scaling the score value and appending unit to name
|
||||
ScoreType::SumOfValues(sum) => Some(
|
||||
(kind, vec![&name, "_millisecond"], sum * 1000),
|
||||
),
|
||||
|
||||
// using the unmodified metric name
|
||||
ScoreType::AverageValue(avg) => Some((kind, vec![&name], avg)),
|
||||
_ => None /* do not export min and max */
|
||||
// using the unmodified metric name
|
||||
ScoreType::AverageValue(avg) => Some((kind, vec![&name], avg)),
|
||||
_ => None, /* do not export min and max */
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
);
|
||||
});
|
||||
|
||||
let counter = app_metrics.counter("counter_a");
|
||||
let timer = app_metrics.timer("timer_b");
|
||||
|
|
|
@ -8,8 +8,12 @@ fn main() {
|
|||
let metrics = metrics(
|
||||
// Metric caching allows re-use of the counter, skipping cost of redefining it on each use.
|
||||
cache(12, (
|
||||
to_statsd("localhost:8125", "myapp.").expect("Could not connect to statsd"),
|
||||
to_stdout())));
|
||||
to_statsd("localhost:8125", "myapp.").expect(
|
||||
"Could not connect to statsd",
|
||||
),
|
||||
to_stdout(),
|
||||
)),
|
||||
);
|
||||
|
||||
loop {
|
||||
metrics.counter("counter_a").count(123);
|
||||
|
|
|
@ -17,4 +17,3 @@ pub fn raw_write() {
|
|||
let counter = metrics_log.new_metric(Kind::Counter, "count_a", FULL_SAMPLING_RATE);
|
||||
metrics_log.new_scope()(Scope::Write(&counter, 1));
|
||||
}
|
||||
|
||||
|
|
|
@ -1,10 +1,8 @@
|
|||
//! Use statically defined app metrics & backend.
|
||||
//! This pattern is likely to emerge
|
||||
extern crate dipstick;
|
||||
#[macro_use] extern crate lazy_static;
|
||||
|
||||
use std::time::Duration;
|
||||
use dipstick::*;
|
||||
#[macro_use]
|
||||
extern crate lazy_static;
|
||||
|
||||
/// The `metric` module should be shared across the crate and contain metrics from all modules.
|
||||
/// Conventions are easier to uphold and document when all metrics are defined in the same place.
|
||||
|
@ -16,25 +14,15 @@ pub mod metric {
|
|||
// This makes it uglier than it should be when working with generics...
|
||||
// and is even more work because IDE's such as IntelliJ can not yet see through macro blocks :(
|
||||
lazy_static! {
|
||||
/// Central metric storage
|
||||
static ref AGGREGATE: (AggregateSink, AggregateSource) = aggregate();
|
||||
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
|
||||
|
||||
/// Application metrics are send to the aggregator
|
||||
pub static ref METRICS: AppMetrics<Aggregate, AggregateSink> = metrics(AGGREGATE.0.clone());
|
||||
|
||||
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
|
||||
pub static ref TIMER_B: Timer<Aggregate> = METRICS.timer("timer_b");
|
||||
pub static ref COUNTER_A: Counter<String> = METRICS.counter("counter_a");
|
||||
pub static ref TIMER_B: Timer<String> = METRICS.timer("timer_b");
|
||||
}
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let (to_aggregate, _from_aggregate) = aggregate();
|
||||
let app_metrics = metrics(to_aggregate);
|
||||
|
||||
loop {
|
||||
// The resulting application code is lean and clean
|
||||
metric::COUNTER_A.count(11);
|
||||
metric::TIMER_B.interval_us(654654);
|
||||
}
|
||||
|
||||
// The resulting application code is lean and clean
|
||||
metric::COUNTER_A.count(11);
|
||||
metric::TIMER_B.interval_us(654654);
|
||||
}
|
||||
|
|
101
src/lib.rs
101
src/lib.rs
|
@ -1,7 +1,100 @@
|
|||
//! A fast and modular metrics library decoupling app instrumentation from reporting backend.
|
||||
//! Similar to popular logging frameworks, but with counters and timers.
|
||||
//! Can be configured for combined outputs (log + statsd), random sampling, local aggregation
|
||||
//! of metrics, recurrent background publication, etc.
|
||||
/*!
|
||||
A fast and modular metrics toolkit for all Rust applications.
|
||||
Similar to popular logging frameworks, but with counters, markers, gauges and timers.
|
||||
|
||||
Out of the box, Dipstick _can_ aggregate, sample, cache and queue metrics (async).
|
||||
If aggregated, statistics can be published on demand or on schedule.
|
||||
|
||||
Dipstick does not bind application code to a single metrics output implementation.
|
||||
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
|
||||
Adding a new module is easy and PRs are welcome :)
|
||||
|
||||
Dipstick builds on stable Rust with minimal dependencies.
|
||||
|
||||
```rust
|
||||
use dipstick::*;
|
||||
let app_metrics = metrics(to_log("metrics:"));
|
||||
app_metrics.counter("my_counter").count(3);
|
||||
```
|
||||
|
||||
Metrics can be sent to multiple outputs at the same time.
|
||||
```rust
|
||||
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
|
||||
```
|
||||
Since instruments are decoupled from the backend, outputs can be swapped easily.
|
||||
|
||||
Metrics can be aggregated and scheduled to be published periodically in the background.
|
||||
```rust
|
||||
use std::time::Duration;
|
||||
let (to_aggregate, from_aggregate) = aggregate();
|
||||
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
|
||||
let app_metrics = metrics(to_aggregate);
|
||||
```
|
||||
Aggregation is performed locklessly and is very fast.
|
||||
Count, sum, min, max and average are tracked where they make sense.
|
||||
|
||||
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
|
||||
```rust
|
||||
let (_to_aggregate, from_aggregate) = aggregate();
|
||||
publish(from_aggregate, to_log("my_custom_stats:"),
|
||||
|kind, name, score| match score {
|
||||
HitCount(hit) => Some((Counter, vec![name, ".per_thousand"], hit / 1000)),
|
||||
_ => None
|
||||
});
|
||||
```
|
||||
|
||||
Metrics can be statistically sampled.
|
||||
```rust
|
||||
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
|
||||
```
|
||||
A fast random algorithm is used to pick samples.
|
||||
Outputs can use sample rate to expand or format published data.
|
||||
|
||||
Metrics can be recorded asynchronously.
|
||||
```rust
|
||||
let app_metrics = metrics(async(48, to_stdout()));
|
||||
```
|
||||
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
|
||||
|
||||
Metric definitions can be cached to make using _ad-hoc metrics_ faster.
|
||||
```rust
|
||||
let app_metrics = metrics(cache(512, to_log()));
|
||||
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
|
||||
```
|
||||
|
||||
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
|
||||
```rust
|
||||
#[macro_use] external crate lazy_static;
|
||||
|
||||
lazy_static! {
|
||||
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
|
||||
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
|
||||
}
|
||||
COUNTER_A.count(11);
|
||||
```
|
||||
|
||||
Timers can be used multiple ways.
|
||||
```rust
|
||||
let timer = app_metrics.timer("my_timer");
|
||||
time!(timer, {/* slow code here */} );
|
||||
timer.time(|| {/* slow code here */} );
|
||||
|
||||
let start = timer.start();
|
||||
/* slow code here */
|
||||
timer.stop(start);
|
||||
|
||||
timer.interval_us(123_456);
|
||||
```
|
||||
|
||||
Related metrics can share a namespace.
|
||||
```rust
|
||||
let db_metrics = app_metrics.with_prefix("database.");
|
||||
let db_timer = db_metrics.timer("db_timer");
|
||||
let db_counter = db_metrics.counter("db_counter");
|
||||
```
|
||||
|
||||
|
||||
*/
|
||||
|
||||
#![cfg_attr(feature = "bench", feature(test))]
|
||||
|
||||
|
|
Loading…
Reference in New Issue