Adopt cargo readme & cargo make

This commit is contained in:
Francis Lalonde 2017-09-29 16:45:09 -04:00
parent 5608f21e93
commit 3897ec6d55
10 changed files with 235 additions and 66 deletions

View File

@ -1,15 +1,15 @@
# Dipstick
# dipstick
A fast and modular metrics toolkit for all Rust applications.
A fast and modular metrics toolkit for all Rust applications.
Similar to popular logging frameworks, but with counters, markers, gauges and timers.
Out of the box, Dipstick _can_ aggregate, sample, cache and queue metrics (async).
If aggregated, statistics can be published on demand or on schedule.
Dipstick does not bind application code to a single metrics output implementation.
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
Dipstick does not bind application code to a single metrics output implementation.
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
Adding a new module is easy and PRs are welcome :)
Dipstick builds on stable Rust with minimal dependencies.
```rust
@ -22,8 +22,8 @@ Metrics can be sent to multiple outputs at the same time.
```rust
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
```
Since instruments are decoupled from the backend, outputs can be swapped easily.
Since instruments are decoupled from the backend, outputs can be swapped easily.
Metrics can be aggregated and scheduled to be published periodically in the background.
```rust
use std::time::Duration;
@ -31,38 +31,48 @@ let (to_aggregate, from_aggregate) = aggregate();
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
let app_metrics = metrics(to_aggregate);
```
Aggregation is performed locklessly and is very fast.
Aggregation is performed locklessly and is very fast.
Count, sum, min, max and average are tracked where they make sense.
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
```rust
let (_to_aggregate, from_aggregate) = aggregate();
publish(from_aggregate, to_log("my_custom_stats:"),
publish(from_aggregate, to_log("my_custom_stats:"),
|kind, name, score| match score {
HitCount(hit) => Some((Counter, vec![name, ".per_thousand"], hit / 1000)),
_ => None
});
```
```
Metrics can be statistically sampled.
```rust
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
```
A fast random algorithm is used to pick samples.
Outputs can use sample rate to expand or format published data.
A fast random algorithm is used to pick samples.
Outputs can use sample rate to expand or format published data.
Metrics can be recorded asynchronously.
```rust
let app_metrics = metrics(async(48, to_stdout()));
```
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
Metric definitions can be cached to make using _ad-hoc metrics_ faster.
```rust
let app_metrics = metrics(cache(512, to_log()));
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
```
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
```rust
#[macro_use] external crate lazy_static;
lazy_static! {
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
}
COUNTER_A.count(11);
```
Timers can be used multiple ways.
```rust
@ -81,9 +91,11 @@ Related metrics can share a namespace.
```rust
let db_metrics = app_metrics.with_prefix("database.");
let db_timer = db_metrics.timer("db_timer");
let db_counter = db_metrics.counter("db_counter");
let db_counter = db_metrics.counter("db_counter");
```
## Design
Dipstick's design goals are to:
- support as many metrics backends as possible while favoring none
@ -93,13 +105,13 @@ Dipstick's design goals are to:
## Performance
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
Defining a metric `cache()` reduces that cost for recurring metrics.
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
Defining a metric `cache()` reduces that cost for recurring metrics.
Run benchmarks with `cargo +nightly bench --features bench`.
## TODO
Although already usable, Dipstick is still under heavy development and makes no guarantees
## TODO
Although already usable, Dipstick is still under heavy development and makes no guarantees
of any kind at this point. See the following list for any potential caveats :
- META turn TODOs into GitHub issues
- generic publisher / sources
@ -115,7 +127,11 @@ of any kind at this point. See the following list for any potential caveats :
- more tests & benchmarks
- complete doc / inline samples
- more example apps
- A cool logo
- A cool logo
- method annotation processors `#[timer("name")]`
- fastsinks (M / &M) vs. safesinks (Arc<M>)
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
- fastsinks (M / &M) vs. safesinks (Arc<M>)
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
License: MIT/Apache-2.0
_this file was generated using cargo readme_

43
README.tpl Normal file
View File

@ -0,0 +1,43 @@
# {{crate}}
{{readme}}
## Design
Dipstick's design goals are to:
- support as many metrics backends as possible while favoring none
- support all types of applications, from embedded to servers
- promote metrics conventions that facilitate app monitoring and maintenance
- stay out of the way in the code and at runtime (ergonomic, fast, resilient)
## Performance
Predefined timers use a bit more code but are generally faster because their initialization cost is is only paid once.
Ad-hoc timers are redefined "inline" on each use. They are more flexible, but have more overhead because their init cost is paid on each use.
Defining a metric `cache()` reduces that cost for recurring metrics.
Run benchmarks with `cargo +nightly bench --features bench`.
## TODO
Although already usable, Dipstick is still under heavy development and makes no guarantees
of any kind at this point. See the following list for any potential caveats :
- META turn TODOs into GitHub issues
- generic publisher / sources
- dispatch scopes
- feature flags
- derive stats
- time measurement units in metric kind (us, ms, etc.) for naming & scaling
- heartbeat metric on publish
- logger templates
- configurable aggregation
- non-aggregating buffers
- framework glue (rocket, iron, gotham, indicatif, etc.)
- more tests & benchmarks
- complete doc / inline samples
- more example apps
- A cool logo
- method annotation processors `#[timer("name")]`
- fastsinks (M / &M) vs. safesinks (Arc<M>)
- `static_metric!` macro to replace `lazy_static!` blocks and handle generics boilerplate.
License: {{license}}
_this file was generated using cargo readme_

View File

@ -13,9 +13,19 @@ fn main() {
let app_metrics = metrics((to_quick_aggregate, to_slow_aggregate));
publish_every(Duration::from_secs(3), from_quick_aggregate, to_stdout(), summary);
publish_every(
Duration::from_secs(3),
from_quick_aggregate,
to_stdout(),
summary,
);
publish_every(Duration::from_secs(10), from_slow_aggregate, to_stdout(), all_stats);
publish_every(
Duration::from_secs(10),
from_slow_aggregate,
to_stdout(),
all_stats,
);
let counter = app_metrics.counter("counter_a");
loop {

View File

@ -1,12 +1,13 @@
//! A sample application asynchronously printing metrics to stdout.
#[macro_use] extern crate dipstick;
#[macro_use]
extern crate dipstick;
use std::thread::sleep;
use std::time::Duration;
use dipstick::*;
use dipstick::core::{Sink, self};
use dipstick::core::{self, Sink};
fn main() {

View File

@ -1,7 +1,8 @@
//! An app demonstrating the basics of the metrics front-end.
//! Defines metrics of each kind and use them to print values to the console in multiple ways.
#[macro_use] extern crate dipstick;
#[macro_use]
extern crate dipstick;
use std::thread::sleep;
use std::time::Duration;
use dipstick::*;

View File

@ -15,25 +15,39 @@ fn main() {
// schedule aggregated metrics to be printed every 3 seconds
let to_console = to_stdout();
publish_every(Duration::from_secs(3), from_aggregate, to_console, |kind, name, score|
publish_every(Duration::from_secs(3), from_aggregate, to_console, |kind,
name,
score| {
match kind {
// do not export gauge scores
Kind::Gauge => None,
_ => match score {
// prepend and append to metric name
ScoreType::HitCount(hit) => Some((Kind::Counter, vec!["name customized_with_prefix:", &name, " and a suffix: "], hit)),
_ => {
match score {
// prepend and append to metric name
ScoreType::HitCount(hit) => Some((
Kind::Counter,
vec![
"name customized_with_prefix:",
&name,
" and a suffix: ",
],
hit,
)),
// scaling the score value and appending unit to name
ScoreType::SumOfValues(sum) => Some((kind, vec![&name, "_millisecond"], sum * 1000)),
// scaling the score value and appending unit to name
ScoreType::SumOfValues(sum) => Some(
(kind, vec![&name, "_millisecond"], sum * 1000),
),
// using the unmodified metric name
ScoreType::AverageValue(avg) => Some((kind, vec![&name], avg)),
_ => None /* do not export min and max */
// using the unmodified metric name
ScoreType::AverageValue(avg) => Some((kind, vec![&name], avg)),
_ => None, /* do not export min and max */
}
}
}
);
});
let counter = app_metrics.counter("counter_a");
let timer = app_metrics.timer("timer_b");

View File

@ -8,8 +8,12 @@ fn main() {
let metrics = metrics(
// Metric caching allows re-use of the counter, skipping cost of redefining it on each use.
cache(12, (
to_statsd("localhost:8125", "myapp.").expect("Could not connect to statsd"),
to_stdout())));
to_statsd("localhost:8125", "myapp.").expect(
"Could not connect to statsd",
),
to_stdout(),
)),
);
loop {
metrics.counter("counter_a").count(123);

View File

@ -17,4 +17,3 @@ pub fn raw_write() {
let counter = metrics_log.new_metric(Kind::Counter, "count_a", FULL_SAMPLING_RATE);
metrics_log.new_scope()(Scope::Write(&counter, 1));
}

View File

@ -1,10 +1,8 @@
//! Use statically defined app metrics & backend.
//! This pattern is likely to emerge
extern crate dipstick;
#[macro_use] extern crate lazy_static;
use std::time::Duration;
use dipstick::*;
#[macro_use]
extern crate lazy_static;
/// The `metric` module should be shared across the crate and contain metrics from all modules.
/// Conventions are easier to uphold and document when all metrics are defined in the same place.
@ -16,25 +14,15 @@ pub mod metric {
// This makes it uglier than it should be when working with generics...
// and is even more work because IDE's such as IntelliJ can not yet see through macro blocks :(
lazy_static! {
/// Central metric storage
static ref AGGREGATE: (AggregateSink, AggregateSource) = aggregate();
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
/// Application metrics are send to the aggregator
pub static ref METRICS: AppMetrics<Aggregate, AggregateSink> = metrics(AGGREGATE.0.clone());
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
pub static ref TIMER_B: Timer<Aggregate> = METRICS.timer("timer_b");
pub static ref COUNTER_A: Counter<String> = METRICS.counter("counter_a");
pub static ref TIMER_B: Timer<String> = METRICS.timer("timer_b");
}
}
fn main() {
let (to_aggregate, _from_aggregate) = aggregate();
let app_metrics = metrics(to_aggregate);
loop {
// The resulting application code is lean and clean
metric::COUNTER_A.count(11);
metric::TIMER_B.interval_us(654654);
}
// The resulting application code is lean and clean
metric::COUNTER_A.count(11);
metric::TIMER_B.interval_us(654654);
}

View File

@ -1,7 +1,100 @@
//! A fast and modular metrics library decoupling app instrumentation from reporting backend.
//! Similar to popular logging frameworks, but with counters and timers.
//! Can be configured for combined outputs (log + statsd), random sampling, local aggregation
//! of metrics, recurrent background publication, etc.
/*!
A fast and modular metrics toolkit for all Rust applications.
Similar to popular logging frameworks, but with counters, markers, gauges and timers.
Out of the box, Dipstick _can_ aggregate, sample, cache and queue metrics (async).
If aggregated, statistics can be published on demand or on schedule.
Dipstick does not bind application code to a single metrics output implementation.
Outputs `to_log`, `to_stdout` and `to_statsd` are currently provided.
Adding a new module is easy and PRs are welcome :)
Dipstick builds on stable Rust with minimal dependencies.
```rust
use dipstick::*;
let app_metrics = metrics(to_log("metrics:"));
app_metrics.counter("my_counter").count(3);
```
Metrics can be sent to multiple outputs at the same time.
```rust
let app_metrics = metrics((to_stdout(), to_statsd("localhost:8125", "app1.host.")));
```
Since instruments are decoupled from the backend, outputs can be swapped easily.
Metrics can be aggregated and scheduled to be published periodically in the background.
```rust
use std::time::Duration;
let (to_aggregate, from_aggregate) = aggregate();
publish_every(Duration::from_secs(10), from_aggregate, to_log("last_ten_secs:"), all_stats);
let app_metrics = metrics(to_aggregate);
```
Aggregation is performed locklessly and is very fast.
Count, sum, min, max and average are tracked where they make sense.
Publishing can use predefined strategies `all_stats`, `summary`, `average` or a custom one.
```rust
let (_to_aggregate, from_aggregate) = aggregate();
publish(from_aggregate, to_log("my_custom_stats:"),
|kind, name, score| match score {
HitCount(hit) => Some((Counter, vec![name, ".per_thousand"], hit / 1000)),
_ => None
});
```
Metrics can be statistically sampled.
```rust
let app_metrics = metrics(sample(0.001, to_statsd("server:8125", "app.sampled.")));
```
A fast random algorithm is used to pick samples.
Outputs can use sample rate to expand or format published data.
Metrics can be recorded asynchronously.
```rust
let app_metrics = metrics(async(48, to_stdout()));
```
The async queue uses a Rust channel and a standalone thread. Its current behavior is to block when full.
Metric definitions can be cached to make using _ad-hoc metrics_ faster.
```rust
let app_metrics = metrics(cache(512, to_log()));
app_metrics.gauge(format!("my_gauge_{}", 34)).value(44);
```
The preferred way is to _predefine metrics_, possibly in a [lazy_static!](https://crates.io/crates/lazy_static) block.
```rust
#[macro_use] external crate lazy_static;
lazy_static! {
pub static ref METRICS: AppMetrics<String, FnSink<String>> = metrics(to_stdout());
pub static ref COUNTER_A: Counter<Aggregate> = METRICS.counter("counter_a");
}
COUNTER_A.count(11);
```
Timers can be used multiple ways.
```rust
let timer = app_metrics.timer("my_timer");
time!(timer, {/* slow code here */} );
timer.time(|| {/* slow code here */} );
let start = timer.start();
/* slow code here */
timer.stop(start);
timer.interval_us(123_456);
```
Related metrics can share a namespace.
```rust
let db_metrics = app_metrics.with_prefix("database.");
let db_timer = db_metrics.timer("db_timer");
let db_counter = db_metrics.counter("db_counter");
```
*/
#![cfg_attr(feature = "bench", feature(test))]