diff --git a/_category/core-infrastructure.md b/_category/core-infrastructure.md new file mode 100644 index 0000000..201a9cf --- /dev/null +++ b/_category/core-infrastructure.md @@ -0,0 +1,4 @@ +--- +team: Core Infrastructure +permalink: "/blog/category/core-infrastructure" +--- diff --git a/_data/authors.yml b/_data/authors.yml index c683a93..7eb6fb1 100644 --- a/_data/authors.yml +++ b/_data/authors.yml @@ -81,3 +81,7 @@ jasonb: lbuschbaum: name: Lori Buschbaum github: lbuschbaum + +jimp: + name: Jim Park + github: jim80net diff --git a/_includes/post-hero.html b/_includes/post-hero.html index c35c5f4..683c1b2 100644 --- a/_includes/post-hero.html +++ b/_includes/post-hero.html @@ -40,6 +40,26 @@ {%- endif -%} + {%- if page.authors -%} +
  • +
    + Author
    + {%- for author in page.authors -%} + + {%- endfor -%} +
    +
    • + {%- endif -%} + {%- if page.date -%}
  • diff --git a/_posts/2019-12-03-managing-pagerduty-rotations.md b/_posts/2019-12-03-managing-pagerduty-rotations.md index 3959270..9ddf3e5 100644 --- a/_posts/2019-12-03-managing-pagerduty-rotations.md +++ b/_posts/2019-12-03-managing-pagerduty-rotations.md @@ -5,6 +5,7 @@ author: hamiltonh tags: - oncall - pagerduty +- monitoring - incident response team: Core Platform --- diff --git a/_posts/2020-02-20-pagerduty-at-scribd.md b/_posts/2020-02-20-pagerduty-at-scribd.md index 6a003c9..89b19f4 100644 --- a/_posts/2020-02-20-pagerduty-at-scribd.md +++ b/_posts/2020-02-20-pagerduty-at-scribd.md @@ -4,6 +4,7 @@ title: "A testimonial for using PagerDuty at Scribd" author: rtyler tags: - pagerduty +- monitoring - oncall - incident response team: Core Platform diff --git a/_posts/2020-04-22-how-scribd-manages-datadog-aws-integration-using-terraform.md b/_posts/2020-04-22-how-scribd-manages-datadog-aws-integration-using-terraform.md new file mode 100644 index 0000000..aaa1be7 --- /dev/null +++ b/_posts/2020-04-22-how-scribd-manages-datadog-aws-integration-using-terraform.md @@ -0,0 +1,105 @@ +--- +layout: post +title: "How Scribd manages Datadog’s AWS integration using Terraform" +authors: +- jimp +- qphou +tags: +- featured +- terraform +- monitoring +team: Core Infrastructure +--- + +Datadog comes with a builtin AWS +[integration](https://docs.datadoghq.com/integrations/amazon_web_services/?tab=allpermissions) +to ship CloudWatch metrics to your Datadog account. Once enabled, the +integration will automatically synchronize whitelisted CloudWatch metrics into +your Datadog account. + +While this integration is powerful and convenient to use, it’s setup process is +actually quite involved. As outlined in [Datadog's documentation](https://docs.datadoghq.com/integrations/amazon_web_services/?tab=allpermissions), there are 18 +manual steps required, including: + +- finding the right AWS account ID +- creating the right IAM policy +- copy pasting the right AWS resource ID into Datadog UI +- etc. + +If you have more than a few AWS accounts, you may prefer to use Terraform. + +In this blog post, we would like to share how Scribd uses Terraform to automate +our Datadog and AWS integration across the organization. + +# Enable Datadog’s builtin AWS integration + +To address this problem, we built the [terraform-aws-datadog +module](https://github.com/scribd/terraform-aws-datadog). With only couple +lines of HCL code, Terraform will perform all the necessary steps to setup +Datadog integration with a specific AWS account with Scribd’s best practices: + +```terraform +module "datadog" { + source = "git::https://github.com/scribd/terraform-aws-datadog.git?ref=master" + aws_account_id = data.aws_caller_identity.current.account_id + datadog_api_key = var.datadog_api_key + env = "prod" + namespace = "team_foo" +} +``` + +The benefit from an AWS Account maintainer point of view is that using the +module is a convenient way to inherit centralized best practice. For module +maintainers, any change to the datadog integration module can be released using +a [standard Terraform module release process](https://www.terraform.io/docs/registry/modules/publish.html). + + +# Cloudwatch log synchronization + +Initially, the module only sets up the base integration. As adoption increased, more +features were added to the module by various teams. One of these features is +automation for setting up log ingestion for cloudwatch. + +Like setting up the official AWS integration app, the [instructions for log +synchronization](https://docs.datadoghq.com/integrations/amazon_web_services/?tab=allpermissions#log-collection) +are a bit overwhelming. + +However, using the terraform-aws-datadog module, we can enable the feature with a single parameter: + +```terraform +module "datadog" { + source = "git::https://github.com/scribd/terraform-aws-datadog.git?ref=master" + datadog_api_key = var.datadog_api_key + env = "prod" + namespace = "project_foo" + cloudwatch_log_groups = ["cloudwatch_log_group_1", "cloudwatch_log_group_2"] +} +``` + +That’s it, Terraform will automatically create the datadog serverless function +and triggers for specified log groups to forward all cloudwatch logs into +Datadog. After running terraform apply, you should be able to see logs showing +up in Datadog within minutes. + + +# Future work + +With both metrics and logs synchronized into Datadog, we are able to leverage +Datadog as the central hub for all things monitoring. We are planning to bring +more features to the module as we migrate Scribd’s infrastructure into AWS. + +Metrics ingested through the official AWS integration are delayed by couple +minutes, which is not ideal to use as signals for monitoring critical systems. +There are opportunities to enable real time metrics synchronization by +automating datadog agent setup. + +The [datadog-serverless-functions +repo](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws) +contains two other lambda based AWS augmentations that we may add as available +features of the module: `vpc_flow_log_monitoring` and `rds_enhanced_monitoring`. + +Stay apprised of future releases by watching our [release page](https://github.com/scribd/terraform-aws-datadog/releases). + +_Special shout out to Taylor McClure and Hamilton Hord for starting the project, as well +as Sai Kiran Burle, Kamran Farhadi and Eugene Pimenov for improvements and bug +fixes._ diff --git a/generate-tags b/generate-tags index 2c798c6..150fa80 100755 --- a/generate-tags +++ b/generate-tags @@ -12,7 +12,7 @@ for filename in glob.glob(post_dir + '*.md'): matcher = r'^tags:$' with open(filename, 'r') as fd: tagged_line = False - for line in fd.xreadlines(): + for line in fd: if tagged_line: if line.startswith('---'): tagged_line = False diff --git a/tag/agile/index.md b/tag/agile/index.md new file mode 100644 index 0000000..235d746 --- /dev/null +++ b/tag/agile/index.md @@ -0,0 +1,6 @@ +--- +layout: tag_page +title: "Tag: agile" +tag: agile +robots: noindex +--- diff --git a/tag/monitoring/index.md b/tag/monitoring/index.md new file mode 100644 index 0000000..c8011c9 --- /dev/null +++ b/tag/monitoring/index.md @@ -0,0 +1,6 @@ +--- +layout: tag_page +title: "Tag: monitoring" +tag: monitoring +robots: noindex +--- diff --git a/tag/terraform/index.md b/tag/terraform/index.md new file mode 100644 index 0000000..8aae77f --- /dev/null +++ b/tag/terraform/index.md @@ -0,0 +1,6 @@ +--- +layout: tag_page +title: "Tag: terraform" +tag: terraform +robots: noindex +---