Senior Datadog Architect & Implementation Engineer
About The Position
We are seeking a hands-on Senior Datadog Architect & Implementation Engineer to lead the Datadog implementation for a multi-account AWS observability rollout across Development, UAT, and Production environments. The platform supports a portfolio of AI agent solutions built using Amazon Bedrock, Bedrock AgentCore, AWS Lambda, and Amazon ECS. This is an implementation-focused position—not an advisory or dashboard-only role. The successful candidate will deliver a fully operational observability solution, including live dashboards, correctly configured alerts, centralized logs, metrics, distributed traces, and production-ready notification routing.
Requirements
- Strong hands-on experience implementing Datadog in multi-account AWS environments.
- Proven experience configuring Datadog AWS metrics polling.
- Hands-on experience deploying and supporting the Datadog Forwarder Lambda.
- Experience configuring CloudWatch Logs subscription filters and centralized log forwarding.
- Strong knowledge of AWS Organizations and organization-wide cloud deployments.
- Hands-on experience with CloudFormation and CloudFormation StackSets.
- Strong understanding of cross-account IAM roles, trust policies, permissions, and security controls.
- Experience with AWS X-Ray, OpenTelemetry, distributed tracing, metrics, logs, and spans.
- Hands-on experience supporting AWS Lambda, Amazon ECS/Fargate, CloudWatch Logs, and CloudWatch Metrics.
- Experience building production Datadog dashboards, monitors, alerts, and notification workflows.
- Ability to implement solutions against an existing enterprise architecture.
- Experience working through security, compliance, and IAM approval processes.
- Strong troubleshooting, cross-team delivery, and production-support skills.
- Excellent written and verbal communication skills.
Nice To Haves
- Experience monitoring AI or generative AI solutions built with Amazon Bedrock or Bedrock AgentCore.
- Experience creating token-usage and cost-monitoring dashboards for AI workloads.
- Knowledge of Datadog tagging strategies and AWS tag taxonomies.
- Datadog or AWS professional certifications.
- Experience delivering observability solutions in regulated enterprise environments.
Responsibilities
- Design and deploy an organization-wide Datadog AWS integration across Dev, UAT, and Prod environments.
- Implement the integration using AWS CloudFormation StackSets, AWS Organizations, and secure cross-account IAM roles.
- Build and maintain the CloudWatch Logs-to-Datadog pipeline using the Datadog Forwarder Lambda and CloudWatch subscription filters.
- Configure Datadog AWS metrics polling across multiple AWS accounts.
- Enable CloudWatch Transaction Search and configure the Datadog AWS X-Ray integration.
- Ensure AgentCore trace and span data is visible and searchable within Datadog.
- Implement distributed tracing and telemetry using AWS X-Ray and OpenTelemetry.
- Create tag-driven dashboards organized by environment, AWS account, application, and AI solution.
- Build dashboards for AI token consumption, platform usage, performance, reliability, and cost monitoring.
- Define, configure, and tune Datadog monitors and alert thresholds.
- Establish notification routing, escalation paths, and production-support procedures.
- Troubleshoot issues involving missing logs, metrics, traces, permissions, tags, and cross-account integrations.
- Work with security, cloud infrastructure, platform, and application teams to complete IAM and compliance reviews.
- Provide production support and maintain the implemented Datadog platform.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
