Senior Software Engineer

Microsoft•Redmond, WA

About The Position

The CoreServices Telemetry team, part of Agent 365 Foundation, builds and operates the telemetry platform for Dataverse, CoreServices Fault Domain Islands, and Dynamics 365 Finance and Operations. The broader foundation supports Dynamics 365, Power Platform, and Copilot and agent workloads. The platform covers signal collection, processing, storage, querying, and alerting across virtual machines, Service Fabric, and Kubernetes-based Core Services Containers, using Geneva, Azure Data Explorer (Kusto), and data lake pipelines. The team also builds tools that help engineers and AI assistants discover and use telemetry. The work focuses on keeping data timely, accurate, accessible, and cost-effective across regions and sovereign clouds while enforcing privacy and data residency boundaries. This is a hands-on software engineering role building the telemetry and observability foundation behind business applications and emerging agent workloads. The engineer will write and debug collection and processing services, improve ingestion and query performance, and ship changes safely across a distributed production fleet. The role also contributes to AI-assisted investigation tools that connect engineers to relevant signals and help diagnose incidents. The engineer owns outcomes across pipeline behavior, missing or delayed data detection, and tradeoffs among fidelity, latency, retention, and cost, and participates in on-call operations.

Requirements

Bachelor's degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

Master's degree in Computer Science or a related technical field and 6+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python; or bachelor's degree in Computer Science or a related technical field and 8+ years of such experience; or equivalent experience.
Solid software engineering fundamentals, including data structures, concurrency, asynchronous processing, automated testing, and diagnosing performance and reliability problems in production.
Practical experience with distributed data processing or telemetry pipelines, including ingestion, buffering, retries, partitioning, data quality, and throughput, latency, and cost tradeoffs.
Production experience in a modern programming language such as C#, Go, Java, or C++, and ability to learn the team's languages and tools.
Demonstrated experience making services resilient under stress, validating failure and recovery behavior, and using logs, metrics, and queries to explain system behavior.
Experience operating services, including on-call ownership, live-site debugging, and post-incident improvement.
Hands-on experience with Azure Data Explorer and KQL, including ingestion, query optimization, retention policies, and managing shared analytics capacity.
Experience with observability technologies such as Geneva, OpenTelemetry, Prometheus, or comparable collection, metrics, and monitoring platforms.
Experience operating services on Kubernetes or Service Fabric, building stream-processing or data lake pipelines, and automating infrastructure and deployments in Azure.
Familiarity with multi-tenant cloud platforms, privacy-aware data handling, regional data residency, or sovereign cloud operations.
Background in capacity planning, telemetry cost optimization, or measuring service health and customer impact in large-scale cloud systems.
Experience building AI-assisted developer tools, MCP integrations, or incident investigation automation, including evaluating output quality and applying appropriate access controls and operational safeguards.

Responsibilities

Design, implement, and operate telemetry collection and processing services across Dataverse, Service Fabric, Kubernetes, and Finance and Operations.
Partner with teams building Copilot and agent workloads to make their platform dependencies observable.
Own data quality and pipeline behavior, including event schemas, filtering, transformations, routing, ingestion latency, completeness, and recovery from delayed or failed delivery.
Make delivery guarantees and failure modes explicit; build resilience through bounded buffering, backpressure, retries, circuit breakers, and dependency isolation.
Validate failure recovery and capacity through automated tests, load testing, and controlled fault injection without putting source workload availability at risk.
Enforce privacy, security, and data residency throughout collection, routing, storage, retention, and access; build least-privilege controls and auditable operations that respect regional, EU Data Boundary, and sovereign cloud requirements.
Improve Azure Data Explorer ingestion and query performance, storage efficiency, and capacity management.
Use evidence to balance telemetry coverage and retention against compute, storage, and operational cost, and automate safe deployments and configuration changes.
Instrument the telemetry platform with actionable health signals and alerts.
Drive incident mitigation, root-cause analysis, and lasting repairs; improve service-health measurements and diagnostic tools used by partner teams to understand customer impact.
Partner with service teams on instrumentation, onboarding, query design, and operational readiness.
Build self-service tools and documentation, mentor engineers, and raise the engineering bar through design and code review.
Build and improve AI-assisted diagnostics and Model Context Protocol (MCP) integrations that help engineers discover telemetry, generate useful queries, and correlate evidence across services.
Evaluate AI-assisted diagnostics for accuracy and usefulness, respect data-access boundaries, and keep production-changing actions subject to appropriate safeguards.
Participate in on-call operations and own service outcomes.