Senior Data Platform Engineer
NewVison•San Francisco, CA
•Remote
About The Position
We are seeking a Senior Data Platform Engineer to design, optimize, and manage enterprise-grade data ecosystems. The ideal candidate will have mastery of database architecture, performance tuning, and transaction management, along with hands-on experience in cloud-native platforms, big data frameworks, and advanced analytics pipelines. You will be comfortable working across distributed systems, modern data engineering tools, and collaborating with architects, DevOps, and business teams to deliver scalable, secure, and high-performing solutions.
Requirements
- Advanced SQL, query optimization, partitioning, and transaction handling.
- Proven ability to diagnose and resolve complex performance bottlenecks.
- Hands-on experience with Azure Service Fabric, Azure Data Factory, Azure Databricks, and Azure Storage.
- Strong knowledge of Spark, Delta Lake, and distributed data processing.
- Familiarity with Power BI and data modeling best practices.
- Ability to resolve issues across multi-tiered systems under pressure.
- Excellent communication and stakeholder management skills.
Nice To Haves
- Experience with CI/CD pipelines for data solutions.
- Familiarity with data governance, security, and compliance frameworks.
- Exposure to machine learning workflows and real-time streaming architectures.
Responsibilities
- Design and implement robust, scalable, and secure database architectures for transactional and analytical workloads.
- Define partitioning strategies, indexing, and normalization for optimal performance and maintainability.
- Conduct advanced query optimization and execution plan analysis.
- Implement proactive monitoring and troubleshooting for high-volume, mission-critical systems.
- Deploy and manage microservices-based applications on Azure Service Fabric clusters.
- Ensure high availability, fault tolerance, and scalability of distributed services.
- Optimize communication patterns and stateful/stateless service design for data-intensive workloads.
- Design and orchestrate complex ETL/ELT pipelines for batch and streaming data using Azure Data Factory (ADF).
- Integrate diverse data sources into centralized data platforms.
- Build and optimize big data processing workflows using Apache Spark on Databricks.
- Implement Delta Lake for ACID transactions on large-scale data lakes.
- Develop scalable solutions for machine learning and advanced analytics.
- Manage versioned data storage for reliability and reproducibility using Delta Tables.
- Optimize schema evolution and data compaction strategies.
- Collaborate with BI teams to design semantic models and optimize data for reporting using Power BI (PBI).
- Ensure data pipelines deliver clean, consistent, and performant datasets for visualization.
- Work closely with architects, DevOps, and business stakeholders to align technical solutions with strategic objectives.
- Provide mentorship and technical guidance to junior engineers and developers.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
