Senior Data Platform Engineer
NCC Group
•Hybrid
About The Position
We are seeking a skilled Data Engineer to join our Engineering team, responsible for designing, building, and optimising scalable data pipelines that power advanced analytics and machine learning solutions. You will play a key role in enabling data-driven decision-making by delivering high-quality, reliable datasets to tools such as Amazon SageMaker and other analytics platforms.
Requirements
- Strong experience in data engineering within AWS cloud environments.
- Hands-on experience with AWS big data technologies such as EMR, S3 and SageMaker.
- Proficiency in Python for building scalable data pipelines and processing frameworks.
- Experience with Apache Spark for distributed data processing.
- Experience designing and maintaining scalable batch and real-time data pipelines.
- Solid understanding of ETL/ELT design patterns and data modelling techniques.
- Experience with workflow orchestration tools such as Apache Airflow (ideally deployed on AWS).
- Familiarity with containerisation and orchestration using Docker and Kubernetes (EKS).
- Experience with infrastructure as code (e.g. Terraform) and CI/CD/GitOps practices.
- Proven ability to optimise performance and reduce cloud costs through partitioning, clustering and workload management.
- Understanding of data security principles, including data loss prevention (DLP).
Nice To Haves
- Experience with Databricks or similar third-party big data platforms.
- Knowledge of real-time streaming technologies (e.g. Kafka, Kinesis).
- Experience implementing data governance and compliance frameworks.
- Familiarity with monitoring and observability tools in AWS environments.
- Exposure to Lakehouse or modern data platform architectures.
Responsibilities
- Develop robust data pipelines that feed analytics and machine learning tools such as Amazon SageMaker and third-party platforms like Databricks.
- Leverage AWS technologies such as EMR, S3, EKS and Airflow to process and orchestrate high-volume datasets, ensuring solutions are scalable, resilient and cost-efficient.
- Embed data loss prevention (DLP) principles and controls into data pipelines to protect sensitive information.
- Ensure data is reliable, accessible, well-governed and optimised for downstream consumption.
Benefits
- Flexible Working
- Generous Holiday Allowance (25 days + bank holidays, option to buy up to 5 additional days)
- Medicash & Critical Illness Scheme
- Pension
- Life Assurance
- Share Save Scheme
- Community & Volunteering Programmes
- Green Car Scheme
- Cycle Scheme
- Special Time Off (marriage/civil partnership, becoming a grandparent, welcoming a new pet)
- Generous maternity and paternity leave
- Time off and support for fertility treatments
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
