Senior Data Engineer

Kainos•Halifax, NS

117d

About The Position

Kainos is recognized as a leading AI and data business with a decade-long track record of delivering impactful, production-grade Data and AI solutions for clients across government, healthcare, defence, and commercial sectors. Kainos is at the forefront of AI innovation, trusted by Microsoft, AWS, and others to deliver advanced AI and data solutions at citizen scale. Our 150-strong AI and Data Practice brings together deep expertise in data, machine learning, generative AI, and agentic AI. We are pioneers in responsible AI, and we partner with leading organisations to ensure AI is deployed ethically, securely and with measurable business value. Our teams are at the cutting edge of data and AI research, and delivery, making it an exciting team to join Kainos as we further grow our capability.

Requirements

Strong software development experience in one of Java, Scala, or Python
Software development experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
Experience of developing substantial components for large-scale data processing solutions and deploying into a production environment
Proficient in SQL and SQL extensions for analytical queries
Solid understanding of ETL/ELT data processing pipelines and design patterns
Aware of key features and pitfalls of distributed data processing frameworks, data stores and data serialisation formats
Able to write quality, testable code and has experience of automated testing
Experience with Continuous Integration and Continuous Deployment techniques

Nice To Haves

Experience of data pipeline development using Apache Nifi
Experience of Docker and Kubernetes
Experience of performance tuning
Experience of data visualisation and complex data transformations
Experience with streaming and event-processing architectures including technologies such as Kafka and change-data-capture (CDC) products
Expertise in continuous improvement and sharing input on data best practice

Responsibilities

Designing and developing data processing and data persistence software components for solutions which handle data at scale.
Providing strong development leadership and taking responsibility for significant technical components of data systems.
Working within a multi-skilled agile team to design and develop large-scale data processing software to meet user needs in demanding production environments.
Working to develop data processing software primarily for deployment in Big Data technologies.
Encompassing the full software lifecycle including design, code, test and defect resolution.
Working with Architects and Lead Engineers to ensure the software supports non-functional needs.
Collaborating with colleagues to resolve implementation challenges and ensure code quality and maintainability remains high.
Leading by example in code quality.
Working with operations teams to ensure operational readiness.
Advising customers and managers on the estimated effort and technical implications of user stories and user journeys.
Coaching and mentoring team members.
Making sound, reasoned decisions in a fast-paced environment.
Learning about new technologies and approaches.
Supporting colleagues and more junior colleagues, providing direction and support as challenging problems are solved together.