Senior Data Engineer - Data Platform
Algolia
·
Posted:
August 1, 2023
·
Remote
About the position
We are seeking a Senior Data Engineer to join our Data team at Algolia. As part of this team, you will play a crucial role in making Algolia a data-driven organization by providing relevant and consistent data to help other teams make informed decisions. Your responsibilities will include developing data pipelines, improving and maintaining our data warehouse, and supporting the Analytics Engineering team in building data models. Additionally, you will have the opportunity to mentor junior engineers and contribute to the continuous improvement of our data-driven approach. This position offers flexible remote work options.
Responsibilities
- Developing data pipelines and ETL workflows mainly with Python, Airflow (AWS MWAA), Spark (AWS Glue, EMR), other AWS services managed with Terraform.
- Improving and maintaining the data warehouse (AWS Redshift).
- Supporting the Analytics Engineering team to build data models with dbt to be used by Analysts on business reports.
- Interacting with Engineering and Business teams to understand requirements.
- Designing, building, enriching, and scaling up data pipelines.
- Working with engineers, data analysts, and business analysts to capture and model data.
- Monitoring data integrity and growth.
- Helping improve the data lake and data warehouse architecture to increase performance, simplicity, and autonomy of the users.
- Helping analysts industrialize reports and dashboards to improve company productivity.
- Ensuring every product is released with a data-driven approach.
Requirements
- Strong experience designing and building data pipelines
- Experience orchestrating pipelines with Airflow
- Professional knowledge of Python
- Strong experience working with cloud platforms and architecting them
- Strong experience with data warehouses
- Experience about infrastructure topics specific to data engineering
- Interest in Big Data challenges (~600Tb of data, +14Tb per week, ~700M files, +1,2M per week, ~300 dbt models, ~70 Airflow DAGs)
- Interest in understanding the data and business requirements
- An authorization to work in the EU
- Excellent spoken and written English skills
- Humble, curious, proactive, and strike a balance between creativity, resourcefulness, and pragmatism
- Experience using and managing AWS Redshift (nice to have)
- Experience with a parallel data processing framework such as Apache Spark (nice to have)
- Experience analyzing data quality using dbt (nice to have)
- Experience at our current stage and beyond ($50-200M ARR range, high growth, lots of change and building internal infrastructure) (nice to have)
Benefits
- Flexible remote work options
- Opportunity to work with a data-driven organization
- Significant impact on decision-making processes
- Collaboration with various teams (Operations, Marketing, Product, Infrastructure)
- Opportunity to work with new technologies and make decisions in that direction
- Mentorship opportunities for junior engineers
- Tackling data-related challenges
- Designing, building, and scaling data pipelines
- Working with engineers and analysts to capture and model data
- Monitoring data integrity and growth
- Improving data lake and data warehouse architecture
- Industrializing reports and dashboards to improve company productivity
- Releasing products with a data-driven approach
- Opportunity to work in a high-trust environment with autonomy over work location
- Global presence with physical offices in multiple locations
- Being part of a pioneer and market-leading company
- Opportunity to work with AI-powered search and discovery platform
- Investment in market-leading platform to better serve customers
- Inclusive and diverse workplace environment