Data Engineer - Remote US
Seamless.AI
·
Posted:
July 31, 2023
·
Remote
About the position
We are seeking an experienced Data Engineer with a minimum of 3 years of experience in building data ingestion pipelines for large datasets. As a key player in our team, you will be responsible for designing, building, and managing our data infrastructure, supporting our data-driven decision-making capability. You'll need to be proficient in Python, AWS, and common frameworks used for data ingestion, transformation, and consolidation. The ideal candidate will be passionate about data, a strong team player, and have a continuous learning mentality.
Responsibilities
- Design, develop and optimize data ingestion pipelines for real-time and batch data streams using various sources.
- Utilize extensive knowledge of Python and AWS to engineer solutions for transforming, consolidating, and storing large datasets.
- Collaborate with data scientists and stakeholders to understand data needs and translate them into data systems and pipelines.
- Enhance the data ecosystem by implementing industry best practices for testing, deployment, and runtime environments.
- Drive continuous improvements in data reliability, efficiency, and quality.
- Document data architectures, procedures, and data flows, maintaining excellent communication with the team and stakeholders.
- Monitor data systems performance, troubleshoot data issues, perform root cause analysis, and implement optimal solutions.
- Participate in data governance and ensure adherence to data security and privacy standards.
- Minimum of 3 years of experience as a Data Engineer or in a similar role.
- Strong proficiency in Python, AWS, and common frameworks used in data ingestion and transformation.
- Hands-on experience in building and optimizing large-scale data pipelines, architectures, and datasets.
- Knowledge of data warehousing concepts.
Requirements
- Minimum of 3 years of experience as a Data Engineer or in a similar role
- Proficiency in Python, AWS, and common frameworks used in data ingestion and transformation
- Experience in building data ingestion pipelines for large datasets
- Ability to design, develop, and optimize data ingestion pipelines for real-time and batch data streams
- Knowledge of data warehousing concepts
- Collaboration skills to work with data scientists and stakeholders to understand data needs and translate them into data systems and pipelines
- Familiarity with industry best practices for testing, deployment, and runtime environments in data ecosystems
- Strong problem-solving skills to troubleshoot data issues and perform root cause analysis
- Documentation skills to create data architectures, procedures, and data flows
- Understanding of data governance and adherence to data security and privacy standards
Benefits
- Competitive salary and compensation package
- Opportunity to work with large datasets and build data ingestion pipelines
- Chance to contribute to data-driven decision-making capability
- Proficiency in Python, AWS, and common frameworks for data ingestion, transformation, and consolidation
- Passionate and collaborative team environment
- Continuous learning and professional growth opportunities
- Positive culture and work environment
- Impactful work that contributes to the growth of the company
- Design and optimize data ecosystems using industry best practices
- Documentation of data architectures and procedures
- Monitoring and troubleshooting of data systems
- Adherence to data security and privacy standards
- Potential for working with other programming languages and big data tools (Java, Scala, Hadoop, Spark)