Big Data Engineer

Codetech•Malvern, PA

10h

About The Position

The Big Data Engineer writes ETL processes, designs database systems, and develops tools for real-time and offline analytics. The role troubleshoots software and processes to maintain data consistency and integrity, and integrates data from multiple sources to help business partners generate insights and make decisions. The engineer translates business specifications into design specifications and code; writes programs, ad hoc queries, and reports; and ensures code is structured, documented, maintainable, and reusable. The position partners with internal clients to understand business functions and information needs, develops working knowledge of relevant tools and systems, and participates in all phases of solution development. It also includes explaining technical considerations in meetings, testing code, reviewing completed work with clients, providing data-analysis guidance and user support, testing and implementing software releases, and working with vendors to resolve issues and move software into production. The engineer may also participate in special projects and perform other assigned duties.

Requirements

1–3 years of data engineering experience.
Experience with SQL, Python, and AWS.
Experience with CI/CD and Jira.
Experience with data quality.
Experience setting up data lineage.
Undergraduate degree or equivalent combination of training and experience.

Responsibilities

Write ETL processes, design database systems, and develop tools for real-time and offline analytic processing.
Troubleshoot software and processes to support data consistency and integrity.
Integrate data from a variety of sources for business partners to generate insights and make decisions.
Translate business specifications into design specifications and code.
Write programs, ad hoc queries, and reports.
Ensure code is well structured, sufficiently documented, and easy to maintain and reuse.
Partner with internal clients to understand business functions and information needs.
Develop working knowledge of tools, technologies, applications, databases, and company-wide systems relevant to business areas.
Participate in all phases of solution development.
Explain technical considerations at meetings, including meetings with business clients.
Thoroughly test code for accuracy and intended purpose.
Review end products with clients to ensure adequate understanding.
Provide data-analysis guidance as required.
Provide tool and data support to business users and team members.
Test and implement new software releases through regression testing.
Identify issues and work with vendors to resolve them and elevate software into production.
Participate in special projects and perform other duties as assigned.