Data Engineer III
About The Position
The Data Engineer III will be responsible for identifying business problems and formulating technical solutions using analytical methodologies. This role involves understanding use cases, desired outcomes, and applying expertise to develop and improve action plans. The engineer will support the development of business cases, drive project delivery, and assist in process updates and changes. Key responsibilities include identifying suitable data sources, performing initial data quality checks, extracting and transforming data using appropriate techniques, and developing knowledge of current data science and analytics trends. The role also involves analyzing complex data elements, designing logical and physical data models (including data warehouse and data mart designs), defining relational tables, and evaluating existing data models. Additionally, the engineer will develop efficient data flows, analyze system integration challenges, create training documentation, and train end-users. They will also oversee the tasks of less experienced programmers and provide system troubleshooting support. The position requires writing code to develop solutions, creating test cases, developing proofs of concept, testing code, and deploying software to production servers. Contributions to code documentation, playbook maintenance, and timely progress updates are expected. The role operates within Agile/Scrum frameworks, ensuring iterative delivery and alignment with sprint goals, while implementing robust data security and governance standards.
Requirements
- Translating business & product requirements into technical data specifications & semantic definitions
- Analyzing upstream data using Java/Python/SQL, collaborating with upstream teams to understand transformations, dependencies & SLAs
- Automating cloud environment setup and maintenance using scripting languages and CI/CD frameworks to ensure consistent system configurations using Terraform or CDK
- Architecting optimized data structures and reusable server-side logic using structured query techniques to facilitate efficient data retrieval and modeling using BigQuery or ADF or Redshift
- Implementing database schema enhancements and attribute validations, ensuring backward-compatible system updates through standardized version control protocols using BigQuery or ADF or CodeCommit
- Managing enterprise-level analytical reporting and metric definitions to ensure data consistency and accuracy in key performance indicator (KPI) tracking and executive decision-making tools using Dynatrace, Power Bi or QuickSight
- Performing database performance tuning and query optimization to improve system efficiency, reduce operational costs, and accelerate data processing times
- Investigating & resolving data quality, reconciliation, and mismatch issues through debugging
- Ensuring data reliability & SLA adherence by validating outputs, monitoring refresh behavior, and continuously improving data models
- Driving production stability by validating releases across Dev, Pre-prod, and Prod environments through rigorous testing & peer reviews
- Collaborating within Agile/Scrum frameworks to ensure iterative delivery & alignment with sprint goals
- Implementing robust data security and governance standards across all cloud data assets.
Responsibilities
- Identify possible options to address business problems within one's discipline through relevant analytical methodologies.
- Demonstrate understanding of use cases and desired outcomes.
- Apply expertise to the development, execution, and improvement of action plans by providing expert advice and guidance.
- Support the development of business cases and recommendations.
- Drive delivery of project activity and tasks assigned by others.
- Support process updates and changes.
- Support, under guidance, in solving business issues.
- Identify the most suitable source for data that is fit for purpose.
- Perform initial data quality checks on extracted data.
- Extract data from identified databases.
- Create data pipelines and transform data to a structure that is relevant to the problem by selecting appropriate techniques.
- Develop knowledge of current data science and analytics trends.
- Analyze complex data elements, systems, data flows, dependencies, and relationships to contribute to conceptual, physical, and logical data models.
- Develop the Logical Data Model and Physical Data Models including data warehouse and data mart designs.
- Define relational tables, primary and foreign keys, and stored procedures to create a data model structure.
- Evaluate existing data models and physical databases for variances and discrepancies.
- Develop efficient data flows.
- Analyze data-related system integration challenges and propose appropriate solutions.
- Create training documentation and train end-users on data modeling.
- Oversee the tasks of less experienced programmers and stipulate system troubleshooting supports.
- Write code to develop the required solution and application features by determining the appropriate programming language and leveraging business, technical, and data requirements.
- Create test cases to review and validate the proposed solution design.
- Create proofs of concept.
- Test the code using the appropriate testing approach.
- Deploy software to production servers.
- Contribute code documentation, maintain playbooks, and provide timely progress updates.
- Collaborate within Agile/Scrum frameworks to ensure iterative delivery & alignment with sprint goals.
- Implement robust data security and governance standards across all cloud data assets.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
