Hadoop and PySpark Data Engineer

Infosys

1d•$82,419 - $126,600

About The Position

The Hadoop and PySpark Data Engineer works in the assigned job role of Technology Consultant 2 within Infosys Financial Services. The role contributes to digital transformation projects for financial institutions, supporting software and data solutions across banking, wealth management, and insurance. The position participates across the software development lifecycle, from requirements elicitation and application design through coding, integration, testing, deployment support, and production issue resolution. It documents requirements, design decisions, code, testing techniques, and support activities; helps maintain application stability and quality; and develops training, documentation, and support frameworks for new software applications. The work also involves collaboration with cross-functional teams and software integration.

Requirements

Experience with data warehousing architectural approaches.
Sound understanding and experience with the Hadoop ecosystem, including Cloudera.
Experience working with a Big Data implementation in a production environment.
Experience with Big Data technologies such as Hadoop, Hive, Spark, and Python.
Experience with Python and Unix shell scripting.
Experience with an orchestration tool such as Autosys or Airflow.
Sound knowledge of relational databases (SQL) and experience with large SQL-based systems.
Understanding of Agile methodologies and technologies.
Bachelor’s degree or foreign equivalent from an accredited institution, or three years of progressive experience in the specialty in lieu of each year of education.
Applicants must be authorized to work in the United States.

Nice To Haves

Ability to understand and explore evolving tools in the Hadoop ecosystem and apply them to relevant problems.
Knowledge and experience with cloud and containerization technologies.
Experience with data visualization tools such as Tableau.

Responsibilities

Document assigned portions of business requirements as part of requirements elicitation, following provided guidance.
Facilitate software application design discussions and document design decisions for the technical team.
Participate in coding and integrate new features or updates into existing applications while maintaining system stability.
Conduct code reviews, make codebase changes, and maintain code repositories.
Implement test strategies, analyze results, and coordinate bug fixes to maintain software quality.
Develop user training programs, documentation, and support frameworks for transitions to new software applications.
Participate in resolving production issues and recommend preventive strategies to improve system reliability.
Maintain records of code, testing techniques, and support activities for the knowledge base and similar projects.
Work across end-to-end SDLC phases, translating requirements into development tasks and collaborating with cross-functional teams on software integration.