Lead Big Data Engineer - PySpark

Logic20/20 Inc. • Seattle, WA

2h • $156,348 - $194,040

About The Position

As a Lead Developer joining our AI & Analytics practice, you'll be responsible for delivering client value and ensuring high client satisfaction. You'll be expected to be adept at recognizing, subscribing, and applying best practices, methodologies, tools, and techniques to meet client requirements, timelines, and budgets.

Requirements

• 5-10+ years of data engineering experience using Python and Spark (PySpark).
• 2+ years of experience leading the design and development of cloud ETL and data pipelines.
• Experience working with fast streaming data.
• Experience with Flink, Kafka, and Kubernetes.
• Experience developing CI/CD pipelines.
• Experience with Big Data Technologies (Hadoop, Spark/PySpark, MongoDB).
• Experience working with Azure, AWS, and/or GCP.
• Demonstrated ability to identify business and technical impacts of user requirements and incorporate them into the project schedule.
• Ability to travel at least once per quarter.

Nice To Haves

• Extensive experience working with Palantir Foundry.
• Java experience is highly preferred.
• Consulting experience and/or experience working within the Utilities industry is highly preferred.

Responsibilities

• Perform data assessment, analysis, and data model design.
• Develop CI/CD pipelines and other DataOps fundamentals.
• Communicate at the appropriate level to both business and technology teams to understand business needs and pain points.
• Be creative in meeting the client’s core needs with their technology.
• Explain technical benefits and deficits to non-technical audiences.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed