Data Engineer
About The Position
Sterling is seeking a highly motivated and skilled Data Engineer to join our growing team. In this role, you will be instrumental in building, maintaining, and optimizing our data infrastructure, enabling data-driven decision-making across the organization. If you're passionate about working with cutting-edge technologies, thriving in an agile environment, and collaborating closely with other technical experts, we want to hear from you! As a Data Engineer, you will be a key player in our data ecosystem, responsible for the design, development, and maintenance of robust and scalable data pipelines. You will work with a variety of data sources, ensuring data quality, accessibility, and reliability. This role requires a blend of strong technical skills, an eagerness to learn, and communication abilities to gather requirements and collaborate effectively.
Requirements
- Databricks Proficiency: Proven experience with Databricks, including Spark, Delta Lake, and related services, for large-scale data processing.
- Python Programming: Strong proficiency in Python for data manipulation, scripting, and pipeline development.
- SQL Mastery: Expert-level SQL skills with experience in optimizing complex queries.
- Agile Methodologies: Solid understanding and practical experience with Agile Scrum development processes and tools like Jira.
- Collaboration & Communication: Excellent interpersonal skills with a demonstrated ability to work effectively in a team environment, communicate technical concepts to non-technical audiences, and gather precise requirements.
- Ownership & Accountability: A strong sense of ownership over data pipelines and data quality, with a proactive approach to problem-solving and maintenance.
- Adaptability & Excitement: Eagerness to learn new technologies, adapt to evolving challenges, and contribute to a fast-paced, innovative data team.
Responsibilities
- Data Pipeline Ownership: Design, build, deploy, and maintain resilient and efficient data pipelines using Databricks and Python, ensuring data is accurately and timely delivered to various stakeholders.
- Greenfield Development: Lead and contribute to the design and implementation of new data solutions and platforms from inception, bringing innovative approaches to our data architecture.
- Data Platform Modernization: Actively participate in the modernization of our existing data platforms, enhancing their scalability, efficiency, and capabilities.
- Data Curation: Curate, prepare, and provide high-quality datasets for consumption by data analytics teams, data scientists, and machine learning engineers, ensuring data readiness for various analytical and modeling needs with high visibility.
- Data Catalog Management: Take ownership of our data catalog, ensuring it is comprehensive, accurate, and up-to-date, facilitating data discovery and governance.
- SQL Expertise: Develop complex SQL queries for data extraction, transformation, and loading (ETL) processes, and optimize existing queries for performance.
- Requirement Gathering & Collaboration: Proactively engage with cross-functional teams to understand data needs, gather detailed requirements, and translate them into technical solutions.
- Agile Development: Participate actively in our Agile Scrum methodology, including sprint planning, daily stand-ups, and retrospectives, contributing to continuous improvement.
- Problem Solving & Troubleshooting: Identify, diagnose, and resolve data-related issues, ensuring data integrity and system stability.
- Innovation & Best Practices: Stay abreast of the latest industry trends and technologies in data engineering, advocating for and implementing best practices to keep our data platform at the cutting edge.
Benefits
- A dynamic and collaborative work environment where your contributions make a real impact.
- Opportunities to work with state-of-the-art data technologies and learn continuously.
- A culture that values innovation, continuous improvement, and professional growth.
- Competitive salary and benefits package.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
