Big Data Engineer
About The Position
Developing and supporting scalable, extensible, and highly available data solutions. Deliver on critical business priorities while ensuring alignment with the wider architectural vision Identify and help address potential risks in the data supply chain Follow and contribute to technical standards. Design and develop analytical data models. Considers the business implications of the application of technology to the current business environment; identifies and communicates risks and impacts. Employs developed communication and diplomacy skills to exchange potentially complex/sensitive information. Demonstrates attention to quality and timeliness of service to ensure the effectiveness of the team and group. Provides informal guidance or on-the-job-training to new team members.
Requirements
- 2+ years of hands-on experience in building data pipelines with Apache Spark.
- Strong experience in Big Data platforms including Hadoop, Hive, or Snowflake for data storage and processing.
- Strong proficiency in either Python or Java programming.
- Expertise in data modeling techniques, including the design and structuring of data models.
- Comprehensive understanding of Data Warehousing principles, alongside experience with RDBMS (Oracle, MSSQL, MySQL) and NoSQL databases (MongoDB, DynamoDB).
- Familiarity with DevOps concepts, specifically CI/CD platforms and version control.
- Exposure to data quality management, controls, validation, and enrichment.
- Understanding of containerization technologies like Docker and Kubernetes.
- Experience with various event, file, and table formats, such as Parquet, ORC, and Iceberg.
- Basic knowledge of job schedulers (e.g., Autosys) and entitlement management.
- Bachelor's/University degree or equivalent experience
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Number of Employees
5,001-10,000 employees
