Senior Data Platform Engineer
About The Position
The Senior Data Platform Engineer of the Shared Data Platform plays a pivotal role in designing and developing innovative services to enhance Arrowstreet Capital's data infrastructure. The primary objective is to streamline the ingestion and transformation of new data sources, enabling rapid delivery of actionable analytics across the firm. You will collaborate with stakeholders from various departments to gather requirements and identify process improvements that optimize data flow. Additionally, you will design flexible abstractions and frameworks, ensuring that platform solutions can be efficiently tailored to meet the diverse needs of internal clients.
Requirements
- Bachelor's degree in Computer Science, Computer Engineering or a related discipline
- At least 5 years of experience working with data lakes, data warehousing, and/or database systems, including processing frameworks like Spark, Hadoop, Presto, etc.
- You strive to write elegant code, and you're comfortable with picking up new technologies independently
- You are fluent in at least one major programming language (e.g. Python, C#)
- Hands-on experience working with AWS. and automation through infrastructure as code with one of the Terraform, Pulumi, Cloudformation and CDK etc.
- Experience designing and deploying APIs and client libraries using tools such as OpenAPI, FastAPI, or similar frameworks.
- You have experience with real-time streaming processing with Kafka, Kinesis, and Apache Flink etc.
Responsibilities
- Define and execute the technical roadmap for the platform that can support batch and real-time data sets using modern data processing frameworks.
- Oversee and contribute to building features for data discovery, lineage, governance, ingestion operation, cost management, performance, etc.
- Lead and assist the team in investigating potential improvements achieved by adopting new data technologies and frameworks.
- Achieve high quality metrics through automated testing, robust operational tooling, and highly-available cloud architecture.
- Production support of the platform to avoid disruption to investment processes.
- Collaborate closely with business stakeholders to understand their data needs, translate business objectives into scalable and efficient data engineering solutions.
- Partner with IT operations and cloud security teams to ensure platform solutions are secure, scalable, and reliable.
- Develop and mentor a junior Data Engineers
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Industry
Securities, Commodity Contracts, and Other Financial Investments and Related Activities
Number of Employees
251-500 employees
