Senior Data Engineer - Associate

Morgan Stanley•New York, NY

1d•$90,000 - $150,000

About The Position

6+ years of relevant work experience Strong with programming in Python to perform Batch data engineering on Apache Spark and populate downstream batch data stores (such as data mart) for BI use cases & to generate downstream feeds (ie., flat & wide tables or compressed files) for Data Science use cases Real-time service integration to process business events off Kafka and persist in operational MS SQL/MongoDB and/or Neo4j Graph data stores for fraud investigation Near real-time stream processing to derive features for ML model inference. Strong with SQL Server & Hadoop based implementations Strong SQL & Stored Procedure skills Exposure to modern operational NewSQL or NoSQL database technologies such as Neo4j, MongoDB, MemSQL etc., Good hands-on experience with at least one of the job scheduling tools like Autosys (Preferred), Control-M etc. Experience of working in a Linux environment and can write Python/Shell scripts Strong data architecture and modeling experience. Especially, in dimensional modeling for data mart design and development to support BI use cases Strong data analytics skills Strong oral and written communication skills Excellent interpersonal skills and professional approach Strong analytical and problem-solving skills Ability to learn quickly and pick up new techniques and/or technologies Experience in building & maintaining data solutions for Operational, BI & Data Science use cases Experience with Azure (Databricks, Data Factory, Synapse, Azure Data Lake) and/or AWS (AWS S3, AWS Athena, AWS Glue) Data ecosystem & Snowflake Data Cloud Experience in building virtual data access layer using Data Virtualization technologies to support BI & Data Science use cases Experience of the full software development life cycle Experience of working in an Agile team Experience of working with version control systems Experience with bash scripting Experience of working with Continuous Integration systems Experience in Fraud detection and prevention business in Financial Services We do it in a way that's differentiated - and we've done that for 90 years. Our values - putting clients first, doing the right thing, leading with exceptional ideas, committing to diversity and inclusion, and giving back - aren't just beliefs, they guide the decisions we make every day to do what's best for our clients, communities and more than 80,000 employees in 1,200 offices across 42 countries. Our teams are relentless collaborators and creative thinkers, fueled by their diverse backgrounds and experiences. We are proud to support our employees and their families at every point along their work-life journey, offering some of the most attractive and comprehensive employee benefits and perks in the industry. There's also ample opportunity to move about the business for those who show passion and grit in their work. To learn more about our offices across the globe, please copy and paste https://www.morganstanley.com/about-us/global-offices into your browser. Expected base pay rates for the role will be between $90,000 and $150,000 per year at the commencement of employment. Consequently, our recruiting efforts reflect our desire to attract and retain the best and brightest from all talent pools. We want to be the first choice for prospective employees. It is the policy of the Firm to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, creed, age, sex, sex stereotype, gender, gender identity or expression, transgender, sexual orientation, national origin, citizenship, disability, marital and civil partnership/union status, pregnancy, veteran or military service status, genetic information, or any other characteristic protected by law.

Requirements

6+ years of relevant work experience
Strong with programming in Python to perform Batch data engineering on Apache Spark and populate downstream batch data stores (such as data mart) for BI use cases & to generate downstream feeds (ie., flat & wide tables or compressed files) for Data Science use cases
Real-time service integration to process business events off Kafka and persist in operational MS SQL/MongoDB and/or Neo4j Graph data stores for fraud investigation
Near real-time stream processing to derive features for ML model inference.
Strong with SQL Server & Hadoop based implementations
Strong SQL & Stored Procedure skills
Exposure to modern operational NewSQL or NoSQL database technologies such as Neo4j, MongoDB, MemSQL etc.
Good hands-on experience with at least one of the job scheduling tools like Autosys (Preferred), Control-M etc.
Experience of working in a Linux environment and can write Python/Shell scripts
Strong data architecture and modeling experience. Especially, in dimensional modeling for data mart design and development to support BI use cases
Strong data analytics skills
Strong oral and written communication skills
Excellent interpersonal skills and professional approach
Strong analytical and problem-solving skills
Ability to learn quickly and pick up new techniques and/or technologies
Experience in building & maintaining data solutions for Operational, BI & Data Science use cases
Experience with Azure (Databricks, Data Factory, Synapse, Azure Data Lake) and/or AWS (AWS S3, AWS Athena, AWS Glue) Data ecosystem & Snowflake Data Cloud
Experience in building virtual data access layer using Data Virtualization technologies to support BI & Data Science use cases
Experience of the full software development life cycle
Experience of working in an Agile team
Experience of working with version control systems
Experience with bash scripting
Experience of working with Continuous Integration systems
Experience in Fraud detection and prevention business in Financial Services