Lead Cloud Data and AI/ML Engineer, AVP

State Street•Quincy, MA

22h

About The Position

Lead workstreams end to end and deliver business outcomes. Researching, learning, and applying new tools and techniques rapidly and suggesting new concepts to improve performance. Explore various AWS services and perform proof of concepts to determine the usage within the applications. Document the best practices and strategies associated with application deployment and infrastructure support. Produce reusable, efficient, and scalable programs, and cost-effective migration strategies. Develop Data Engineering and Machine Learning pipelines in Databricks and use different AWS services, including S3, EC2, API, RDS, Kinesis/Kafka, OpenSearch and Lambda. Work jointly with the IT team and other departments to migrate data engineering and Machine Learning applications to Databricks/AWS and support model life cycle management. Design and develop GenAI and Agentic AI based applications. Evaluate and onboard new AI tools, frameworks, and cloud services to enhance platform capabilities. Performing data and system analysis. Comfortable working on tight timelines, when required. Learn and work on vendor products. Effectively communicate technical concepts and project updates to stakeholders, ensuring alignment between business goals and technical solutions. Supporting implementation of the Software Development Life Cycle, in agile and/or Kanban approaches.

Requirements

Bachelor's degree in computer science, Information Systems, or equivalent education or work experience.
Around 5+ years of experience as a Data Engineer with cloud technologies.
Solid understanding of Databricks fundamentals/architecture and have hands on experience in Databricks modules (Data Engineering, Machine Learning and SQL warehouse).
Solid Knowledge of medallion architecture, DLT and unity catalog within Databricks.
Knowledge of Machine learning model development process.
Understanding of core AWS services, uses, and AWS architecture best practices
Hands-on experience in different domains, like database architecture, business intelligence, machine learning, advanced analytics, big data, etc.
Knowledge of Agentic AI frameworks.
Solid knowledge of Airflow.
Solid knowledge of CI/CD pipelines in AWS technologies.
Application migration of RDBMS, java/python applications, model code, elastic etc.
Solid programming background on scala, python and/or Java.
Effective communication skills to interact with various stakeholders and management.

Nice To Haves

Any AWS and/or Databricks certification will be a plus.
Experience with Docker and Kubernetes is a plus.
Keen to learn new technology and enjoy hands-on development.

Responsibilities

Lead workstreams end to end and deliver business outcomes.
Researching, learning, and applying new tools and techniques rapidly and suggesting new concepts to improve performance.
Explore various AWS services and perform proof of concepts to determine the usage within the applications.
Document the best practices and strategies associated with application deployment and infrastructure support.
Produce reusable, efficient, and scalable programs, and cost-effective migration strategies.
Develop Data Engineering and Machine Learning pipelines in Databricks and use different AWS services, including S3, EC2, API, RDS, Kinesis/Kafka, OpenSearch and Lambda.
Work jointly with the IT team and other departments to migrate data engineering and Machine Learning applications to Databricks/AWS and support model life cycle management.
Design and develop GenAI and Agentic AI based applications.
Evaluate and onboard new AI tools, frameworks, and cloud services to enhance platform capabilities.
Performing data and system analysis.
Comfortable working on tight timelines, when required.
Learn and work on vendor products.
Effectively communicate technical concepts and project updates to stakeholders, ensuring alignment between business goals and technical solutions.
Supporting implementation of the Software Development Life Cycle, in agile and/or Kanban approaches.

Benefits

our retirement savings plan (401K) with company match
insurance coverage including basic life, medical, dental, vision, long-term disability, and other optional additional coverages
paid-time off including vacation, sick leave, short term disability, and family care responsibilities
access to our Employee Assistance Program
incentive compensation including eligibility for annual performance-based awards (excluding certain sales roles subject to sales incentive plans)
eligibility for certain tax advantaged savings plans
inclusive development opportunities
flexible work-life support
paid volunteer days
vibrant employee networks