Senior Data/AI Engineering
About The Position
Responsible for the development of high performance, distributed computing systems using Big Data technologies for GenAI development. Build scalable multi-threaded Spark clusters using Databricks interfacing with NoSQL, including data mining using various distributed technologies on the Azure Cloud platform. Design and deliver GenAI technologies at scale with real time results using OpenAI and large language models to service chat experience for business partners across the company. Engage with clients to understand business requirements and work with teams to identify architecture needed to deliver necessary features. Deliver insights and improvements in GenAI results and influence development decisions for optimal software delivery. Build auto LLM to automate discovery of the best configuration to be identified in leaderboard results using RAG models. Build truth set auto generator to optimize leaderboard ranking based on standardized questions and answers. Integrate various calculation and models for GenAI truth set results to rank best performing results. Integrate Lang Chain loaders to embed model packages to be able to process all types of data. Use LLM large language models for optimized data retrieval methodology and further augment and optimize results using RAG Use Vector DB to manage chunks and improve configurations in optimized VectorDB management. Process data from Snowflake, ServiceNow for GenAI integration. Document all GenAI work and train team on new capabilities. Build Python Fast API rest services to support GenAI integration across systems. Implement WebSocket ends to enable iPhone app for voice streaming GenAI experience. Utilize Big data technologies, Hadoop, NoSQL and text mining.
Requirements
- Bachelor’s degree, or foreign equivalent degree in Computer Science or Computer Engineering
- 1 year of experience in the job offered, or 1 year of experience in a related occupation building scalable multi-threaded Spark clusters using Databricks interfacing with NoSQL, including data mining using various distributed technologies on the Azure Cloud platform
- 1 year of experience designing and delivering GenAI technologies at scale with real time results using OpenAI and large language models to service chat experience for business partners across the company
- 1 year of experience building Python Fast API rest services
- 1 year of experience implementing WebSocket ends to enable iPhone app for voice streaming GenAI experience
- 1 year of experience utilizing Big data technologies, Hadoop, NoSQL and text mining
Responsibilities
- Development of high performance, distributed computing systems using Big Data technologies for GenAI development.
- Build scalable multi-threaded Spark clusters using Databricks interfacing with NoSQL, including data mining using various distributed technologies on the Azure Cloud platform.
- Design and deliver GenAI technologies at scale with real time results using OpenAI and large language models to service chat experience for business partners across the company.
- Engage with clients to understand business requirements and work with teams to identify architecture needed to deliver necessary features.
- Deliver insights and improvements in GenAI results and influence development decisions for optimal software delivery.
- Build auto LLM to automate discovery of the best configuration to be identified in leaderboard results using RAG models.
- Build truth set auto generator to optimize leaderboard ranking based on standardized questions and answers.
- Integrate various calculation and models for GenAI truth set results to rank best performing results.
- Integrate Lang Chain loaders to embed model packages to be able to process all types of data.
- Use LLM large language models for optimized data retrieval methodology and further augment and optimize results using RAG.
- Use Vector DB to manage chunks and improve configurations in optimized VectorDB management.
- Process data from Snowflake, ServiceNow for GenAI integration.
- Document all GenAI work and train team on new capabilities.
- Build Python Fast API rest services to support GenAI integration across systems.
- Implement WebSocket ends to enable iPhone app for voice streaming GenAI experience.
- Utilize Big data technologies, Hadoop, NoSQL and text mining.
Benefits
- Medical/Dental/Vision coverage
- 401(k) plan
- Tuition reimbursement program
- Paid Time Off and Holidays (based on date of hire, at least 23 days of vacation each year and 9 company-designated holidays)
- Paid Parental Leave
- Paid Caregiver Leave
- Additional sick leave beyond what state and local law require may be available but is unprotected
- Adoption Reimbursement
- Disability Benefits (short term and long term)
- Life and Accidental Death Insurance
- Supplemental benefit programs: critical illness/accident hospital indemnity/group legal
- Employee Assistance Programs (EAP)
- Extensive employee wellness programs
- Employee discounts up to 50% off on eligible AT&T mobility plans and accessories, AT&T internet (and fiber where available) and AT&T phone
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
