Data Engineer (I)
About The Position
The Data Engineer executes complete, tactical tasks within the scope of a larger project, with guidance from a manager, team drive, or senior colleagues and within bounded time. The role applies standard tools, resources, and processes to solve defined problems, builds working relationships outside the team, and contributes to cross-project collaborations. It requires a foundational understanding of core data and role-related knowledge, relevant technologies, and processes, and translating business needs into technical data solutions. The position builds agentic pipelines for data processing, analysis, and visualization, as well as tools for model analysis and debugging. It cleans, structures, and catalogs raw spatial and research datasets, and coordinates with researchers to provide processed data outputs. The role also coordinates with an India-based team on task expectations, annotation job queues, annotation quality analysis, and feedback to raters; this may involve working odd hours. The Data Engineer supports data visualization and reporting, exploratory data analysis and profiling, and data-driven decision-making by collecting, transforming, and publishing data. With guidance, the role contributes to machine-learning model deployment and training, data infrastructure and frameworks, data models and pipelines, and secure, scalable, reliable data processing systems. It works with users, partners, decision-makers, application engineers, and data scientists to clarify requirements, data sources, validation standards, storage, and ML training needs.
Requirements
- Solid programming skills, including Python experience, particularly building tools, web UIs, and scripts for data collection, annotation, and processing.
- Data science and data engineering skills, including data quality verification, agentic data-processing pipelines, data visualization, and data analysis.
- Experience conducting human studies or surveys.
- Strong communication skills to discuss technical designs with researchers and software engineers and data annotation with non-technical raters.
- Proficiency in code comprehension and programming, information gathering, project management, data exploration, big data infrastructure, machine-learning knowledge, stakeholder management, ETL pipeline design, data modeling, statistics, and BI tools.
Nice To Haves
- Knowledge of VLM evaluation and evaluation of multimodal generative AI models, such as image- or video-generation models.
Responsibilities
- Build agentic pipelines for data processing, analysis, and visualization, along with tools for model analysis and debugging.
- Clean, structure, and catalog raw spatial and research datasets to ensure high data quality.
- Coordinate with researchers to provide processed data outputs.
- Coordinate with the India-based team to communicate task expectations, enqueue and dequeue annotation jobs, analyze annotation quality, and provide feedback to the rating team.
- Create or consult on data visualizations and visualization features using internal BI tools and external tools, with guidance.
- Maintain reports, queries, and dashboards and provide ongoing support to data users, with guidance.
- Perform exploratory data analysis and profiling using relevant tools, custom data infrastructure, or existing data models, with guidance.
- Work with clients to understand needs and clarify requirements; collect, transform, and publish data to enable data-driven decision-making.
- Leverage, deploy, and continuously train pre-existing machine-learning models; learn and implement new storage, MPP, and ML-serving systems, with guidance.
- Implement business solutions and infrastructure to build and scale common frameworks, with guidance.
- Follow local technical best practices for data discoverability, data lifecycle management, and master data management.
- Design, build, operationalize, secure, and monitor data processing systems, emphasizing security and compliance, scalability and efficiency, reliability and fidelity, and flexibility and portability.
- Develop and maintain data models, pipelines, and exchange formats for visualization, analysis, interpretation, and ML training and models, with direct guidance.
- Consult users, partners, and decision-makers to identify data sources, required data elements, and data validation standards, with guidance.
- Consult application engineers about logging and transactional storage, and data scientists about ML training and feature engineering.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Entry Level
Education Level
No Education Listed
