Lead Backend Software Engineer, Digital Assembly Line

Avala • San Francisco, CA

10d • $100,000 - $165,000

About The Position

Avala is a mission-driven company that operates as a two-sided marketplace at the intersection of the labor economy and the AI revolution. At this intersection lies an incredible opportunity to redefine the future of labor while ensuring we build AI systems that are highly aligned with human values and intent. Our platform works towards this alignment by enabling the continuous integration of human feedback and human input at scale. Unlike traditional crowdsourcing or data annotation, Avala dynamically orchestrates digital assembly lines with the precision and unit economics of a Gigafactory. This allows us to rapidly scale to meet the demands of complex data labeling workflows and high-volume content moderation tasks to a global audience. We understand that the development of AI has the potential to bring about significant social and economic changes, and we're committed to ensuring that these changes are positive and inclusive. Avala is designed to empower individuals and communities on every corner of the Earth with a platform to participate in the curation and moderation of AI systems. As a lead member of the Avala Engineering team, you will design and implement a diverse set of tools that orchestrate Avala’s Digital Assembly Line. Supporting our AI-assisted human labeling and various machine learning workflows. The Digital Assembly Line is a central part of Avala. The systems you build will have a large impact on the entire lifecycle of Avala’s full-stack ML Ops platform. This includes data curation, processing, data discovery, annotation, and visualization. This also includes tools that help us automate the entire workflow of training, validation, and deploying customer models to production. As a backend engineer, you bring top-notch software engineering skills and can contribute to our tooling systems immediately. Knowledge of machine learning, computer vision, or neural networks is a plus but not required. That can be learned on the job. A strong candidate for the team will either be an excellent software generalist or someone who is exceptionally strong in backend engineering.

Requirements

• BS in Computer Science, Physics, Computer Engineering, Electrical Engineering or proof of exceptional skills in related fields or practical software engineering experience.
• Strong knowledge of at least one programming language related to data engineering. Python is required.
• Solid back-end experience in building RESTful Python applications.
• Knowledge of various Python backend frameworks (e.g.Flask, Django, Tornado). Django is preferred.
• Experience with SQL, both relational and non-relational databases (SQL Server, MySQL, Redis)
• Experience working with Docker and container orchestration technologies such as Kubernetes
• You understand how to create an efficient, reliable, scalable, and maintainable software architecture.
• Hands on experience with CI/CD pipelines to deploy fast and often
• Experienced in end-to-end development processes, including unit, integration & functional testing.
• Good understanding of logging, monitoring and alerting system.
• Experience with cloud computing such AWS EC2/S3/RDS is a plus.
• Experience with ElasticSearch or other scalable search systems is a plus.
• Experience with real-time data processing is a plus.
• Exceptional communication skills, both oral and written.

Nice To Haves

• Experience with AWS services such as EC2/S3/RDS/SQS.
• Experience with big data systems such as Spark, Hadoop/MapReduce
• Experience with scientific computing libraries such as numpy, pandas, or scikit-learn
• In-depth knowledge of container orchestrators and cluster management software.
• Experience with data stores and indexers like PostgreSQL, ElasticSearch, Redis.

Responsibilities

• Take ownership of the architecture, design, development, deployment, and operations of the backend API stack you will develop, using software engineering best practices, and other cutting-edge methodologies
• Design and implement large-scale data processing pipelines that handle a diverse set of Avala-related data such as images, videos, text, audio, neural networks, and human annotations.
• Design and implement tools, tests, metrics, and dashboards to accelerate the development cycle of our model training.
• Work closely with frontend engineers to seamlessly integrate with backend systems.
• Influence architectural decisions with focus on security, scalability, reliability and high performance.
• Setup and maintain monitoring, metrics & reporting systems for fine-grained observability and actionable alerting.
• Collaborate with colleagues in design and code reviews.
• Develop means for deploying, scaling, and monitoring software systems.
• Characterize complex problems related to the scalability, reliability, performance, and security of the software.
• Own projects and always seeks ways to improve efficiency and operations.
• Contribute relevant, clean, concise quality documentation to Avala’s knowledge base to support/increase organizational information sharing.
• Evangelize best practices in Software development and Software development lifecycle.

Benefits

• Along with competitive pay and generous stock, as a full-time Avala employee, you are eligible for the following benefits at day 1 of hire:
• An opportunity to change the world for the better, engineering solutions to solve problems that impact the most underserved communities in our world, while innovating at the bleeding edge of AI
• Growth potential. We rapidly advance team members who have an outsized impact
• Excellent medical, dental, and vision insurance
• Flexible parental leave
• Flexible time off + paid holidays
• Generous Equity + 401(k) plan

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level