Software Engineer (Ruby)
About The Position
Turing is building LLM evaluation and training datasets to help train LLMs on realistic software engineering problems. This project creates verifiable software engineering tasks from public repository histories using a synthetic, human-in-the-loop approach, while expanding coverage across programming languages and difficulty levels. The role is for experienced software engineers at a tech-lead level who know high-quality public GitHub repositories. It involves hands-on engineering work, including development environment automation, issue triage, and evaluating test coverage and quality. The engineer will help assess how LLMs interact with real code and contribute to AI-assisted software development research. There may also be opportunities to lead junior engineers. This is a fully remote contractor assignment. The required commitment is at least four hours per day and 20 hours per week, with four hours of overlap with PST. Available commitment options are 20, 30, or 40 hours per week. The assignment does not include medical benefits or paid leave.
Requirements
- At least 3 years of overall experience.
- Strong experience with Ruby or another listed language; Ruby is the language specified in the title and posting.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfort running, modifying, and testing real-world projects locally.
- Commit to at least 4 hours per day and at least 20 hours per week, with 4 hours of overlap with PST.
Nice To Haves
- Experience contributing to or evaluating open-source projects.
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.
Responsibilities
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and development environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Potentially lead a team of junior engineers working on projects.
Benefits
- Fully remote work environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Career Level
Senior
Education Level
No Education Listed
