Software Engineer (Ruby)

Turing

1d•Remote

About The Position

Turing is building LLM evaluation and training datasets for realistic software engineering problems. This project creates verifiable software engineering tasks from public repository histories using a synthetic, human-in-the-loop approach, while expanding coverage across programming languages and difficulty levels. The role is for experienced software engineers at a tech-lead level who are familiar with high-quality public GitHub repositories. It involves hands-on engineering work, including development environment automation, issue triage, and evaluating test coverage and quality. The engineer will assess how LLMs perform on real code and collaborate with researchers on challenging repositories and issues. There may also be opportunities to lead a team of junior engineers. This is a fully remote contractor assignment. The available time commitments are 20, 30, or 40 hours per week, with at least four hours per day and four hours of overlap with PST. The assignment does not include medical benefits or paid leave.

Requirements

At least 3 years of overall experience.
Strong experience with Ruby.
Proficiency with Git, Docker, and basic software pipeline setup.
Ability to understand and navigate complex codebases.
Comfort running, modifying, and testing real-world projects locally.
Commitment of at least 4 hours per day and at least 20 hours per week, with 4 hours of overlap with PST.

Nice To Haves

Experience contributing to or evaluating open-source projects.
Previous participation in LLM research or evaluation projects.
Experience building or testing developer tools or automation agents.

Responsibilities

Analyze and triage GitHub issues across trending open-source libraries.
Set up and configure code repositories, including Dockerization and environment setup.
Evaluate unit test coverage and quality.
Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
Potentially lead a team of junior engineers working on projects.