Software Engineer (Ruby)
About The Position
Turing is seeking experienced software engineers at a tech lead level who are familiar with high-quality public GitHub repositories to help build LLM evaluation and training datasets. The project creates verifiable software engineering tasks based on public repository histories using a synthetic, human-in-the-loop approach, while expanding coverage across programming languages and difficulty levels. The role involves hands-on software engineering, including development environment automation, issue triaging, and evaluating test coverage and quality. Engineers will work with researchers to identify repositories and issues that challenge LLMs, and may have opportunities to lead junior engineers. The position is fully remote and offered as a contractor assignment with no medical or paid leave. Work commitments are available at 20, 30, or 40 hours per week, with at least four hours per day and four hours of overlap with PST required.
Requirements
- At least 3 years of overall experience.
- Strong experience with Ruby or at least one of the listed programming languages; Ruby is the language specified in this posting.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfort running, modifying, and testing real-world projects locally.
- Commitment of at least 4 hours per day and 20 hours per week, with 4 hours of overlap with PST.
Nice To Haves
- Experience contributing to or evaluating open-source projects.
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.
Responsibilities
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Potentially lead a team of junior engineers collaborating on projects.
Benefits
- Fully remote work environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Career Level
Senior
Education Level
No Education Listed
