Remote Software Engineer (Ruby)
About The Position
Turing is seeking experienced software engineers at a tech-lead level to help build LLM evaluation and training datasets using realistic software engineering problems. The project creates verifiable software engineering tasks from public repository histories, using a synthetic approach with human review, while expanding coverage across programming languages and difficulty levels. This hands-on role focuses on evaluating how LLMs work with real code. Work includes automating development environments, triaging issues, assessing test coverage and quality, and collaborating with researchers to identify challenging repositories and issues. There may also be opportunities to lead junior engineers. The role is fully remote and offered as a contractor assignment with no medical or paid leave. Commitments are available at 20, 30, or 40 hours per week, with at least four hours per day and four hours of overlap with PST required.
Requirements
- At least 3 years of overall experience.
- Strong experience with Ruby.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfort running, modifying, and testing real-world projects locally.
- Availability for at least 4 hours per day and 20 hours per week, with 4 hours of overlap with PST.
Nice To Haves
- Experience contributing to or evaluating open-source projects.
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.
Responsibilities
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories, including Dockerization and development environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.
- Potentially lead a team of junior engineers on projects.
Benefits
- Fully remote work environment.
- Opportunity to work on cutting-edge AI projects with leading LLM companies.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Career Level
Senior
Education Level
No Education Listed
