Senior SW Engineer, Data & Optmization

Roberts Recruiting•Boston, MA

About The Position

We are looking for a Senior Software Engineer with a real passion for building highly-scalable, distributed Big Data applications based on the Lambda Architecture. We want people who take ownership of a project. As part of engineering, you’ll work on an agile team designing and building a Forecasting and Optimization platform capable of: Processing 2 Billion Bid requests a day, 1.5TB of data processed daily, Running in a distributed environment with 500+ servers across a hybrid cloud environment, Data processing in AWS, 200k QPS at low latency. The optimizer and forecaster take in mathematical models in order to make as well as predict the outcome of "fuzzy" decisions related to inventory management, bidding, and performance of advertising campaigns.

Requirements

Minimum 5 years software engineering experience
Experience with large scale distributed systems and big data processing
Familiarity with real time data streaming technologies like Spark, Kafka etc
Strong verbal and communication skills
BS, MS, or equivalent experience
Experience with GIT version control

Nice To Haves

Scala or Java
SBT, Akka
Kafka, Zookeeper, Spark, Hadoop, EMR
Solr, Cassandra, ElasticSearch
Ansible/Puppet, DataDog

Responsibilities

Hands-on programming capabilities in Java/Scala, Spark, Hadoop etc.
Work with a team of Data Scientists in revamping our Forecaster and Optimization systems feeding directly into the bidder
Design/architect and implement new features and systems with a focus on high quality well tested code
Collaborate with other team members, participating in architecture discussions and code reviews.
Provide production support, monitoring the performance of our Forecaster and Optimization systems and troubleshoot/fix issues that arise with deployed software