Data Engineer

Skillerszone•Los Angeles, CA

About The Position

Optiboostmedia is a leading provider of affiliate marketing and recruitment software solutions. We help businesses optimize their marketing and recruitment processes through innovative, user-friendly technology. We are seeking a talented Data Engineer to join our team and build the infrastructure to support the collection, storage, and processing of large data sets, ensuring that we have reliable access to the data we need to drive decisions and innovation. The Data Engineer will be responsible for designing, building, and maintaining data pipelines that ensure the efficient collection, storage, and processing of data across different systems. You will work closely with data scientists, analysts, and other stakeholders to ensure that data is accessible, accurate, and reliable for business analysis and decision-making.

Requirements

Proven experience as a Data Engineer, Data Analyst, or in a similar data-related role.
Strong knowledge of data engineering principles, data modeling, and data architecture.
Proficiency with databases (e.g., SQL, NoSQL) and data processing frameworks (e.g., Apache Spark, Hadoop).
Experience with ETL tools and processes, including data integration and transformation.
Familiarity with cloud platforms and data storage solutions (e.g., AWS, Azure, Google Cloud).
Knowledge of programming languages such as Python, Java, or Scala for data processing and scripting.
Experience with data warehousing technologies (e.g., Redshift, BigQuery, Snowflake) and tools for building data pipelines.
Strong analytical and problem-solving skills with a focus on optimizing data processes.
Ability to work in a fast-paced environment and collaborate with cross-functional teams.
Bachelors degree in Computer Science, Engineering, or a related field; advanced degrees or certifications are a plus.

Responsibilities

Design, build, and maintain scalable and efficient data pipelines that handle large volumes of structured and unstructured data.
Integrate data from various internal and external sources, ensuring seamless data flow between systems, databases, and storage solutions.
Develop and manage data warehouses and data lakes, ensuring that data is properly organized, stored, and optimized for analysis.
Implement and optimize ETL (Extract, Transform, Load) processes to ensure data is transformed and stored correctly for downstream analysis.
Manage and optimize databases (e.g., SQL, NoSQL) to ensure high performance, availability, and scalability.
Monitor data pipelines to identify and resolve issues, ensuring high data quality, integrity, and reliability.
Work closely with data scientists, analysts, and business teams to understand data requirements, provide data access, and ensure alignment with business goals.
Implement automation strategies to reduce manual intervention and optimize data processing workflows.
Optimize and fine-tune data processes and queries to enhance performance and reduce latency.
Document data pipeline processes, workflows, and best practices, ensuring clarity for internal teams and future development.