Lead Data Engineering

Cloudious•Charlotte, NC

About The Position

We are seeking a highly skilled Lead Data Engineer to join our dynamic team. The ideal candidate will possess strong hands-on experience in advanced Oracle RDMS, SQL, PL/SQL, and various data engineering tools and technologies. This role requires a deep understanding of data architecture, data modeling, and the ability to work collaboratively in a fast-paced environment. The Lead Data Engineer will be responsible for designing, implementing, and maintaining data pipelines and ensuring data integrity and availability for analytics and reporting.

Requirements

Advanced Oracle RDMS expertise, including object creation (tables, views, indexes, partitioned objects).
Proficient in SQL and PL/SQL for data manipulation and analysis.
Strong experience with K shell scripting, including sed, awk, grep, and invoking SQL Plus shell.
Hands-on experience with NDM (Connect:Direct) for file transfers.
Familiarity with Autosys for job scheduling and monitoring.
Proficient in Python and PySpark for data processing and analytics.
Experience using GitHub as a code repository.
Comfortable using GitHub CoPilot for AI development efficiencies.

Nice To Haves

Experience with data warehousing concepts and technologies.
Knowledge of cloud-based data solutions (e.g., AWS, Azure, Google Cloud).
Familiarity with data visualization tools (e.g., Tableau, Power BI).
Understanding of data governance and compliance standards.

Responsibilities

Design, develop, and optimize complex SQL queries and PL/SQL scripts for data extraction, transformation, and loading (ETL).
Create and manage Oracle database objects including tables, views, indexes, and partitioned objects.
Implement and manage file transfers using NDM (Connect:Direct) to ensure secure and efficient data movement.
Utilize Autosys for job scheduling and monitoring to ensure timely execution of data processes.
Develop and maintain data processing workflows using Python and PySpark for large-scale data processing.
Leverage GitHub as a code repository for version control and collaboration on data engineering projects.
Utilize GitHub CoPilot to enhance development efficiency and streamline coding processes.
Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
Ensure data quality and integrity through rigorous testing and validation processes.
Stay updated with industry trends and best practices in data engineering and contribute to continuous improvement initiatives.