Software / Data Engineer (1451) (Remote)

Zapata Technology • Augusta, GA

5h • Remote

About The Position

Zapata Technology is seeking an experienced Software / Data Engineer for a remote role for west coast hours. The Data Engineer will lead the transformation of our data infrastructure, utilizing a custom ASP.NET application, Postgres stored procedures, and migrating existing VBA scripts to modern platforms to build and manage ETL processes. This position is required to meet the customer’s current needs but must also be prepared to transition to modern ETL tools such as Apache NiFi and Airflow in the future. The ideal candidate will have extensive experience in data engineering, database management, and designing efficient ETL workflows without relying on traditional ETL tools.

This position can be performed remotely.

Requirements

• Design and develop modular ETL solutions that facilitate data ingestion and delivery across various systems, utilizing custom ASP.NET applications and Postgres stored procedures.
• Develop and maintain scalable data storage systems, focusing on Postgres or other relational databases, to support efficient data retrieval and transformation processes.
• Develop optimized SQL queries and stored procedures that process and transform both structured and unstructured data, ensuring performance and reliability in data handling.
• Build and manage data pipelines tailored for reporting and analytics, leveraging custom scripting rather than relying on third-party ETL tools to maintain flexibility and control.
• Collaborate with cross-functional teams to understand data requirements and implement custom data solutions that support the organization’s analytics and reporting needs.
• Create custom scripts and applications to enhance data transformation capabilities and address specific data management requirements.
• Document data ingestion and transformation processes thoroughly, ensuring compliance with data governance frameworks, including HRPP, Privacy, HIPAA, and SOP guidelines.
• Perform quality assurance tasks to validate data accuracy and troubleshoot data quality issues, upholding system integrity and reliability.
• Support metadata management practices, ensuring all data processes comply with governance standards and industry best practices.
• 4+ years of Software Development
• This position requires an active Secret clearance with the ability to obtain TS/SCI.

Nice To Haves

• Experience with Python, ASP.NET, C#, and related web application development frameworks.
• Advanced knowledge of Postgres, particularly in writing complex stored procedures, query optimization, and transitioning data infrastructure to Postgres.
• Strong familiarity with AWS cloud environments, with experience in migrating data systems to AWS.
• Experience with low-code/no-code platforms and distributed data processing tools such as Spark, Databricks, or similar.
• Knowledge of real-time data processing, streaming applications, and data warehousing concepts, especially in cloud-native environments.
• Proficiency in shell scripting and automation to support cloud migrations and system integrations.
• Experience working in Agile methodologies, particularly within cloud-based or hybrid cloud environments.
• Requires Security+ Certification.

Responsibilities

• Design and develop modular ETL solutions that facilitate data ingestion and delivery across various systems, utilizing custom ASP.NET applications and Postgres stored procedures.
• Develop and maintain scalable data storage systems, focusing on Postgres or other relational databases, to support efficient data retrieval and transformation processes.
• Develop optimized SQL queries and stored procedures that process and transform both structured and unstructured data, ensuring performance and reliability in data handling.
• Build and manage data pipelines tailored for reporting and analytics, leveraging custom scripting rather than relying on third-party ETL tools to maintain flexibility and control.
• Collaborate with cross-functional teams to understand data requirements and implement custom data solutions that support the organization’s analytics and reporting needs.
• Create custom scripts and applications to enhance data transformation capabilities and address specific data management requirements.
• Document data ingestion and transformation processes thoroughly, ensuring compliance with data governance frameworks, including HRPP, Privacy, HIPAA, and SOP guidelines.
• Perform quality assurance tasks to validate data accuracy and troubleshoot data quality issues, upholding system integrity and reliability.
• Support metadata management practices, ensuring all data processes comply with governance standards and industry best practices.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed
• Number of Employees: 11-50 employees