Data Engineer

Purpose Financial • Greenville, SC

19d

About The Position

The Data Engineer will play a critical role in designing, building, and maintaining our data infrastructure and pipelines to enable efficient collection, storage, processing, and analysis of large-scale datasets. Collaborate with cross-functional teams to build and acquire underlying data for application owners, end-users, and design strategic data services for the organization. The right Data Engineer will need to have working knowledge of AWS and/or Snowflake cloud environments.

Requirements

• Two years of experience in data engineering, data pipelines, and data services preferred.
• Familiarity with Agile/Scrum based development and methodology.
• Strong proficiency in programming languages such as Python, SQL, Spark, or Java, with experience in data manipulation, transformation, and analysis.
• Hands on experience with cloud-based data platforms and services such as AWS and Snowflake.
• Experience with distributed computing frameworks such as Apache Spark, Kafka, etc.
• Proficiency in database systems, data warehousing, data patterns/architectures, and SQL query optimizations.
• Familiarity with containerization and orchestration technologies such as Docker or Kubernetes.
• Excellent problem-solving skills, attention to detail, and ability to work effectively in a fast-paced and collaborative environment.
• Strong communication, interpersonal and teamwork skills, with the ability to interact with stakeholders at all levels of the data team.
• Excellent written and verbal communications skills; adaptability and flexibility to a changing environment; and comfortable working in a dynamic, high volume, fast-paced environment.
• Ability to understand and ensure compliance with policies, procedures, and laws governing our industry/business and products.
• Sitting for long periods of time; standing occasionally; walking; bending; squatting; kneeling; pushing/pulling; reaching; twisting; frequent lifting of less than 10 lbs., occasional lifting of up to 20 lbs.; driving and having access during the workday to an insured and reliable transportation; typing; data entry; grasping; transferring items between hands and/or to another person or receptacle; use of office equipment to include computers; ability to travel to, be physically present at, and complete the physical requirements of the position at any assigned location.
• Must be eligible to work in the USA and able to pass a background check.

Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, semi-structured, and unstructured data from various sources into our data lake and warehouse environments.
• Implement data integration solutions to consolidate data from disparate sources, including databases, API’s, streaming platforms, and third-party services (e.g., Snowpipe, SnowPark, Dynamo, Kafka).
• Optimize data processing workflows for performance, efficiency, and scalability using distributed computing or parallel processing frameworks such as FiveTran, dbt, Snowpark, Snowpipe, etc.…
• Collaborate cross-functionally with IT & business stakeholders to understand data requirements, define data models, and develop solutions to support data services, reporting, and Software Development.
• Partner with data, IT, and business teams to improve design and building of metrics to enhance our analytic capabilities.
• Implement data quality checks, data validation processes, and error handling mechanisms to ensure the accuracy, completeness, and reliability of data across all stages of the data lifecycle.
• Support the design and maintenance of data schemas, and metadata repositories for governance documentation of data lineage, definitions, and dependencies.
• Support the development and maintenance of data governance policies, standards, and best practices to ensure compliance with data privacy regulations and industry standards.
• Apply best practices for AWS and Snowflake architectures, data pipelines and data models.
• Monitor, troubleshoot, and optimize the performance and availability of data systems and infrastructure using monitoring and logging tools such as Prometheus.
• Stay current with emerging technologies, tools, and trends in data engineering, cloud architectures, and cloud computing to evaluate their potential impact and relevance to our data platforms.

Benefits

• Competitive Wages
• Health/Life Benefits
• Health Savings Account plus Employer Seed
• 401(k) Savings Plan with Company Match
• Paid Parental Leave
• Company Paid Holidays
• Paid Time Off including Volunteer Time
• Tuition Reimbursement
• Business Casual Environment
• Rewards & Recognition Program
• Employee Assistance Program
• Office in downtown Greenville that offers free parking, onsite gym, free snacks/drinks

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 101-250 employees