Remote | Big Data Engineer — $30–$80/hour

24-Mag•New York, NY

19h•Remote

About The Position

24-MAG LLC is offering a fully remote, part-time independent contractor consulting engagement for an experienced Big Data Engineer contributing technical expertise to an advanced AI development project. The work focuses on large-scale data pipelines, distributed systems, databases, and modern data-engineering workflows. The engineer will design and evaluate scalable data solutions involving pipeline architecture, integration, transformation, distributed processing, storage, and system reliability. Compensation is $30–$80 per hour. Project scope, workload, timing, and duration may evolve according to project requirements. Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party. The opportunity is available through 24-MAG LLC, which connects experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. Applicant information may be processed for recruitment and opportunity matching in accordance with the platform’s Privacy Policy.

Requirements

Proven professional experience in big-data engineering, including hands-on responsibility for large-scale data pipelines and architectures.
Advanced proficiency in Python for data processing, automation, integration, and related engineering workflows.
Deep understanding of relational and NoSQL databases, including optimisation, management, and performance considerations.
Practical experience with distributed data-processing frameworks such as Hadoop, Spark, Flink, or comparable technologies.
Strong foundation in data modelling, ETL processes, data integration, and data-warehousing principles.
Ability to troubleshoot distributed systems, database performance, pipeline failures, and data-quality issues.
Excellent written and verbal communication skills, including the ability to explain technical concepts to engineering and non-technical stakeholders.
Strong attention to detail, initiative, and ability to work independently in remote environments.

Nice To Haves

Experience supporting fast-paced, startup-like, or globally distributed teams.
Familiarity with cloud-based data platforms such as AWS, GCP, or Azure.
Exposure to machine-learning operations, data-science workflows, or adjacent AI infrastructure.

Responsibilities

Design, build, and maintain scalable data pipelines and big-data architectures.
Develop reliable workflows for ingesting, transforming, processing, and delivering large volumes of data.
Translate technical and business requirements into robust data-engineering solutions and apply suitable architectural patterns for distributed, high-volume systems.
Develop Python-based data-processing, automation, and integration solutions, including ETL and transformation workflows across structured and unstructured data sources.
Work with distributed processing technologies and modern big-data frameworks to create reproducible and efficient workflows.
Identify opportunities to improve data-processing efficiency, reliability, and maintainability.
Develop, manage, and optimise relational, NoSQL, distributed database, and storage systems.
Apply data-modelling and data-warehousing principles to support analytical and operational workloads.
Monitor system performance and troubleshoot bottlenecks, failures, performance, availability, and integration issues across distributed data environments.
Optimise storage, queries, and processing workflows for efficiency and resilience.
Apply data-quality, security, and governance standards across engineering workflows.
Collaborate with technical and cross-functional stakeholders to understand data requirements and delivery priorities.
Document architectures, data flows, implementation decisions, and technical trade-offs.
Communicate complex engineering concepts to technical and non-technical audiences.
Support consistent engineering standards across remote and distributed project environments.