Data Engineer / Bangalore -Advantest

ITC Infotech

1d • Hybrid

About The Position

We are looking for a highly skilled Senior Data Engineer with deep expertise in Azure Stack to join our growing team. As a Data Engineer, you will be instrumental in designing and developing robust data pipelines, data lakes, and cloud-based solutions to support our business intelligence and analytics initiatives. You will work in a hybrid cloud environment to manage data services, ensuring high scalability, security, and performance for data-driven decision-making.

Requirements

• Experience: 3-8 years of experience in data engineering, with at least 2+ years of experience working specifically with Azure Stack and Azure cloud services in a hybrid environment.
• Azure Stack Expertise: Proficient in Azure Stack Hub, Azure Data Factory, Azure SQL Database, Azure Synapse Analytics, Azure Databricks, and Azure Blob Storage.
• Programming Skills: Proficiency in SQL, Python, or similar scripting languages for building data pipelines and processing large datasets.
• ETL Tools: Experience with building and optimizing ETL workflows using Azure Data Factory, Apache Airflow, or other similar tools. Data Integration: Experience working with data integration tools, including both structured and unstructured data sources, and technologies such as Kafka, Spark, Hadoop, or others.
• Data Warehousing & Storage: Strong understanding of data warehousing concepts (e.g., star/snowflake schemas) and expertise with Azure Data Lakes and SQL/NoSQL storage solutions.
• Soft Skills: Excellent analytical and problem-solving abilities.
• Strong communication skills to articulate technical details to non-technical stakeholders. Ability to work collaboratively in a distributed team environment. Self-motivated, proactive, and able to work under minimal supervision. Strong attention to detail, ensuring data quality and security.

Nice To Haves

• Big Data Technologies: Familiarity with big data technologies like Apache Hadoop, Apache Spark, Databricks, and Kafka is a plus.

Responsibilities

• Data Engineering: Design and implement scalable data pipelines, ETL workflows, and processing frameworks within Azure Stack and hybrid cloud environments.
• Cloud Architecture: Leverage the capabilities of Azure Stack, Azure Data Factory, Azure Synapse Analytics, and other Azure cloud services to integrate, manage, and optimize enterprise data solutions.
• Data Integration & Storage: Build and optimize data lakes, data warehouses, and cloud-based storage systems on Azure, integrating structured and unstructured data from multiple sources.
• Performance Optimization: Continuously improve data pipelines for performance, scalability, and cost-efficiency. Implement automation and monitoring solutions to ensure smooth operation.
• Cross-Functional Collaboration: Work closely with product managers, data scientists, and business intelligence teams to implement data models, data visualizations, and analytics solutions.
• Mentorship: Provide mentorship and technical leadership to junior engineers, helping them improve their skills in cloud data technologies, coding standards, and engineering best practices.
• Cloud Migration: Assist in the migration of on-premises data infrastructure to Azure Stack, working closely with infrastructure teams to ensure a seamless transition.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 5,001-10,000 employees