Senior Data Engineer
About The Position
Avaya LLC is seeking a Senior Data Engineer (Irving, TX) to spearhead the design, construction, and upkeep of our AI-enabled data pipeline. Collaborate closely with project managers, engineers, and analysts to elevate our data ecosystem. Develop and maintain scalable data pipelines for ingesting, transforming, and distributing large datasets. Design and implement secure data storage solutions. Optimize data processing performance and cost efficiency through monitoring and tuning. Integrate cloud services effectively for data engineering tasks while ensuring security and compliance. Implement data governance measures and ensure compliance with regulations. Utilize automation tools to streamline data operations. Collaborate with teams to understand data needs and deliver actionable solutions. Enhance real-time data insights for big data processing and analytics. Support BI initiatives through integration with visualization tools. Local telecommuting permitted.
Requirements
- Bachelor’s degree in Computer and Information Science or a related field
- 6 years of experience building and maintaining big data, advanced data pipelines and scalable data infrastructure.
- Experience with Cloud-based technologies, (Azure and Data Lakes)
- Experience with Lakehouse Storage Technologies and formats such as Azure Data Lake 2 (ADSL2), Delta Lake
- Experience with AI, machine learning, and modern data processing tools (Pandas, NumPy, and Matplotlib)
- Experience with Automation tools (Terraform and GitHub Actions)
- Experience with Database Skills (SQL, ETL processes)
- Experience with Big data technologies (Hive, Spark, PySpark, Airflow and Databricks)
- Experience with Visualization tools (Power BI, Looker)
- Experience with Programming languages (Python, Go, Scala, and Java)
Responsibilities
- Spearhead the design, construction, and upkeep of our AI-enabled data pipeline.
- Collaborate closely with project managers, engineers, and analysts to elevate our data ecosystem.
- Develop and maintain scalable data pipelines for ingesting, transforming, and distributing large datasets.
- Design and implement secure data storage solutions.
- Optimize data processing performance and cost efficiency through monitoring and tuning.
- Integrate cloud services effectively for data engineering tasks while ensuring security and compliance.
- Implement data governance measures and ensure compliance with regulations.
- Utilize automation tools to streamline data operations.
- Collaborate with teams to understand data needs and deliver actionable solutions.
- Enhance real-time data insights for big data processing and analytics.
- Support BI initiatives through integration with visualization tools.
Benefits
- Local telecommuting permitted.
- Eligible for the Employee Referral Program
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
