Data Engineering Specialist (Azure Platform)

Data Engineering Specialist (Azure Platform)

Job Description

The ideal candidate should have a strong background in data engineering, cloud data architecture, and ETL pipeline development within the Microsoft Azure ecosystem, with proficiency in Python and SQL, and a deep understanding of how to leverage these skills to support packaging engineering and consumer insights. As a Data Engineering Specialist on the AE&D Packaging Beverages team, you will play a pivotal role in advancing our data-driven packaging capabilities and enabling insights into consumer-packaging interactions. You will work closely with packaging engineers and the Consumer Insights team to identify, classify, and transform raw data from diverse sources and formats into structured, analysis-ready datasets. Modern cloud analytics tools and platforms (such as Azure data services and Databricks), combined with robust data management practices, are widely recognized as essential components of our engineering toolbox, empowering us to convert unorganized data into actionable insights that drive packaging innovation. Our goal is to harness data to deepen our understanding of how consumers interact with packaging and to drive breakthrough packaging innovations that delight consumers, ensuring robust design and performance from concept through consumer use and disposal.

Key Responsibilities:
Lead development of advanced data integration and analytics capabilities, leveraging cloud-based tools (particularly Microsoft Azure) and best practices in data engineering to enhance packaging lifecycle performance.
Act as a subject matter expert in data engineering and analytics, continuously researching and adopting the latest Azure services and data technologies to benefit the team’s objectives.
Design and implement robust ETL processes and data pipelines that transform unorganized data from various sources into structured, high-quality datasets ready for analysis and visualization.
Drive packaging innovation by integrating data-driven insights into existing systems and workflows for real-time monitoring and decision-making, supporting data-informed package design and development.
Support the development of the overall data strategy, aligning data architecture and pipeline initiatives with business goals and packaging innovation objectives.
Work closely with packaging engineers and the Consumer Insights team to identify, gather, and ingest data from diverse internal sources, ensuring that valuable consumer and operational data is captured and made accessible in the Azure data platform.
Maintain open communication and build strong relationships with multiple functions, including Packaging R&D, data engineering and Consumer Insights to ensure data solutions meet cross-functional needs and drive collaborative innovation.
Collaborate with executive leadership to understand organizational objectives and translate them into actionable data initiatives and strategic solutions.
Ensure proper data governance, security, and confidentiality when handling information, including adherence to Non-Disclosure Agreements and corporate data protection policies when engaging with third parties.

Critical Competencies:
Strong analytical and problem-solving skills, with the ability to translate business requirements into effective data solutions.
Broad knowledge of data management and analytics principles, including database design, and data warehousing.
Experience designing and optimizing ETL processes and data pipelines in cloud environments, ensuring data integrity and efficient performance at scale.
Analytical thinking and attention to detail in working with complex datasets and troubleshooting data issues.
Proficiency in Azure Data Services: Deep understanding and hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure SQL Database for orchestrating and managing scalable data workflows.
Expertise in Databricks and Spark-based Processing: Demonstrated ability to develop and optimize distributed data processing pipelines using Azure Databricks and Apache Spark for large-scale data transformation and analytics.
Advanced SQL and Python Development: Strong command of SQL for data querying and transformation, and Python for scripting ETL logic, data wrangling, and integration with Azure-based services.
Data Modeling and Schema Design: Experience designing efficient, scalable, and maintainable data models and schemas for structured and semi-structured data, ensuring compatibility with BI tools and downstream analytics.
Monitoring and Performance Optimization: Ability to implement robust monitoring, logging, and performance tuning strategies for data pipelines and storage systems within the Azure environment to ensure reliability, scalability, and cost-efficiency.
Strong knowledge of data architecture and database systems, with familiarity in designing scalable data models and storage solutions.
Intellectual curiosity and a continuous learning mindset to stay current with emerging technologies and methodologies in data engineering.
Building strong technical relationships and cross-functional collaboration skills to effectively work with engineering, insights, and business teams.
Experience with data engineering and analytics tools, such as Azure Data Factory, Azure Databricks, Azure SQL Database, and Python libraries (e.g., Pandas, NumPy).
Project management skills, with the ability to manage multiple data projects, prioritize tasks, and deliver results on time.
Teamwork and collaboration, with a proven track record of working effectively in diverse, cross-functional teams.
Motivated and results-driven, with a passion for leveraging data to drive innovation and improve processes.
Ability to work in an ambiguous and dynamic work environment, remaining flexible and resourceful amid changing priorities.
Experience with data visualization and BI tools (such as Power BI or Tableau) is a plus, demonstrating the ability to turn data into actionable insights for stakeholders

Education & Experience:
Bachelor’s MS or PhD in Data Science, computer science or related technical field preferably with 2+ years proven experience in developing and implementing AI/ML and/or digital twin solutions in related consumer goods field (preferably in rigid and flexible packaging).
Demonstrated expertise on application of data science and analytics principles in industrial applications.
Strong leadership qualities, verbal and written communication skills, and technical analysis and problem solving skills are required.

Notes:
Monday-Friday, 8AM-5PM

VIVA is an equal opportunity employer. All qualified applicants have an equal opportunity for placement, and all employees have an equal opportunity to develop on the job. This means that VIVA will not discriminate against any employee or qualified applicant on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.