Senior IT Data Engineer

Senior IT Data Engineer

Job Description

Position Requirements:
Participates in the planning and execution of policies, practices, and projects that acquire, control, protect, and enhance the value of data assets. Facilitates obtaining data from a variety of different sources, in the right formats, assuring that it adheres to data quality standards, as well as resolving any information flow and content issues. Builds a robust data pipeline that cleans, transforms, and aggregates unorganized data into databases.

Proficiency in developing, maintaining, monitoring, and the long-term operations of data pipelines or processing systems running in Cloudera Data Platform. Understanding of data extraction, transformation, loading, and performance tuning of solutions utilizing multiple streams of input data code-based, git and DevOps-enabled technologies using Python or SQL such as PySpark, pandas, or dbt
5+ years of experience in application/data development (i.e., Python)
5+ years of experience with data integration/ingestion tools (i.e., Apache NiFi)
Advanced level knowledge of SQL, Java, Microsoft SQL Server, distributed data/computing platforms (i.e., Apache NiFi, Hadoop, MapReduce, Hive, HBase, Kafka, Spark)
Experience with Scrum and Kanban methodologies
Experience with UNIX/Linux including basic commands and shell scripting
Experience in implementing and maintaining continuous integration continuous delivery (CI/CD) pipelines and data platform management

VIVA is an equal opportunity employer. All qualified applicants have an equal opportunity for placement, and all employees have an equal opportunity to develop on the job. This means that VIVA will not discriminate against any employee or qualified applicant on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.