Senior Data Engineer

We are looking for a Senior Data Engineer with solid experience in the design and development of data pipelines. The ideal candidate will be responsible for building and maintaining robust pipelines to extract, transform, and load (ETL) data from a variety of sources.

Requirements:

• 5+ years of experience in a data engineering role.
• Strong experience with Python and SQL.
• Proven expertise with workflow management tools like Dagster or Airflow.
• Deep understanding of AWS cloud services related to data engineering, including S3, Redshift, EMR, Glue, and Athena.
• Experience with data modeling, data warehousing, and ETL processes.
• Familiarity with data governance and data security best practices.
• Excellent communication and collaboration skills.

Bonus Points:

• Experience with other cloud platforms (GCP, Azure).
• Experience with containerization technologies (Docker, Kubernetes).
• Contributions to open-source projects.

Responsibilities:

• Design, develop, and implement data pipelines using tools like Dagster or Airflow for ingestion, transformation, and loading of data from various sources.
• Build and maintain our data warehouse architecture on AWS, leveraging services like S3,
• Redshift, EMR, Glue, and Athena.
• Implement data quality checks and monitoring systems to ensure data accuracy and integrity.
• Optimize data pipelines for performance and cost-effectiveness.
• Collaborate with data scientists and analysts to understand their data needs and provide them with reliable and performant data solutions.
• Stay up-to-date with the latest technologies and trends in the data engineering field.
• Contribute to the development and improvement of our data engineering best practices.

Location: Remote

Time shift: Full time