Middle Data Engineer

About the role

We are looking for a Middle Data Engineer to help build and maintain a data platform that supports data engineering, BI, Machine Learning, and AI initiatives. In this role, you will design batch and real-time data pipelines, work with distributed data processing tools, and collaborate with business and technical stakeholders to deliver reliable data solutions.

Key responsibilities

• Work with business stakeholders and cross‑functional teams to understand data requirements and deliver scalable data solutions.
• Design, develop, and maintain ETL processes that move data from different sources into the data platform.
• Build batch and event‑driven data pipelines using cloud and on‑premises hybrid data platforms.
• Collaborate with data architects to review solutions and data models, ensuring alignment with best practices.
• Implement data quality standards and collaborate with governance teams to support compliance with data policies.
• Optimize data integration workflows for performance, reliability, and maintainability.
• Troubleshoot and resolve data integration and processing issues.

Required profile

• 3–5 years of experience building batch and real‑time data pipelines using big data technologies.
• Hands‑on experience designing, developing, and maintaining ETL processes and data pipelines.
• Proficiency in Python and/or Scala, with ability to optimise data processing scripts.
• Strong knowledge of SQL and experience designing scalable data models.
• Familiarity with cloud‑based data platforms such as Snowflake, Databricks, AWS, Azure, or GCP.
• Experience using CI/CD tools and working with Docker and Kubernetes for data pipeline automation.
• Basic understanding of data governance, data quality frameworks, and compliance requirements.

Required skills

• Spark
• Hadoop
• Airflow
• NiFi
• Kafka
• Python
• Scala
• SQL
• Snowflake
• Databricks
• AWS
• Azure
• GCP
• Docker
• Kubernetes
• CI/CD tools

What we offer

• Up to 15 paid vacation days annually plus 18 Colombian public holidays.
• Learning & growth opportunities through Sombra University workshops, internal learning programs, tech communities, language courses, mentorship, and access to the Platzi online learning platform.
• Company‑provided work equipment and internal referral program.
• Bonuses for marriage or the birth of a child, plus Sombra events and internal initiatives.