Senior Big Data Software Engineer (AWS + Databricks)

About the role

This role involves designing and developing scalable data processing solutions in AWS environments, working with large-scale and distributed data. You will own reliable data pipelines and help build the data platform, working with engineering, AI and product teams to support analytics, automation and AI-enabled solutions.

Key responsibilities

• Design, develop and maintain scalable batch and distributed data processing pipelines
• Build and optimize data processing solutions, owning their performance, reliability, scalability and cost efficiency
• Collaborate with engineering, AI and product teams to support analytics, automation and AI use cases
• Develop data solutions with Python, SQL, Apache Spark, Databricks and AWS services such as AWS Glue, Amazon Athena and Amazon Aurora
• Work with PostgreSQL and MySQL for data processing and integration workflows
• Maintain workflow orchestration with Apache Airflow
• Contribute to CI/CD, infrastructure as code, testing, troubleshooting and production support with Terraform or similar tools
• Take part in architecture discussions and improve data platforms and engineering practices

Required profile

• 5+ years of professional experience in Big Data, Data Engineering or related fields
• Upper-intermediate or higher English for cross-team collaboration
• Plus: familiarity with GenAI, MLOps, machine learning workflows or AI platform integrations

Required skills

• Python and SQL; Java considered a plus
• Apache Spark, Databricks and distributed data processing
• AWS Glue, Amazon Athena, Amazon Aurora, PostgreSQL and MySQL
• Apache Airflow and Apache Kafka
• Terraform or similar infrastructure as code, CI/CD and version control