Senior Data Engineer (with MLOps expertise)
Пълно описание
Over 20 years of market experience, we brings together technologists, creators and innovators in Europe, North and Latin America, and the Middle East. Join our international team and take the mission to solve the advanced tech challenges of tomorrow!
What project we have for you:
Our customer is a Dutch multinational developer and creator of location technology and consumer electronics headquartered in Amsterdam. They are looking for a skilled Data Engineer with strong MLOps experience to lead the design and automation of production-grade data and ML pipelines on Azure Databricks. You’ll work at the intersection of data engineering, infrastructure automation, and machine learning — transforming prototype workflows into robust, observable, and cost-efficient pipelines.
What you will do:
Design and maintain end-to-end data and ML pipelines using Databricks Workflows, Delta Lake, and Unity Catalog (bronze–silver–gold layers, schema evolution, access policies). Build reproducible training and deployment workflows integrated with tools for experiment tracking, model registry, and artifact management. Implement data quality frameworks and observability metrics aligned with industry best practices. Build and monitor dashboards (e.g. past experience with Lakeview, Grafana, or similar) for data quality, model performance, and operational metrics. Automate data ingestion and feature generation jobs, leveraging PySpark, SQL Warehouses, and Databricks Asset Bundles (DAB) under CI/CD (GitHub Actions or Azure DevOps). Manage access and security, ensuring compliance and reliability. Optimize compute performance and cost (spot/autoscaling, cluster tuning, caching, partitioning).
What you need for this:
Strong proficiency in Python is essential, along with experience in shell scripting and potentially other languages like Java. Hands-on experience with at least one major cloud service provider (Azure is preferable) Experience in Azure Databricks Workflows, Delta Lake, and Unity Catalog Experience in PySpark, SQL Warehouses, and Databricks Asset Bundles (DAB) under CI/CD (GitHub Actions or Azure DevOps). Strong software engineering practices, including testing, code optimization, and design patterns. Excellent communication and collaboration skills to bridge the gap between technical and non-technical teams.
What it’s like to work with us:
We are committed to being an equal opportunity employer, fostering equity, diversity, and inclusion. We welcome and celebrate the differences of all qualified applicants. Join our team for a career where your unique perspectives are not only valued but crucial to our success.
What project we have for you:
Our customer is a Dutch multinational developer and creator of location technology and consumer electronics headquartered in Amsterdam. They are looking for a skilled Data Engineer with strong MLOps experience to lead the design and automation of production-grade data and ML pipelines on Azure Databricks. You’ll work at the intersection of data engineering, infrastructure automation, and machine learning — transforming prototype workflows into robust, observable, and cost-efficient pipelines.
What you will do:
Design and maintain end-to-end data and ML pipelines using Databricks Workflows, Delta Lake, and Unity Catalog (bronze–silver–gold layers, schema evolution, access policies). Build reproducible training and deployment workflows integrated with tools for experiment tracking, model registry, and artifact management. Implement data quality frameworks and observability metrics aligned with industry best practices. Build and monitor dashboards (e.g. past experience with Lakeview, Grafana, or similar) for data quality, model performance, and operational metrics. Automate data ingestion and feature generation jobs, leveraging PySpark, SQL Warehouses, and Databricks Asset Bundles (DAB) under CI/CD (GitHub Actions or Azure DevOps). Manage access and security, ensuring compliance and reliability. Optimize compute performance and cost (spot/autoscaling, cluster tuning, caching, partitioning).
What you need for this:
Strong proficiency in Python is essential, along with experience in shell scripting and potentially other languages like Java. Hands-on experience with at least one major cloud service provider (Azure is preferable) Experience in Azure Databricks Workflows, Delta Lake, and Unity Catalog Experience in PySpark, SQL Warehouses, and Databricks Asset Bundles (DAB) under CI/CD (GitHub Actions or Azure DevOps). Strong software engineering practices, including testing, code optimization, and design patterns. Excellent communication and collaboration skills to bridge the gap between technical and non-technical teams.
What it’s like to work with us:
We are committed to being an equal opportunity employer, fostering equity, diversity, and inclusion. We welcome and celebrate the differences of all qualified applicants. Join our team for a career where your unique perspectives are not only valued but crucial to our success.