Support the design, build, and operation of the Data Platform, blending infrastructure engineering with data engineering practices to keep the Databricks/Azure environment reliable, automated, and self-service for platform users.
- Design and maintain CI/CD pipelines (Azure DevOps/GitHub Actions) for infrastructure and application deployments.
- Write and maintain Terraform IaC for Azure services and Azure Databricks (Unity Catalog, workspaces, networking, compute policies).
- Develop and support application code in Python and/or .NET for platform tooling and automation.
- Build and maintain shared libraries/modules used by data engineers and analysts across the platform.
- Design and implement platform management processes covering cost tracking, governance, provisioning, and monitoring.
- Collaborate with data engineers, analysts, and security/infrastructure teams to translate requirements into platform capabilities.
- Troubleshoot infrastructure, pipeline, and integration issues, including networking, authentication, and connectivity.
- Contribute to platform documentation and best-practice standards.
Required Skills & Experience
- Strong experience with Microsoft Azure infrastructure, including Entra ID, Key Vault, networking, and storage.
- Hands-on experience with Azure Databricks, including Unity Catalog and cluster/job management.
- Terraform expertise, preferably in multi-environment deployments.
- Proficiency in Python and SQL; .NET experience is an advantage.
- Experience building and maintaining CI/CD pipelines.
- Understanding of data platform governance and cost management.
- Ability to work effectively across infrastructure and data engineering domains.
Nice to Have
- Experience with Medallion Architecture (Bronze/Silver/Gold).
- Exposure to FinOps and cost attribution practices.
- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related field.
- 5+ years of experience in Data Engineering, Data Platform Engineering, or Big Data development.
- Proven experience designing, building, and supporting scalable enterprise data platforms.
- Strong understanding of data architecture, data modeling, and distributed computing concepts.
- Experience working in Agile/Scrum development environments.
- Excellent analytical, problem-solving, and troubleshooting skills.
