The job is about joining Karbon as a Data Engineer, where you will have the exciting opportunity to build a new unified data platform that powers AI tools and insights. This role is crucial for supporting both the AI and Insights teams, allowing you to make a significant impact on the Karbon platform. The team values collaboration and innovation, creating a dynamic environment where your contributions will be recognized and celebrated.
You'll be responsible for
🏗️
Developing a unified data platform
Develop our new unified data platform on Databricks, establishing the Medallion Architecture using DLT for data modeling and transformations.🔄
Developing data pipelines
Create and manage resilient data pipelines for both batch and real-time processing from various sources in our Azure data ecosystem.✅
Championing data quality and governance
Establish best practices for data quality, integrity, and observability, building automated quality checks and monitoring for all data assets.Skills you'll need
🛠️
Data engineering
Proven experience in building and scaling data platforms, with a focus on creating robust data solutions.☁️
Databricks
Experience with Databricks for developing unified data platforms and managing data pipelines.🐍
Python
Strong proficiency in Python for data processing and pipeline development.View more