InDebted
Senior Data Engineer
To be the preferred choice of customers to support their debt free journey, everywhere.
Data and analytics
Full-time
Remote | GMT+8, GMT+9, GMT+10
Visa sponsorship · No
Senior · A role for someone with advanced knowledge and skills. May involve mentoring, leading others or specialisation. Typically at least 5 years of experience.
·
Why InDebted
Committed to making a difference, InDebted was founded to change the world of consumer debt recovery for good. We're disrupting a broken industry and creating a better way for businesses to recover debt - where the customer comes first.
We’re a global, digital first and customer-obsessed team of over 230 people, living and working across 11 different countries.
And we’ve also recently been named the AFR BOSS Best Place to Work! By focusing on the happiness and wellbeing of our people, we’ve created a truly flexible, purpose driven and engaging workplace. We’re all here with one shared vision - to remove barriers to financial freedom for all people.
About the role
Role Overview We are seeking a Senior Data Engineer to join our foundational engineering team. This role is pivotal as we embark on building a new data lake from the ground up using AWS native components alongside Spark, Iceberg, and DBT. Our data journey is just beginning, with our initial few billion events falling within a 1-20 terabyte range. We anticipate the increasing relevance of big data technologies such as Spark, Kafka, and Samza in the near future.
Key Responsibilities:
Design, build, and maintain scalable data pipelines and infrastructure to support our data lake initiative. Utilise AWS native components in conjunction with Spark, Iceberg, and DBT to construct efficient data solutions. Collaborate with cross-functional teams to ensure data accuracy, integrity, and availability for various business needs. Continuously evaluate and integrate emerging big data technologies (e.g., Spark, Kafka, Samza) to enhance our data architecture. Implement infrastructure as code using CDK and Terraform to manage and deploy cloud resources. Embrace a language-agnostic approach, selecting the best tools and technologies suited to the problem space. Requirements: Minimum of 5 years of experience as a data engineer. Strong expertise in AWS and experience with CDK and Terraform or equivalent IaC frameworks. Proficiency in big data technologies, with prior exposure to Spark and Kafka preferred. Demonstrated ability to design and implement scalable, secure, and cost-effective data solutions. Language-agnostic, with a deep understanding of various data technologies and frameworks. Excellent problem-solving skills and the ability to work autonomously. Strong values of truth, knowledge, and transparent collaboration across different functions. Preferred Qualifications: Experience with data lake architecture and implementation. Familiarity with real-time data processing frameworks such as Kafka and Samza. Prior experience in building and scaling data platforms for high-volume transactions. Knowledge of data modelling, ETL processes, and data warehousing best practices.
What We Offer:
Competitive salary and benefits package. Opportunity to work in a fast-growing, profitable company with a global reach. Collaborative and innovative work environment. Autonomy and trust to shape the future of our data infrastructure. The chance to make a significant impact on the consumer debt recovery industry.
What you'll be responsible for
- 🔀
Data Pipelines
Develop and maintain processes for the extraction, flow, and transformation of data
- 🔲
SQL and Databases
Use SQL to query databases to extract and process data
- 🤖
Scripting and Automation
Develop scripts and automated workflows to improve the efficiency and accuracy of repeated tasks
Skills you'll need
- 🧮
Numerical problem solving
Works with numerical information and performs mathematical calculations to solve problems
- 💭
Critical thinking
Identifies and synthesizes patterns and trends amongst various sources of information to reach a meaningful conclusion, perspective or insight
- 🔍
Attention to detail
Accurately identifies and rectifies discrepancies or errors that exists in information and deliverables