Data Engineer | Kuala Lumpur, MY | Remote

Seeking a Data Platform Architect & Lead Engineer to oversee Databricks and Microsoft Fabric environments. You will architect enterprise dimensional models, build Spark/PySpark pipelines, and mentor engineering teams. Requires deep expertise in Delta Lake, OneLake, CI/CD, and FinOps, alongside proven ability to integrate retail/POS data sources into high-performance analytics ecosystems

Role Summary

A Lead Data Architect / Senior Data Engineering Lead who owns the overall data platform strategy, guides engineering teams, and builds high-performing, cost-efficient data infrastructure to power advanced analytics and reporting.

Core Responsibilities

• Drive Platform Architecture: Design and scale the enterprise data platform across Databricks and Microsoft Fabric.
• Lead Engineering & Pipelines: Oversee the build of fast, reliable data pipelines and dimensional models.
• Lead & Mentor Teams: Guide and upskill data engineers and data modelers in technical best practices.
• Bridge Business & Tech: Work with leadership and analysts to turn business goals into actionable data roadmaps.
• Governance & Quality: Establish clear standards for data quality, testing, lineage, and documentation.
• Optimization & DevOps: Streamline workflows for high performance and low cost using DevOps/CI/CD and FinOps practices.

Key Technical Skills

• Platforms: Databricks (Delta Lake, Unity Catalog) & Microsoft Fabric (Lakehouse, OneLake, Data Factory).
• Engineering & SQL: Advanced Spark/PySpark, batch/streaming/CDC pipelines, and expert-level SQL query tuning.
• Data Modeling: Dimensional modeling (star/snowflake), Data Vault, and metadata-driven frameworks.
• Operations: CI/CD automation, DevOps principles, and cost optimization (FinOps).
• System Integration: Connecting disparate sources like POS, ERP, cloud systems, and APIs into seamless data flows.
• Domain Expertise: Experience with Retail and POS data (or an ability to pick it up quickly).