Senior Data Engineer Cloud & Analytics

Role Summary

We are seeking a highly skilled

Senior Data Engineer

with strong expertise in cloud-based data platforms, big data processing, and modern data architectures. The ideal candidate will have hands-on experience in building scalable data pipelines, implementing lakehouse architectures, and enabling advanced analytics and machine learning use cases across enterprise environments.
Key Responsibilities

Data Engineering & Architecture
Design and implement scalable

data pipelines (ETL/ELT)

for batch and real-time processing.
Build and maintain

modern data platforms

using lakehouse architecture (Bronze, Silver, Gold layers).
Develop and optimize

data models (star/snowflake schemas)

for analytics and reporting.
Ensure high data quality, integrity, and governance across systems.
Cloud & Platform Management

Develop and deploy solutions on

Microsoft Azure and/or AWS ecosystems

• Work with services such as:
  Azure Data Factory, Azure Databricks, ADLS Gen2
  Azure SQL, Key Vault, Azure Dev

Ops
AWS S3, Redshift, EMR, Glue, Lambda

Implement secure, scalable, and cost-efficient data storage solutions.
Big Data & Processing

Develop large-scale data processing workflows using:
Apache Spark / Py

Spark
Kafka, Hive, Hadoop, Airflow

Optimize performance of distributed data processing systems.
Microsoft Fabric & Lakehouse (Preferred)

Implement

Microsoft Fabric-based data solutions

including:
Lakehouse architecture
Medallion design (Bronze/Silver/Gold)
Delta Lake optimization

Build Fabric pipelines and integrate with Power BI.
Data Integration & Migration

Lead data migration initiatives from legacy/on-prem systems to cloud platforms.
Integrate multiple data sources (SAP, Oracle, SQL Server, APIs, etc.).
Implement incremental data loading and performance optimization techniques.
Analytics & BI Enablement

Enable business intelligence and reporting using tools like:
Power BI, SSRS, Kibana, Grafana

Row-Level Security (RLS)

and data access controls.
Machine Learning & Advanced Analytics (Good to Have)

Support ML pipelines using frameworks such as:
Scikit-learn, Tensor

Flow, Py

Torch, Keras

Collaborate with data scientists for model deployment and integration.
Dev

Ops & Automation

Implement CI/CD pipelines using Azure Dev

Ops/Git.
Use Docker for containerization.
Automate data validation, monitoring, and deployment processes.
Monitoring, Security & Governance

Implement monitoring using Azure Monitor, Log Analytics, etc.
Ensure compliance with data governance frameworks (e.g., Purview).
Maintain security standards using Key Vault, IAM, and encryption mechanisms.
Required Skills & Qualifications

Technical Skills

Strong programming skills in

Python and/or Java
Advanced

SQL and data warehousing concepts
Hands-on experience with:
Spark / Py

Spark
ETL/ELT pipeline development
Data modeling and optimization
Cloud Expertise

Experience with

Microsoft Azure (preferred)

or AWS
Exposure to

Databricks, ADF, ADLS, Snowflake

is highly desirable
Big Data Technologies

Apache Spark, Kafka, Hive, Airflow
Tools & Technologies

Git, Docker, CI/CD pipelines
BI tools (Power BI preferred)
Experience Requirements

7-10 years

of experience in Data Engineering / Big Data
Proven experience in:
Designing scalable data architectures
Cloud data platform implementations
Data migration and transformation projects
Educational Qualifications

Bachelor's degree in

Computer Science, Engineering, or related field
Certifications in

Azure/AWS/Data Engineering

are a plus
Soft Skills

Strong problem-solving and analytical thinking
Ability to work in

Agile/Dev

Ops environments
Excellent stakeholder communication and collaboration skills
Ability to lead technical discussions and mentor junior engineers
Nice to Have

Microsoft Fabric

real-time streaming architectures
Knowledge of

AI/ML pipelines
Experience in

enterprise-scale data governance