Senior / Lead / Staff Data Engineer ( Locals and only USC and GC)

Senior / Lead / Staff Data Engineer

Location: Chicago, IL (Hybrid – 2 Days Onsite, Local Candidates Only)

Duration: 12+ Months (Multi-Year Project)

Work Authorization: Open to All Authorized Candidates

Interview Process: 2–3 Rounds

Job Overview

We are looking for experienced Senior, Lead, and Staff Data Engineers to join Epsilon's Attribution & Forecasting Engineering team. The ideal candidate will design, build, and optimize large-scale distributed data platforms while developing high-performance batch and streaming data pipelines in secure cloud environments.

You will work closely with Product, Data Science, Security, Privacy, and Platform Engineering teams to deliver scalable, privacy-preserving data solutions supporting advertising attribution, measurement, forecasting, and analytics.

Required Skills

• 5+ years (Senior), 8+ years (Lead), or 10+ years (Staff) of Data Engineering experience
• Strong hands-on experience with Apache Spark
• Expert-level Scala programming
• Advanced SQL for large-scale datasets
• Strong Python development for automation and data pipelines
• Experience building large-scale Batch & Streaming Data Pipelines
• Experience processing TB-scale data
• Strong knowledge of Data Warehousing and Lakehouse architectures
• Experience with Databricks, Delta Lake, or equivalent platforms
• Cloud experience with AWS (preferred), GCP, or Azure
• Experience with workflow orchestration tools such as Apache Airflow, AWS Step Functions, or similar
• Experience with CI/CD, Git, automated testing, and production support

Data Governance & Privacy (Must Have)

Candidates should have experience working with secure, privacy-focused data platforms including:

• Processing sensitive datasets containing PII or regulated data
• Privacy-preserving analytics
• Secure data-sharing environments / Clean Rooms
• Data Governance
• Access Controls
• Data Classification
• Data Lineage
• Encryption & Key Management
• Zero Trust Architecture
• Tokenization & Pseudonymization
• Aggregated Data Export
• Cloud-native Security

Responsibilities

• Design, develop, and optimize distributed data processing solutions using Spark, Scala, SQL, and Python
• Build scalable batch and streaming ETL pipelines
• Develop cloud-native data platforms on AWS/GCP/Azure
• Design and maintain trusted data environments and secure data-sharing platforms
• Implement data governance, lineage, privacy controls, and access policies
• Collaborate with Product, Data Science, Security, and Platform teams
• Build monitoring, observability, and alerting solutions
• Optimize performance of large-scale distributed systems
• Support architecture reviews and engineering best practices
• Mentor junior engineers (Senior), lead technical initiatives (Lead), or drive architecture and engineering strategy (Staff)

Preferred Skills

• Databricks
• Delta Lake
• Unity Catalog
• AWS Clean Rooms
• AWS Glue Data Catalog
• Apache Atlas
• OpenLineage
• Docker
• Kubernetes
• ELK Stack
• Grafana
• OpenTelemetry
• Advertising, Attribution, Measurement, Retail Media, or Marketing Analytics domain experience
• Experience working with Security, Privacy, Risk, or Compliance teams

Ideal Candidate

We're looking for engineers with strong distributed systems experience who can build highly scalable, cloud-native data platforms while ensuring security, governance, and privacy of sensitive datasets. Candidates should possess excellent communication skills, thrive in Agile environments, and be passionate about delivering enterprise-scale data engineering solutions.