Job Title : Lead Data Engineer/Architect – GenAI & Databricks
Job Location: Chicago, Illinois, United States
Study Level : Bachelor's Degree
Skill Required : Data-Databricks Data platform with Spark, Python/Scala/Java, Kafka, HDFS, Hive, NoSQL.
Primary Skill : DataBricks, PySpark, AWS
Experience : 10+years
Job Duration:6 Months
Job Description
Lead Data Engineer
Experience 10+ Years to work at the client place in Chicago
Key Responsibilities
- We are looking for a highly skilled Data Engineer – Tech Lead with strong expertise in AWS, Databricks, PySpark, ETL, and Delta Live Tables (DLT).
- The candidate should be capable of understanding business requirements, interacting directly with clients, proposing scalable data architectures, and driving solution discussions to build client confidence.
- Lead design and development of GenAI and Agentic AI solutions aligned with business objectives.
- Understand client requirements and translate them into scalable technical solutions.
- Drive architecture discussions, solution design, and technical strategy with stakeholders and clients.
- Collaborate with cross-functional teams to deliver AI-driven products and platforms.
- Provide technical leadership, mentoring, and guidance to development teams.
- Ensure best practices in AI/ML architecture, integration, security, and scalability.
- Design and implement scalable data engineering solutions on AWS and Databricks.
- Develop and optimize ETL/ELT pipelines using PySpark and DLT.
- Lead architecture discussions and recommend best-fit data solutions.
- Collaborate with business stakeholders and clients to understand requirements and translate them into technical solutions.
- Drive technical conversations, solution presentations, and client communication.
- Mentor engineering teams and ensure best practices in data engineering and cloud architecture.
- Ensure performance, scalability, security, and reliability of data platforms.
- Lead design and development of GenAI and Agentic AI solutions aligned with business objectives.
- Strong hands-on experience in AWS, Databricks, PySpark, ETL, and Delta Live Tables (DLT).
- Good understanding of modern data lakehouse architecture.
- Experience in data modeling, orchestration, and pipeline optimization.
- Strong communication and stakeholder management skills.
- Ability to lead client discussions and provide solution-oriented approaches.
- Experience in Agile delivery and team leadership.
- Strong experience in Generative AI, LLMs, AI Agents, and orchestration frameworks.
- Understand client environment and their pain points.
- Articulate client concerns, take part & propose solutions, make POC, and win client confidence.
- Experience in AWS - GenAI, AgenticAI.
