Platform Engineer

Join Delphi - Where Innovation meets transformation

At Delphi, we believe in creating an environment where our people thrive. Our hybrid work model empowers you to choose where you work—whether it's from the office, your home, or a mix of both so you can prioritize what matters most. We are committed to supporting your personal goals, family, and overall well-being while driving transformative results for our clients.

We welcome exceptional talent from anywhere across the globe. Interviews and onboarding are conducted virtually, reflecting our digital-first mindset.

Rooted in the region, we specialize in delivering tailored, impactful solutions in Data, Advanced Analytics and AI, Infrastructure, Cloud Security, and Application Modernization. Whether it’s enabling predictive analytics, transforming operations with automation, or driving customer engagement with intelligent platforms, we are the trusted partner for organizations ready to embrace a smarter, more efficient future.

About the Role: LLMOps Engineer or Azure AI Platform Engineer - LLMOps.

Location:

India Remote

Experience - 10–14+ years overall, operating at Lead / Principal level

Employment Type - Full time

We are seeking a Lead Azure GenAIOps / LLMOps Engineer to design, build, and operate a secure, observable, governed Azure GenAI platform that can be reused by multiple product and business teams. This role is not focused on model training or fine-tuning. Instead, it owns LLM operationalization, governance, observability, safety, cost control, and platform reliability across enterprise environments. You will work at the intersection of AI Platform Engineering, LLMOps, Cloud Architecture, and Dev

Sec

Ops, partnering closely with application teams, security teams, and cloud platform teams.

Key Responsibilities

1. Azure GenAI Platform Ownership

• Architect and operate a shared, multi-tenant Azure GenAI platform using:

o Azure OpenAI

o Azure AI Foundry (must-have)

• Define reference architectures for RAG, agents, and LLM-powered apps.

• Decide and document usage patterns across:

o AKS, App Service, and Azure ML

(Candidate should have strong experience with at least one; platform design should

support multiple runtimes.)

2. LLM Runtime, Agent & Tool Governance

• Implement AI Gateway / Azure API Management for:

o Model routing and abstraction

o Throttling and quota enforcement

o Authentication and authorization

• Govern agent runtimes, including:

o Tool access control

o Permissions and identity boundaries

o Authentication, audit logging, and traceability

• Define MCP server / tool governance standards:

o Function calling approvals

o Tool versioning

o Change control and auditability

3. CI/CD, Environment Promotion & Configuration Management

• Build reusable pipeline templates for GenAI workloads.

• Define environment promotion models across:

o DEV → NON-PROD → PROD

• Enforce:

o Git-based prompt, agent, and config versioning

o Approval workflows

o Rollback and hotfix strategies

• Manage golden datasets and regression test suites for:

o Prompts

o Agents

o RAG pipelines

4. Observability, Quality & Reliability

• Implement LLM observability using tools such as:

o Langfuse

o Open

Telemetry

o Azure Monitor / Application Insights

• Enable:

o Prompt & response tracing

o Retrieval tracing

o Tool-call tracing

o Token usage tracking

o Cost and latency dashboards

• Define and enforce SLIs/SLOs for GenAI workloads.

• Own incident response, on-call readiness, rollback, and DR testing.

5. RAG Quality & Evaluation

• Implement continuous monitoring for:

o Retrieval quality

o Chunk quality

o Citation quality

o Grounding score

o Hallucination regression

• Automate evaluation gates in CI/CD pipelines.

• Maintain baseline and golden datasets to detect quality drift.

6. GenAI Safety & Responsible AI Controls

• Implement enterprise safety controls:

o Prompt shields

o Jailbreak detection

o Groundedness checks

o Content moderation

o PII / PHI masking

• Design human-in-the-loop review and escalation workflows for risky outputs.

• Collaborate with security teams on policy definitions (ownership is shared, not siloed).

7. Security, Networking & Identity (Design Ownership)

• Design secure Azure architectures using:

o Private networking

o Private Endpoints

o Managed Identities

o Azure Key Vault

o VNet isolation

• Clarify responsibility boundaries:

o Own GenAI platform security design

o Collaborate with core security / platform teams for enterprise controls

• Heavy Dev

Sec

Ops controls (SBOM, image signing, admission checks) are good-to-have unless

mandated by environment.

8. Cost, Routing & Performance Optimization"

• Implement:

o Model routing and fallback strategies

o Throttling and quota management

• Optimize cost by:

o Model

o Application

o User

o Environment

o Tenant

• Build token and cost dashboards for leadership visibility.

9. Compliance & Audit Automation

• Automate compliance evidence generation:

o Policy enforcement proofs

o Audit trails

o Access logs

o Promotion records

• Reduce reliance on manual audit documentation.

Core Deliverables (Expected Outcomes)

• Enterprise-grade Azure GenAI reference architectures

• Reusable CI/CD pipeline templates

• Secure AI Gateway patterns

• Governed agent and tool frameworks

• Observability dashboards and alerts

• Regression test suites and golden datasets

• Platform onboarding guides and standards

Required Skills

Azure & AI Platform

• Azure OpenAI, Azure AI Foundry (mandatory)

• AKS or App Service or Azure ML (deep expertise in at least one)

• Azure API Management / AI Gateway patterns

• Private networking, Managed Identity, Key Vault

LLMOps & Governance

• RAG architectures and evaluation

• Prompt, agent & config lifecycle management

• Model routing, fallback, and throttling strategies

• Multi-tenant GenAI platform experience (strongly preferred)

Automation & Engineering

• Python, Bash, YAML

• REST APIs and SDK-based automation

• CI/CD using Azure Dev

Ops or Git

Hub Actions

• Terraform or Bicep

Observability & Reliability

• Langfuse, Open

Telemetry, Azure Monitor, App Insights

• SLIs/SLOs, incident management, production support

Good to Have

• Semantic Kernel

• Microsoft Agent Framework

• Lang

Chain, Agno

• FastAPI

• Advanced Dev

Sec

Ops controls (SBOM, image signing, admission checks)

• Azure security and architecture certifications

What We Offer:

At Delphi, we are dedicated to creating an environment where you can thrive, both professionally and personally. Our competitive compensation package, performance-based incentives, and health benefits are designed to ensure you're well-supported. We believe in your continuous growth and offer company sponsored certifications, training programs, and skill-building opportunities to help you succeed. We foster a culture of inclusivity and support, with remote work options and a fully supported work-from home setup to ensure your comfort and productivity. Our positive and inclusive culture includes team activities, wellness and mental health programs to ensure you feel supported.