Job Description
Job Title: Lead Azure GenAIOps / LLMOps Engineer
Experience: 10 – 14+ Years
Location: Remote / Hybrid (India)
Role Level: Lead / Principal Architect
Delphi Consulting with its headquarters in Dubai operates throughout the MENA area and the Indian Subcontinent and provides a diverse range of solutions that have a positive impact. We are a multicultural, customer-focused firm that excels at providing enduring results for businesses and communities.
We began our journey in 2013, and after a decade, we are one of the most dependable partners in the area for our B2B and B2C clients from a variety of industry backgrounds.
As a consulting firm, we work to develop business capabilities and offer sharp, actionable insights on consulting projects to aid customers in making sound, well-informed decisions. The organization’s guiding principle is “Trenchant Insights, Savvy Decisions”.
The team at Delphi Consulting has a combined total of three decades of deep and varied business expertise in the areas of Commercial (Sales & Marketing), Strategy, General Management, Project Management, etc.
Role Objective
We are looking for a Platform-First AI Engineer to lead the operationalization of Generative AI. You won’t just build prompts; you will build the enterprise-grade infrastructure that powers them. You will own the “Ops” in GenAIOps—bridging the gap between a successful “Proof of Concept” and a production-ready, multi-tenant AI platform using the Azure AI Foundry ecosystem.
Key Responsibilities
1. Platform Architecture & Orchestration
Agentic Frameworks: Architect and scale multi-agent systems using LangGraph, AutoGen, or Semantic Kernel. Implement persistent state management and deterministic fallback logic for autonomous agents.
Unified AI Gateway: Design and manage a centralized AI Gateway (using Azure APIM) to handle request routing, rate limiting, and cost-attribution across different business units.
Infrastructure-as-Code (IaC): Provision and manage Azure AI resources (Foundry, Search, CosmosDB) using Terraform or Bicep to ensure reproducible environments.
2. LLMOps & Observability
Advanced Tracing: Implement end-to-end distributed tracing for LLM calls using tools like Langfuse, Arize Phoenix, or LangSmith integrated with Azure Monitor/Datadog.
Evaluation Pipelines: Build automated “Evaluation-as-a-Service” pipelines. Use “LLM-as-a-Judge” patterns to score groundedness, relevance, and faithfulness before any code hits production.
Deployment Strategies: Manage the lifecycle of models (GPT-4o, Llama 3.x, Phi-4) including versioning, blue-green deployments, and A/B testing of system prompts.
3. Security & Responsible AI
- Enterprise Security: Enforce Zero Trust security for AI—implementing Private Links, Managed Identities, and Virtual Network isolation for all LLM traffic.
- Guardrails: Deploy and tune Azure AI Content Safety and custom jailbreak detection layers to prevent prompt injection and PII leakage.
- Governance: Monitor token usage and latency metrics to provide FinOps insights and prevent “runaway” agent costs.
Technical Skills Required
Primary Cloud: Expert-level Microsoft Azure (AI Foundry, Azure OpenAI, Azure ML).
Containerization: Deep experience with Azure Kubernetes Service (AKS), Docker, and KEDA for auto-scaling AI workloads. Frameworks: Mastery of LangGraph, LlamaIndex, and FastAPI for building high-concurrency AI backends.
Databases: Hands-on with Vector Stores— Azure AI Search, Pinecone, or Milvus.
DevOps: Proven experience with GitHub Actions or Azure Pipelines for ML/LLM CI/CD.
Soft Skills & Leadership
- Stakeholder Influence: Ability to explain the trade-offs between “Latency vs. Accuracy” to non-technical business leaders.
- Mentorship: Lead a team of 4–6 engineers, setting the technical standard for code reviews and architectural blueprints.
- Innovation: A track record of moving beyond “Simple RAG” into advanced patterns like GraphRAG and Multi-modal pipelines.
Qualifications
- B.Tech/M.Tech in Computer Science or related field (Ph.D. is a plus but not mandatory for this Ops-centric role). Azure Solutions Architect or Azure AI Engineer Associate certification preferred.