AI Platform Engineer

About the Role

The AI Platform Engineer at Rubiscape is the architect of the infrastructure layer that powers RubiAI — our agentic AI copilot — and the shared AI services consumed by all six studios

• You will build the foundations: LLM gateway, RAG orchestration layer, prompt management system, vector store integrations, and the real-time inference bus that serves enterprise customers across SaaS, BYOC, and air-gap topologies. This is a high-ownership role at the core of Rubiscape’s differentiation as a unified Decision Intelligence Platform with 8 international innovation patents.

Key Responsibilities

· Design and operate Rubiscape’s LLM gateway: request routing, model failover, token budgeting, cost attribution, and rate-limit enforcement across multiple LLM providers and self-hosted models.

· Build and maintain the RAG pipeline infrastructure — document chunking, embedding generation, vector store management (Weaviate/Qdrant/pgvector), and retrieval orchestration using LangChain or LlamaIndex.

· Own the prompt management system: versioning, A/B evaluation, regression testing, and deployment of prompt templates used across RubiAI and studio co-pilots.

· Architect low-latency inference APIs that serve both synchronous (sub-200ms) and streaming (SSE/WebSocket) AI responses to the Rubiscape front-end and third-party integrations.

· Implement guardrails, content filtering, and PII redaction layers required for regulated-sector AI deployments in BFSI, healthcare, and government.

· Work with the MLOps team to integrate fine-tuned and quantised open-source models (Llama, Mistral, Phi) into the platform’s model registry and serving infrastructure.

· Establish platform-wide observability: trace AI requests end-to-end from user query through retrieval, LLM call, and response synthesis using OpenTelemetry and internal dashboards.

Nice to Have

· Experience building agentic AI systems with tool use, multi-step planning, and memory management.

· Knowledge of model compression techniques (quantisation, distillation, speculative decoding) for on-premises or edge AI deployment.

· Exposure to enterprise AI governance frameworks and responsible AI principles relevant to regulated sectors.

· Contribution to open-source LLM or RAG projects.

About Rubiscape

Rubiscape is India’s leading Decision Intelligence Platform, unifying data engineering, BI, machine learning, and agentic AI in a single governed platform. Built in Pune and trusted by Fortune 500 enterprises across BFSI, manufacturing, healthcare, and government. 8 international innovation patents. 10 Industry-Academia Labs & COEs. From BI to AI — Platform. Every Decision.

Requirements

Requirements

· 4+ years in platform, infrastructure, or backend engineering with at least 2 years focused on AI/LLM systems in production.

· Deep hands-on experience with LangChain or LlamaIndex for RAG pipeline construction and agentic workflow orchestration.

· Proficiency in Python for high-performance backend services and familiarity with async frameworks (FastAPI, asyncio).

· Practical experience with vector databases (Weaviate, Qdrant, Pinecone, or pgvector) at production query volumes.

· Strong understanding of LLM deployment options: OpenAI/Anthropic API, Azure OpenAI, vLLM, Ollama, and quantised model inference.

· Experience designing multi-tenant AI services with security isolation, audit logging, and usage metering.