Software Development Engineer in Test (API, UI & Agentic Systems)

Job Summary

We are looking for a highly motivated

Software Engineer 3

to help build the next generation of intelligent quality engineering platforms. In this role, you will design and develop scalable automation frameworks that validate modern distributed applications, APIs, web experiences, and AI-powered agentic workflows.

You will work closely with engineering, product management, and quality teams to define automation strategies, improve release confidence, and accelerate product delivery through innovative testing solutions. This position provides an exciting opportunity to work at the intersection of traditional automation engineering and emerging AI technologies.

Job Requirements

• 4+ years of software development or test automation experience.
• Strong programming skills in or more languages: Python, JavaScript/TypeScript
• Hands-on experience building automation frameworks from the ground up.
• Experience testing REST APIs using tools and frameworks such as Pytest, Playwright API Testing
• Experience automating web applications using: Playwright, Cypress
• Strong understanding of software development lifecycle and testing methodologies.
• Experience with Git, CI/CD pipelines, and cloud-native application testing.
• Familiarity with microservices architectures and distributed systems.

Preferred Qualifications

• Experience testing AI/GenAI applications and agentic systems.

• Familiarity with:

  • OpenAI, Azure OpenAI, Anthropic, or similar LLM platforms
  • LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel
  • MCP-based integrations
  • Vector databases and RAG architectures

• Experience building evaluation frameworks for:

  • Agent correctness
  • Hallucination detection
  • Workflow reliability
  • Prompt and model regression testing

• Experience with Kubernetes, Docker, and cloud platforms (AWS, Azure, or GCP).

• Knowledge of performance, security, and resilience testing.

• Exposure to observability tools and telemetry-driven quality engineering.

What Makes You Successful

• Passion for engineering quality rather than simply executing test cases.
• Strong analytical and debugging skills.
• Ability to drive automation strategy and influence engineering practices.
• Growth mindset with curiosity to explore emerging AI technologies.
• Excellent collaboration and communication skills.
• Ability to thrive in a fast-paced, innovation-driven environment.

What You'll Do

• Design and develop reusable automation frameworks for:

  • REST and GraphQL APIs
  • Modern web applications and user interfaces
  • AI-powered applications and agentic workflows

• Build scalable test infrastructure that supports functional, integration, regression, performance, and end-to-end testing.

• Create automated validation mechanisms for LLM-based systems, AI agents, multi-agent workflows, and Retrieval-Augmented Generation (RAG) applications.

• Integrate automation into CI/CD pipelines to enable continuous quality validation.

• Partner with developers and architects to establish shift-left testing practices and improve testability.

• Define quality metrics, coverage goals, and automation standards across teams.

• Investigate production issues, identify automation gaps, and improve test effectiveness.

• Contribute to the adoption of AI-assisted testing techniques and intelligent test generation.

Education

• Bachelor's or Master's degree in Computer Science, Engineering, or related discipline.
• IC - Typically requires a minimum of 4+ years of related experience.

Nice to Have

• Experience in storage, cloud infrastructure, data management, or enterprise platforms.
• Exposure to AI-driven test generation and autonomous testing solutions.
• Contributions to open-source testing frameworks or developer tooling.

• Experience testing AI/GenAI applications and agentic systems.
• Familiarity with:
• Experience building evaluation frameworks for:
• Experience with Kubernetes, Docker, and cloud platforms (AWS, Azure, or GCP).
• Knowledge of performance, security, and resilience testing.
• Exposure to observability tools and telemetry-driven quality engineering.
• Experience in storage, cloud infrastructure, data management, or enterprise platforms.
• Exposure to AI-driven test generation and autonomous testing solutions.
• Contributions to open-source testing frameworks or developer tooling.