Staff Test Engineer/ Senior Staff Test Engineer (AI) - Bangalore

Engineering

8-12 Years

India

The pioneers of software product industry in India, we are a technology and innovation led company simplifying the lives of small and medium businesses over the last three decades. With our cutting-edge technology, we have had the privilege to reach over 2.5 million business globally and cater to more than 7 million users in over 100 countries. We are passionate about the SMEs’ growth and aim to deliver the best for their business through our product innovation and excellence. Our teams develop some of the most ingenious solutions that suit the unique requirements of millions of businesses across the globe. With a market share of 75% in India, we have a strong foothold in the Middle East and are also serving customers in SAARC & APAC, Africa, and North America. Our robust network of 28000+ partners help us deliver unmatched customer experience in sales, support, and services globally.
With a sole purpose of Making Everyone Who Touches Tally, Happier, we aim to be the technology fabric that drives the economic growth of the world, by 2030.

Over the years, we have nurtured a value-based culture that let individuals follow their lead and support them in their growth journey with us. We value honesty and integrity, prioritize a people-first culture, pursue excellence, and drive impactful innovation with simplicity.
Our vision at Tally is to ensure that all employees get access to equal opportunities, with decisions grounded on performance, merit, competence, and potential. We are dedicated to fairness and transparency in our policies. By fostering diversity and equality, we strive to eliminate all forms of discrimination. We are committed to an inclusive leadership where our leaders ensure that our people are empowered to be at their best, professionally, and personally. We take great pride in our work culture which has helped become a proud member of the elite Kincentric Best Employers Club!

Our Engineering centre is located in Bangalore while our sales offices and partners are spread across the country and specific regions outside India. Our Engineering team consists of highly talented engineers who live a purpose and dream to develop the software that will accomplish our goal ‘To be the technology fabric that drives the economic growth of the world’.

To build this network of businesses Tally likes to build its own technology stack to deliver the required products. Major components of the stack are highlighted below.

• Operating system: We use a trimmed-down version of the Linux
• Database system: An Object-oriented database written by Tally to support single view, replication, distributed and multi-tenancy.
• Web server & app server: We shall write our own hosting platform that can handle millions of connections per server.

Engineering @ Tally is responsible for the Design, Development & Testing of all the delightful and flawless products that we release for our customers. We at Engineering do deep technology innovation to deliver unique experiences and capabilities at scale for simplifying business operations across sectors and segments.

We believe in

• Innovate: Build our tech stack with purpose-driven engineers to craft robust, scalable products that power economic growth.
• Design for the future: Lay the foundation of technology that connects businesses, enabling seamless growth and transformation.
• Engineer the Core: Create resilient, and secure software systems that form the backbone of our mission to drive global prosperity.

We are looking for a Staff Test Engineer / Senior Staff Test Engineer with strong technical expertise and proven experience in AI/LLM-based projects. The candidate will own quality end-to-end, from requirements and architecture discussions through testing, release, and production validation. The role requires a combination of AI testing expertise, strong programming skills, test strategy, risk management, stakeholder communication, and technical leadership. The candidate should be able to independently drive quality for complex products and influence engineering and product decisions.

AI / LLM Testing

• Hands-on experience working with AI/ML/LLM-based products or projects.
• Strong understanding of LLMs, Generative AI, Prompt Engineering, and AI-based workflows.
• Experience defining evaluation strategies for AI systems, including accuracy, consistency, hallucination, robustness, context handling, and response quality.
• Experience validating AI agents, tool/function calling, structured outputs, or RAG-based applications is preferred.
• Ability to identify AI-specific edge cases and define appropriate test data, benchmarks, and quality metrics.
• Exposure to LLM evaluation, model comparison, and AI-assisted testing is an advantage.

Technical & Quality Engineering

• Strong programming skills in Python, JavaScript, or similar, with the ability to build test automation and utilities.
• Strong understanding of API testing, databases/SQL, system architecture, networking, distributed systems, and debugging.
• Experience designing scalable automation frameworks and integrating automated tests into CI/CD pipelines.
• Exposure to performance/scalability testing using tools such as JMeter, Gatling, or k6.
• Good understanding of cloud platforms, containers, logging, monitoring, and deployment architecture.

• Quality Ownership: Own QA activities across the complete project lifecycle—from requirements and design to release and production validation.
• Test Strategy & Planning: Define test strategy, scope, test coverage, automation approach, environments, test data, and release criteria.
• Testing Execution: Drive functional, API, integration, regression, performance, and NFR testing based on project requirements.
• Risk Management: Proactively identify technical, functional, integration, and release risks, define mitigation plans, and communicate them early.
• Release Readiness: Take ownership of release readiness and provide data-driven quality recommendations.
• Production Support: Analyse production issues, drive root-cause analysis, and implement preventive quality improvements.