Senior Test Engineer

ERP Consultant –  Senior Test Engineer, software testing, QA, Quality Assurance, .NET, Azure DevOps, CI/CD pipeline, API testing, Security testing, RAG, Model orchestration, hallucination detection, NIST AI RMF, EU AI Act, security validation, threat modelling, Microsoft Azure, £450 – £500 Inside IR35, Contract, On-site, Cambridge

You’ll be part of a Scrum team and will work closely with developers and architects to design effective validation strategies, automated testing frameworks, and AI-specific evaluation tools—with a builder mindset: rapid prototyping and continuous improvement with the agility of a start-up.

Key Skills & Experience:

• 12+ years of experience in software testing, QA, or validation roles for cloud-native applications using Microsoft and .NET technologies.
• Proficient in designing automated testing frameworks.
• Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized test environments.
• Strong understanding of API testing, performance profiling, and security testing (including OWASP top 10).
• Excellent problem-solving skills, with the ability to analyse complex technical challenges and propose scalable solutions.
• Experience working in Agile teams and collaborating across global R&D locations.
• Demonstrated ability to mentor junior team members, fostering a culture of continuous learning and innovation.
• Familiarity with LLM evaluation techniques, output scoring, and validation frameworks.
• Understanding of key concepts such as prompt engineering, RAG, model orchestration, and hallucination detection.
• Experience in testing for accuracy, relevance, and consistency of AI model predictions/generations.
• Defining performance metrics for AI services and testing against them.
• Awareness of AI safety, bias detection, and explainability techniques.
• Experience ensuring compliance with AI regulations and standards (e.g., NIST AI RMF, EU AI Act).

Main Responsibilities:

• Perform functional, performance, and security testing on cloud-native services deployed on Microsoft Azure.
• Design and implement automated test suites for APIs, service components, and AI pipelines.
• Automate the evaluation of AI system outputs to ensure accuracy, consistency, and safety of responses.
• Collaborate with developers and data scientists to establish service-level quality metrics and observability hooks.
• Validate services against AI regulatory frameworks and ensure traceability, fairness, and robustness in outcomes.
• Participate in threat modelling and security validation of exposed APIs and AI services.
• Provide feedback early in the lifecycle to reduce defects and improve design.
• Mentor junior testers, encourage continuous learning, and contribute to a culture of innovation.

Location: Cambridge

Candidates must be eligible to work in this country.