MLOps Testing Methodology

The IML4E MLOps testing methodology aims to provide a schema to systematically apply testing to MLOps processes and thus increase the quality of ML-based application by maintaining efficiency through targeted testing. It is a comprehensive framework that incorporates testing in all phases during developing, integrating, and operating ML-based systems by combining classical software engineering with data science activities to ensure the quality and reliability of ML-based systems.

The methodology divides the development lifecycle into several phases and associates items to be tested, acceptance criteria and test method to each of the phases. The phases are:

• Business Understanding and Inception: Identifying objectives, requirements, and understanding the data context.
• Experimentation and Training Pipeline Development: Evaluating data and modeling approaches, building PoC systems, and developing the training pipeline.
• Training: Creating and validating models using the developed pipeline.
• System Development and Integration: Integrating the ML model into the operational software environment.
• Operation and Monitoring: Monitoring the system in its operational environment to ensure ongoing performance and compliance.

None

MLOps engineers, data scientist, software developpers

None