What is it about?

To check if software tests work, researchers intentionally introduce fake bugs into code and see if the tests detect them. Multiplex is a flexible LLM-powered toolkit that lets researchers easily swap, test, and compare different LLM methods for creating these realistic software bugs.

Featured Image

Why is it important?

It is important for mutation testing researchers to have a tool like multiplex because it: - Enables Fair Comparisons: Allows direct benchmarking of competing LLM mutant generation strategies (such as hazard-guided, mask-and-predict, and unguided techniques) using identical configurations. - Eliminates Code Re-implementation: Saves research time and prevents implementation errors by removing the need to rewrite platform-bound research prototypes. - Supports Multi-Language & Multi-Model Studies: Easily targets multiple languages (e.g., Java and Python) and interfaces with both cloud-based (e.g., GPT-4.1) and locally hosted LLMs (e.g., via Ollama). - Increases AI Transparency: Automatically archives all intermediate prompting steps, syntax pre-screening outcomes, and full test failure logs for reproducible empirical studies.

Perspectives

To advance LLM-based software testing, researchers need to know which mutant generation techniques actually produce the most effective and realistic code defects. We created Multiplex to serve as a standardised workbench—enabling the research community to prototype, share, and fairly compare mutant generation strategies without having to rebuild the underlying test infrastructure every time. By building Multiplex, we can support researchers in prototyping their techniques without coming up with a completely new implementation.

Megan Maton
University of Sheffield

Read the Original

This page is a summary of: multiplex: A Modular LLM-Based Mutation Framework, October 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3832783.3834617.
You can read the full text:

Read

Contributors

The following have contributed to this page