Reliable benchmark datasets for testing optimisation algorithms in changing environments

What is it about?

Many real‑world optimisation problems do not stay the same over time. Transport networks, production systems, logistics, and scheduling tasks often change as conditions evolve, requiring solutions that can adapt quickly. These are known as dynamic optimisation problems. While many algorithms have been proposed to solve them, it has been surprisingly difficult to compare their performance fairly. The main reason is that most existing benchmark datasets for dynamic optimisation are created using random (stochastic) changes. As a result, different researchers often test their algorithms on slightly different problems, making results hard to reproduce or compare. This paper addresses that gap by introducing a deterministic method for generating benchmark datasets for a well‑known combinatorial problem: the Dynamic Multidimensional Knapsack Problem (Dynamic MKP). Starting from existing static knapsack benchmarks, the proposed method generates a sequence of dynamic problem states in a predictable and repeatable way. Each new state evolves smoothly from the previous one, reflecting realistic changes while ensuring that the same dataset can be regenerated exactly by other researchers. Using this approach, the authors created 1,405 fully defined dynamic benchmark datasets, many of which include known optimal solutions. By providing open, repeatable datasets, this work makes it much easier for researchers to test, verify, and compare optimisation algorithms in dynamic settings.

Featured Image

Photo by Growtika on Unsplash

Photo by Growtika on Unsplash

Why is it important?

This work is important because reproducibility is a cornerstone of scientific progress, yet it has been largely missing from research on dynamic combinatorial optimisation. Until now, it has been almost impossible to verify published results or directly compare new algorithms with existing ones. What makes this research unique is its non‑stochastic dataset generation method. Instead of relying on random changes, each dataset state is produced deterministically from the previous state, ensuring full transparency and repeatability. The datasets are also extensible, meaning additional future states can be generated consistently without breaking comparability. By publishing both the datasets and the generation tools openly, this work establishes a common benchmark standard for Dynamic MKP research. It enables fair algorithm comparison, supports rigorous performance evaluation, and accelerates progress in areas such as evolutionary algorithms, swarm intelligence, and other metaheuristic methods used in dynamic environments.

Perspectives