Individual language features and library functions. Testing them in isolation focuses on implementation quality of language constructs, like foreach loops and library function like aos_to_soa(). This category of tests allow covering ISPC functionality quite thoroughly, but doesn't address interplay between different features and compiler optimizations.
Feature combinations and non-obvious optimization effects. It's impractical to cover all combinations, so this kind of tests requires understanding of the specific effects triggered by features combination. An example might be memory operations nesting in non-trivial loops.
More complex cases inspired by real algorithms. Any example reduced from real-life applications and representing either a known performance problem, or a code snippet known to be critical for performance of the real code. When contributing this kind of examples, please mind the license of the original code.

The benchmarks might be used for comparison of different platforms and regression tracking. In this case ISPC implementation is sufficient.

For some features it also makes sense to compare with alternative implementations in other languages. In such case C++ (or other) implementation needs to be provided.

Building and running

Use -DISPC_INCLUDE_BENCHMARKS=ON in your CMake configuration flags to enable building benchmarks. If --recurse-submodules was not used during repo cloning, CMake will fetch Google Benchmark submodule. By default ISPC built from sources will used for building benchmarks. If you need to use pre-build ISPC binary, you can turn off building ISPC by setting -DISPC_BUILD=OFF and provide ISPC in your PATH environment variable.

You can use CMake options BENCHMARKS_ISPC_TARGETS and BENCHMARKS_ISPC_FLAGS to set specific target or ISPC compilation switches. For example, -DBENCHMARKS_ISPC_TARGETS=avx512skx-i32x8,avx2-i32x8 -DBENCHMARKS_ISPC_FLAGS="-O3 --woff".

To run benchmarks, you need to execute them individually. They will be located in benchmarks folder of your install location.

TODO

Individual language features and library functions.

Loops: foreach loops vs ISPC for loops vs C++ for loops. Iteration limits divisible and not divisible by programIndex need to be tested.
Nested loops: multi-dimensional foreach loops vs separate foreach loops vs for loops. Different axis for vectorization need to be tested.
foreach_unique performance.
foreach_active performance.
memory operations: gather/scatter, vector load/store, masked load/store.
aos_to_soa(), soa_to_aos() functions.
other stdlib function.
select() vs cond ? t : f.

Feature combinations and non-obvious optimization effects.

strided loads/stores, which combine to continuous loads/stores.
dot-product and cross-product of short vectors.
other code gen artifacts, triggered by seemingly similar code, but yielding different results.