Benchmark Structure

Most of the COTS^7.1 benchmark suites available for UNIX platforms and Linux in particular target issues such as:

• memory latency and bandwidth,
• I/O device latency and bandwidth,
• context switching,
• system call overhead,
• processor performance, etc.

These tests do not target the desired kernel mechanisms and are unsuitable for benchmarking LinSec.

It transpired that kernel compilation task is fit for the purpose. Each time Linux kernel is compiled gcc is executed on several thousand source files (actual number depends on the kernel configuration in place). Every new invocation of gcc means creating a new process and executing the binary via do_execve. In addition, every instance of gcc reads one or more source files and writes an object file thus triggering the fs access control mechanism several times per run.

The approach to benchmarking using the kernel compilation has three steps:

1. perform multiple kernel compilations on both platforms running LinSec and on platforms running clean Linux kernel,
2. use standard UNIX time utility to measure time spent in kernel space consumed by each of the compilations, and
3. produce benchmark result by averaging and comparing the observed kernel time consumed by kernel compilations on both platforms.

One aspect of LinSec setup used for the benchmarks having an explicit effect on the results is:

• Only one AD Group exists, the default one, containing only file system root as an element. This causes AD Access Control algorithm to exhibit worst case performance as it needs to iterate a number of times that is equivalent to the depth in the file system tree branch of the target file to find a match (file system root in this case). Existence of multiple AD Groups would increase the time needed for process creation linearly with the number of elements each AD Group, a process is member of, contains. This is considered negligible as it is performed only when first process owned by a user is created in the system (linsec_usr being retrieved from LinSec buffers) and when an executable is loaded to be executed.

Different possible capability related configurations do not influence benchmark results as, irrespective of the configuration, the same set of operations is always performed on them. An exception to this is reflecting capability group settings in a permitted capability set of a process whose owner is a member of the capability groups. However, analogous to the above AD case, the time taken for this operation is negligible.

Footnotes

... COTS^7.1

Common Off The Shelf.