56 lines
3.9 KiB
Markdown
56 lines
3.9 KiB
Markdown
# SBC Performance Test Kit
|
|
|
|
Testing procedure used by the DistriLab for benchmarking the performances of SBC boards
|
|
|
|
|
|
## Introduction
|
|
These benchmarking are done in the context of choosing a hardware platform for self hosting open source web applications.
|
|
The testing procedure mainly focusses on energy consuption and CPU/memory/disk speed.
|
|
|
|
The full test procedure during which the energy consumption is measured takes one hour.
|
|
During that period the CPU is heavily stressed for around 40 minutes, the system is idle for 10 minutes and the remaining 10 minutes are taken by the CPU/Memory/Disk tests (these tests can actually last less or more than 10 minutes depending on the performance of the machine being tested but the nergy consumption is done on the 60 first minutes after the styart of the test).
|
|
|
|
By default the disk speed is measured for sequential reads, sequential writes and a mix of random reads and writes with a 2 to 1 ratio (2 reads for 1 write) with block sizes of 4 kiB and 1024 kiB over 512 files of 128 MiB each (64 GiB total)
|
|
But the total disk space that is used is adapted depending on the target board specs. The goal is to use a disk space that is at least 2 times larger than the system memory size to avoid having the results biased by the system file cache.
|
|
|
|
This is a work in progress.
|
|
Do not hesitate to propose your improvements by creating issues or pull requests on this gitlab project.
|
|
|
|
## Credits
|
|
The performance measurements and system stressing are mainly done thanks to [sysbench](https://github.com/akopytov/sysbench)
|
|
|
|
[hdparm](https://sourceforge.net/projects/hdparm/) is also used to get disk information and to measure raw read disk speed
|
|
|
|
[lm_sensors](https://github.com/lm-sensors/lm-sensors) is used to measure CPU temperature
|
|
|
|
System information is gathered with [smartctl](https://www.smartmontools.org/), though the Linux /proc filesystem or using some other small command line utilities like lsblk, uname, grep, cut, ...
|
|
|
|
The script used to automate the tests is run with [bash](https://www.gnu.org/software/bash/) on [Debian GNU/Linux](https://www.debian.org/) or a derivative distribution.
|
|
|
|
|
|
## Testing procedure
|
|
|
|
1. Plug the power supply of the target machine through a Watt meter
|
|
1. Install a debian GNU/Linux based distribution on the target machine and make sure to upgrade to the latest version (`apt update && apt dist-upgrade && reboot`)
|
|
1. Login on the physical console or though ssh
|
|
1. Install git : `apt install git`
|
|
1. Clone this git repository : `cd ~ && git clone https://gitlab.com/distrilab/sbc-performance-test-kit.git`
|
|
1. Install dependencies : `sudo ~/sbc-performance-test-kit/bin/install-dependencies.sh`
|
|
1. Adjust the number of threads and/or the disk space used by the test for boards that have less or more than 4 CPU cores or less than 64 GB disk free space or more than 32 GB of RAM (some documentation needs to be written on this step)
|
|
1. Take note of the power consumption of the taget machine in idle state on the Watt meter
|
|
1. Reset the energy consumption of the watt meter
|
|
1. Run the full benchmarking script : `~/sbc-performance-test-kit/bin/run-full-benchmark.sh some-test-name`
|
|
1. Around 15 minutes after the start of the test, during the warm-up step, make note of the max power consumption
|
|
1. Exactly 60 minutes after the beginning of the test, that should have ended around the 50 minutes mark, take note of the total energy consumption on the Watt meter
|
|
1. Make a backup copy of the log file generated by the test in the current directory
|
|
|
|
If there are additional drives to test on the same board the quick test script can be used :
|
|
1. Change directory to the mount point of another disk that should be tested : `cd /some/where`
|
|
1. Run the quick benchmarking script : `~/sbc-performance-test-kit/bin/run-quick-benchmark.sh some-test-name`
|
|
1. Make a backup copy of the log file generated by the test in the current directory
|
|
|
|
## Sample test results and how to interpret them
|
|
|
|
**To Be Done**
|
|
|