petitviolet blog

    GitHub Action for Rust Project Test and Benchmark



    I've been learning Rust as posted previously.

    Sorted String Table in Rust Implement SSTable to learn Rust and for fun Recently, I've been learning Rust. Then, I started implementing sorted string table in Rust in order to learn Rust deeply and also understand SSTable architecture, which is used by underlying storage engines in various modern database softwares. I'm going to describe what I implemented as my first

    This project has a small amount of tests as well as benchmark. To enable running tests and benchmark to validate how recent changes affect both functionality and performance, I decided to use GitHub Actions as the project's CI platform.

    How To Run Rust Project Benchmark

    Before describing about GitHub Actions, describe about how to run benchmark for Rust project. Nightly Rust supports Benchmark tests, however, I used rather bheisler/, since it seems to have better visualization using gnuplot or plotters.

    To enable benchmark test using criterion, update Cargo.toml as following.

    criterion = "0.3"
    name = "sst_bench"
    harness = false

    Then, create ./benches/ file to be able to run benchmark on the file. For running benchmark test on SSTable that I created and introduced in the last post.

    extern crate criterion;
    use criterion::Criterion;
    use rsstable::sst::SSTable;
    fn test_sstable_performance(c: &mut Criterion) {
        let mut sst = SSTable::new("./test_bench", 1);
        c.bench_function("sstable get from memtable", |b| {
            // prepare
            sst.insert("key", "value");
            // benchmark
            b.iter(|| {
                    .expect("failed to get value");
    criterion_group!(benches, test_sstable_performance);

    Now, we can run cargo bench to run benchmark as implemented.

    GitHub Actions Configuration

    As the next step and main topic of this post, describe how to set up GitHub Actions to run tests and benchmarks.
    You can find a bunch of actions in GitHub Marketplace. I chose actions-rs/rust-toolchain to provision Rust toolchains in addition to actions-rs/rust-cargo for enabling run cargo command in CI environment.

    Actual GitHub Actions configuration files that I've written are available in my GitHub repository: petitviolet/rsstable/.github/workflows

    name: Rust Test and Benchmark
          - master
        types: [opened, synchronize]
        runs-on: ubuntu-latest
          - uses: actions/checkout@v1
          - name: Cache cargo registry
            uses: actions/cache@v1
              path: ~/.cargo/registry
              key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
          - name: Cache cargo build
            uses: actions/cache@v1
              path: target
              key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}
          - uses: actions-rs/toolchain@v1
              toolchain: stable
              override: true
          - uses: actions-rs/cargo@v1.0.1
              command: check
          - name: test
            run: cargo test
          - name: bench
            run: cargo bench
          - uses: actions/upload-artifact@v1
              name: rsstable_${{ github.sha }}_bench
              path: ./target/criterion

    As you can see the configuration, it runs:

    • caching based on Cargo.toml
    • use stable rust toolchain
    • run cargo check, cargo test, and cargo bench in order
    • upload benchmark results as artifact to GitHub, then it'll be available in GitHub actions history

    Note that this action runs only for ubuntu so that it doesn't make it sure the Rust project runs all of environments. If you'd like to do it, using Cross-compilation capability will work.