GitHub Action for Rust Project Test and Benchmark

I've been learning Rust as posted previously.

Sorted String Table in Rust Implement SSTable to learn Rust and for fun Recently, I've been learning Rust. Then, I started implementing sorted string table in Rust in order to learn Rust deeply and also understand SSTable architecture, which is used by underlying storage engines in various modern database softwares. I'm going to describe what I implemented as my first

blog.petitviolet.net

2020-09-15 [Rust, DB]

This project has a small amount of tests as well as benchmark. To enable running tests and benchmark to validate how recent changes affect both functionality and performance, I decided to use GitHub Actions as the project's CI platform.

How To Run Rust Project Benchmark

Before describing about GitHub Actions, describe about how to run benchmark for Rust project. Nightly Rust supports Benchmark tests, however, I used rather bheisler/criterion.rs, since it seems to have better visualization using gnuplot or plotters.

To enable benchmark test using criterion, update Cargo.toml as following.

[dev-dependencies]
criterion = "0.3"

[[bench]]
name = "sst_bench"
harness = false

Then, create ./benches/sst_bench.rs file to be able to run benchmark on the file. For running benchmark test on SSTable that I created and introduced in the last post.

#[macro_use]
extern crate criterion;

use criterion::Criterion;
use rsstable::sst::SSTable;

fn test_sstable_performance(c: &mut Criterion) {
    let mut sst = SSTable::new("./test_bench", 1);

    c.bench_function("sstable get from memtable", |b| {
        // prepare
        sst.insert("key", "value");
        // benchmark
        b.iter(|| {
            sst.get("key")
                .expect("failed to get value");
        })
    });

}

criterion_group!(benches, test_sstable_performance);
criterion_main!(benches);

Now, we can run cargo bench to run benchmark as implemented.

GitHub Actions Configuration

As the next step and main topic of this post, describe how to set up GitHub Actions to run tests and benchmarks.
You can find a bunch of actions in GitHub Marketplace. I chose actions-rs/rust-toolchain to provision Rust toolchains in addition to actions-rs/rust-cargo for enabling run cargo command in CI environment.

Actual GitHub Actions configuration files that I've written are available in my GitHub repository: petitviolet/rsstable/.github/workflows

name: Rust Test and Benchmark

on:
  push:
    branches:
      - master
  pull_request:
    types: [opened, synchronize]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1

      - name: Cache cargo registry
        uses: actions/cache@v1
        with:
          path: ~/.cargo/registry
          key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}

      - name: Cache cargo build
        uses: actions/cache@v1
        with:
          path: target
          key: ${{ runner.os }}-cargo-build-target-${{ hashFiles('**/Cargo.lock') }}

      - uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
          override: true

      - uses: actions-rs/cargo@v1.0.1
        with:
          command: check

      - name: test
        run: cargo test

      - name: bench
        run: cargo bench

      - uses: actions/upload-artifact@v1
        with:
          name: rsstable_${{ github.sha }}_bench
          path: ./target/criterion

As you can see the configuration, it runs:

• caching based on Cargo.toml
• use stable rust toolchain
• run cargo check, cargo test, and cargo bench in order
• upload benchmark results as artifact to GitHub, then it'll be available in GitHub actions history

Note that this action runs only for ubuntu so that it doesn't make it sure the Rust project runs all of environments. If you'd like to do it, using Cross-compilation capability will work.