TLRD

In my case, the match was faster, with a median of 114.70 ns, while the Hashmap had a median of 123.43 ns. Nevertheless, I decided to move on with my Hashmap implementation. If you want to understand a bit more about my experiment, let’s dive a bit deeper.

The function

My goal was to see how much speed I would lose implementing a function that used to be an match implementation to a Hashmap, this function is a simple parser for Nun-db commands, and the question is all about the first seek.

Match Implementation

• The old implementation was a simple match on the first part of the command and would look like the next:

Hashmap Implementation

• The new implementation used a hash map to store the keys as the commands and a function to execute the rest of the parser like the next:

Device I used for the benchmark

I used my own Makbook (on battery) next, a image of its specs

Chip Apple M1 chip

• 8-core CPU with 4 performance cores and 4 efficiency cores
• 8-core GPU
• 16-core Neural Engine

Benchmark details

The Lib

To execute this benchmark, I used the crate called Criterion. I liked them because they were simple to use and Statistics-driven.

All I needed to do was:

1. Include their lib in the Cargo as a dev dependency

   [dev-dependencies]
   #...
   criterion = { version = "0.4", features = ["html_reports"] }
   

2. Create a new benchmark session in the Cargo.toml

   [[bench]]
   name = "nundb_benchmark"
   harness = false
   

3. Create a new folder called benches

4. Add the code I would like to benchmark as it follows

Report details

Median

Let’s start by analyzing the medial, since it will demonstrate that 50% of my calls would be taken up to that value to execute. Next, we can see how the match implementation median was ~8.73ns faster. That is a small price to pay, given the facilities that having these commands on a Hashmap will provide me.

Match Implementation median plot

Hashmap Implementation median plot

Probability Density Function

The PDFs were my plot of choice for these analyses because it shows where the median point is and how bad the outliers are. In this case, I am more worried about the upper bound of the plot and making sure it would not be too bad when comparing. Both had terrific stability of time with the match implementation Severe outliers placing ~118ns while in the Hashmap implementation. They were around ~126.5, almost the same difference as the median, so still a small price to pay.

Match Implementation PDF plot

Hashmap Implementation PDF plot

Reproducing the results locally

I am sorry I did not create a sandbox just for this experiment, but You can also run the same experiment on your computer by:

1. Cloning the Nun-db repo from GitHub, it is MIT and open source. Few free to leave us an issue there if you find any issues running it.
2. Executing the command cargo bench
3. The result in HTML format will be created in the folder target/criterion/, but they are also presented in the console after running it.

Conclusion

When I started this experiment, I was sure the match implementation would be faster. My question was more about how much faster it would be. I had wanted for a long time to store all commands in a collection so it would be easy to list them and automate tasks like finding out which ones are not documented yet, for example. But I was curious about how much of a performance loss I would have to pay for that gain. With this experiment, I could estimate that it would cost me ~9ns, which means 1ms each ~111.111 parsers - a small price to pay. The great learning from this was getting to know the lib Criterion and how simple it is to use. I am definitely using it in my next experiments.