Example usage:

$ python scripts/bench.py -b graphblas -f pagerank -d amazon0302

Codecov Report

Base: 72.67% // Head: 72.67% // No change to project coverage 👍

Coverage data is based on head (8c9d530) compared to base (83bfa40).
has no changes to coverable lines.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #39   +/-   ##
=======================================
  Coverage   72.67%   72.67%           
=======================================
  Files          69       69           
  Lines        2569     2569           
  Branches      475      475           
=======================================
  Hits         1867     1867           
  Misses        531      531           
  Partials      171      171           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Here's the help:

$ python scripts/bench.py --help
usage: bench.py [-h] [-b {graphblas,networkx,scipy}] [-t TIME] [-n N] -d DATA [-j] -f FUNC [--extra EXTRA]

Example usage: python scripts/bench.py -b graphblas -f pagerank -d amazon0302

options:
  -h, --help            show this help message and exit
  -b {graphblas,networkx,scipy}, --backend {graphblas,networkx,scipy}
  -t TIME, --time TIME  Target minimum time to run benchmarks
  -n N                  The number of times to run the benchmark (the default is to run according to time)
  -d DATA, --data DATA  The path to a mtx file or one of the following data names: {amazon0302, as-Skitter,
                        caidaRouterLevel, citationCiteseer, coAuthorsDBLP, coPapersCiteseer, coPapersDBLP, dblp-2010,
                        email-Enron, preferentialAttachment, soc-Pokec, web-Google}; data will be downloaded if necessary
  -j, --json            Print results as json instead of human-readable text
  -f FUNC, --func FUNC  Which function to benchmark
  --extra EXTRA         Extra string to add to the function call

And here example output:

$ python scripts/bench.py -b graphblas -f pagerank -d amazon0302
Downloading data/amazon0302.mtx from https://sparse.tamu.edu/MM/SNAP/amazon0302.tar.gz
===========================================================
Backend = graphblas, function = pagerank, data = amazon0302
-----------------------------------------------------------
Number of runs: 256
first:  12.1 ms
median: 10.2 ms
mean:   10.6 ms
stdev:  1.39 ms
min:    9.99 ms
max:    18.6 ms
===========================================================

and

$ python scripts/bench.py -b networkx -f pagerank -d amazon0302 -n 5 --json
{"backend": "networkx", "function": "pagerank", "data": "amazon0302", "n": 5, "first": 3.455940272193402, "median": 3.430417278315872, "mean": 3.419718873221427, "stdev": 0.028473827004063253, "min": 3.3903320101089776, "max": 3.455940272193402}

This is as far as I can take things for now--I'm going to merge. Comments/feedback welcome--especially in the form of PRs! What other datasets would be good to benchmark against?

I'll figure out where to post benchmark processing scripts and results later.

Hmm, we should probably list the available functions to benchmark in the help message.