criterion performance measurements

overview

want to understand this report?

encode/encodePacker

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.475718184529927e-7 5.630737048423427e-7 6.040089647478438e-7
Standard deviation 2.214220597354752e-8 8.030647204350723e-8 1.6511351186628752e-7

Outlying measurements have severe (0.9477921226781341%) effect on estimated standard deviation.

encode/encodeSimplePokeMonad

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.3305099726139537e-7 4.347009773597342e-7 4.3634824044244875e-7
Standard deviation 4.867015605591515e-9 5.4928491922097154e-9 6.257644692324075e-9

Outlying measurements have moderate (0.11633384132469499%) effect on estimated standard deviation.

encode/encodeStore

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.338118522053943e-7 4.3510081696570493e-7 4.373095241353874e-7
Standard deviation 3.3854072585705934e-9 5.691399002031444e-9 1.0608157297476373e-8

Outlying measurements have moderate (0.12464469369636823%) effect on estimated standard deviation.

encode/encodeCereal

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7264419766685444e-5 1.7541586649162307e-5 1.787655594787388e-5
Standard deviation 9.147132864483453e-7 9.863277386263642e-7 1.0246597328621164e-6

Outlying measurements have severe (0.6438218072696295%) effect on estimated standard deviation.

encode/cbor

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.352772696000648e-6 5.411668575036211e-6 5.494111866866681e-6
Standard deviation 1.7994057204088378e-7 2.3483522700802193e-7 2.7854693611383373e-7

Outlying measurements have severe (0.5500081503875457%) effect on estimated standard deviation.

decode/decodePacker

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.3180302773361786e-6 2.344191039701862e-6 2.3682589316904026e-6
Standard deviation 6.10073821387604e-8 8.526172955312548e-8 1.0812460783728875e-7

Outlying measurements have moderate (0.484779250740743%) effect on estimated standard deviation.

decode/decodeSimplePeek

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.182929162795577e-7 9.307187604093999e-7 9.46487804298666e-7
Standard deviation 3.9651855346054795e-8 4.480066648674503e-8 4.8798599226371805e-8

Outlying measurements have severe (0.6465054408776154%) effect on estimated standard deviation.

decode/decodeStore

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.326492834753665e-7 9.460970147109965e-7 9.616465195054183e-7
Standard deviation 4.6114913205472467e-8 5.112935517742105e-8 5.3971932109644835e-8

Outlying measurements have severe (0.7008241823489774%) effect on estimated standard deviation.

decode/decodeCereal

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.667284540243614e-6 4.729597047284985e-6 4.764701791274515e-6
Standard deviation 8.253412888519859e-8 1.5290953458387076e-7 2.1785903885213245e-7

Outlying measurements have moderate (0.4074172074555851%) effect on estimated standard deviation.

decode/cbor

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.593086486070497e-6 6.68059531797972e-6 6.813501348497385e-6
Standard deviation 2.772203459135689e-7 3.4321247788179703e-7 3.9506631469366606e-7

Outlying measurements have severe (0.6310142617530713%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.