Skip to content

Optimize BlockHeaderDB - #56

Open
edmundnoble wants to merge 6 commits into
evm/mainfrom
push-vwszunmkpuvq
Open

edmundnoble wants to merge 6 commits into
evm/mainfrom
push-vwszunmkpuvq

Conversation

@edmundnoble

Copy link
Copy Markdown
  • Adds benchmarks for BlockHeaderDB.
  • Uses an iterator to implement branchEntries, avoiding repeated lookup overhead.
  • Takes advantage of new unsafe iterator tools in rocksdb-haskell to avoid copying key/value bytes as frequently from C to Haskell.
  • Some smaller micro-optimizations to HashSet usage.

Net results are something like 1.67x speedup.

Old benchmarks benchmarking block-header-db/getBranch/ancestors/trunk-tip (1000 blocks) time 10.94 ms (10.71 ms .. 11.18 ms) 0.998 R² (0.996 R² .. 0.999 R²) mean 10.50 ms (10.37 ms .. 10.64 ms) std dev 337.9 μs (272.8 μs .. 420.5 μs)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip)
time 2.091 ms (2.042 ms .. 2.127 ms)
0.997 R² (0.995 R² .. 0.998 R²)
mean 2.136 ms (2.104 ms .. 2.177 ms)
std dev 116.8 μs (93.00 μs .. 145.9 μs)
variance introduced by outliers: 38% (moderately inflated)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip, lower=fork-point)
time 501.2 μs (493.7 μs .. 509.9 μs)
0.998 R² (0.996 R² .. 0.999 R²)
mean 484.8 μs (479.0 μs .. 490.6 μs)
std dev 20.85 μs (17.25 μs .. 27.56 μs)
variance introduced by outliers: 36% (moderately inflated)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip-500)
time 2.200 ms (2.161 ms .. 2.244 ms)
0.997 R² (0.995 R² .. 0.999 R²)
mean 2.157 ms (2.134 ms .. 2.179 ms)
std dev 85.21 μs (65.99 μs .. 108.2 μs)
variance introduced by outliers: 24% (moderately inflated)

benchmarking block-header-db/seekAncestor/unique rank (tip - 500)
time 17.03 μs (16.95 μs .. 17.12 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 16.85 μs (16.76 μs .. 16.93 μs)
std dev 293.3 ns (246.3 ns .. 363.4 ns)
variance introduced by outliers: 14% (moderately inflated)

benchmarking block-header-db/seekAncestor/fork region (tip - 25)
time 908.2 μs (887.7 μs .. 935.4 μs)
0.995 R² (0.992 R² .. 0.998 R²)
mean 917.8 μs (905.5 μs .. 930.8 μs)
std dev 42.08 μs (35.87 μs .. 53.31 μs)
variance introduced by outliers: 35% (moderately inflated)

benchmarking block-header-db/fork/forkEntry (50 block fork)
time 835.3 μs (829.7 μs .. 841.0 μs)
0.999 R² (0.998 R² .. 0.999 R²)
mean 837.6 μs (829.3 μs .. 846.0 μs)
std dev 28.12 μs (22.05 μs .. 36.38 μs)
variance introduced by outliers: 24% (moderately inflated)

benchmarking block-header-db/fork/branchDiff (50 block fork)
time 848.6 μs (842.0 μs .. 855.5 μs)
0.999 R² (0.999 R² .. 0.999 R²)
mean 837.3 μs (829.8 μs .. 843.8 μs)
std dev 24.26 μs (19.71 μs .. 33.03 μs)
variance introduced by outliers: 18% (moderately inflated)

benchmarking block-header-db/fork/getBranchIncreasing (last 100 blocks)
time 994.8 μs (991.2 μs .. 998.2 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 984.3 μs (980.1 μs .. 988.0 μs)
std dev 13.33 μs (10.36 μs .. 16.68 μs)

New benchmarks benchmarking block-header-db/getBranch/ancestors/trunk-tip (1000 blocks) time 10.31 ms (9.864 ms .. 10.72 ms) 0.993 R² (0.988 R² .. 0.997 R²) mean 10.11 ms (9.959 ms .. 10.29 ms) std dev 451.7 μs (344.5 μs .. 579.0 μs) variance introduced by outliers: 19% (moderately inflated)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip)
time 1.394 ms (1.371 ms .. 1.417 ms)
0.999 R² (0.998 R² .. 1.000 R²)
mean 1.359 ms (1.353 ms .. 1.369 ms)
std dev 29.14 μs (21.02 μs .. 41.62 μs)
variance introduced by outliers: 10% (moderately inflated)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip, lower=fork-point)
time 300.2 μs (297.7 μs .. 302.6 μs)
0.999 R² (0.999 R² .. 1.000 R²)
mean 293.2 μs (291.4 μs .. 294.9 μs)
std dev 6.340 μs (5.258 μs .. 8.571 μs)
variance introduced by outliers: 14% (moderately inflated)

benchmarking block-header-db/getBranch/branchEntries/page (limit 250, maxRank=tip-500)
time 1.398 ms (1.393 ms .. 1.404 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.393 ms (1.385 ms .. 1.398 ms)
std dev 18.81 μs (15.65 μs .. 25.87 μs)

benchmarking block-header-db/seekAncestor/unique rank (tip - 500)
time 16.61 μs (16.54 μs .. 16.69 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 16.52 μs (16.45 μs .. 16.59 μs)
std dev 234.6 ns (180.5 ns .. 291.0 ns)

benchmarking block-header-db/seekAncestor/fork region (tip - 25)
time 867.7 μs (834.6 μs .. 886.4 μs)
0.996 R² (0.995 R² .. 0.998 R²)
mean 846.1 μs (832.7 μs .. 857.0 μs)
std dev 40.34 μs (34.30 μs .. 55.27 μs)
variance introduced by outliers: 38% (moderately inflated)

benchmarking block-header-db/fork/forkEntry (50 block fork)
time 815.6 μs (810.5 μs .. 820.4 μs)
0.999 R² (0.999 R² .. 1.000 R²)
mean 809.2 μs (804.1 μs .. 814.6 μs)
std dev 17.55 μs (14.63 μs .. 22.46 μs)
variance introduced by outliers: 11% (moderately inflated)

benchmarking block-header-db/fork/branchDiff (50 block fork)
time 826.2 μs (820.8 μs .. 830.4 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 821.2 μs (816.5 μs .. 825.6 μs)
std dev 14.91 μs (12.44 μs .. 21.01 μs)

benchmarking block-header-db/fork/getBranchIncreasing (last 100 blocks)
time 963.3 μs (960.2 μs .. 966.5 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 955.8 μs (950.5 μs .. 960.2 μs)
std dev 19.18 μs (13.47 μs .. 27.07 μs)

benchmarking cut-db/local: catch up one 399-block step from genesis
time 839.3 ms (790.9 ms .. 890.5 ms)
1.000 R² (0.998 R² .. 1.000 R²)
mean 980.0 ms (918.6 ms .. 1.075 s)
std dev 91.11 ms (18.69 ms .. 120.2 ms)
variance introduced by outliers: 22% (moderately inflated)

benchmarking cut-db/100ms latency: catch up one 399-block step from genesis
time 1.749 s (1.728 s .. 1.782 s)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.728 s (1.707 s .. 1.738 s)
std dev 20.08 ms (753.3 μs .. 24.14 ms)
variance introduced by outliers: 19% (moderately inflated)

The generic implementation (chainBranchEntries) resolves the branch bounds
and then traverses the branches by looking up the parent of each entry with
an individual point lookup. Each of those lookups seeks the ranked block
header table from its root and allocates a fresh set of read options, and
the upper bounds get resolved twice (once by chainBranchEntries and once
again by getBranch).

Since that table is keyed by (block height, block hash), all entries of a
block height are stored next to each other. This implements branchEntries
for BlockHeaderDb directly on top of a single table iterator:

  * branchStream walks the branch level by level (the iterator equivalent
    of getBranchResolved), yielding upperActive \ lowerActive at each rank
    and descending to the parents.
  * branchParents resolves the parents with a single iterSeek into that
    shared iterator and validates the seek via the value's block hash,
    which is cheaper than decoding the key with iterKey.
  * The bounds are resolved once, and the lower bounds are skipped
    entirely when there are no upper bounds.
  * reduceUpperBounds keeps the chainBranchEntries optimization of
    reducing the upper bounds to ancestors of the requested max rank,
    which bounds the depth of the traversal.

TreeDB now exports applyRankDesc so that the rank filter isn't duplicated.

Benchmarks (bench/Chainweb/BlockHeaderDBBench.hs: a 1000 block trunk with a
50 block fork), mean of before -> after:

  branchEntries/page (limit 250, maxRank=tip)             1.932 ms -> 1.351 ms  -30%
  branchEntries/page (limit 250, maxRank=tip, lower=fork)  453 us ->  297 us   -34%
  branchEntries/page (limit 250, maxRank=tip-500)          1.991 ms -> 1.373 ms -31%

The remaining benchmarks are unchanged, they don't use this code path:

  ancestors/trunk-tip (1000 blocks)                       10.08 ms -> 10.16 ms
  seekAncestor/unique rank (tip - 500)                    16.7 us  -> 17.3 us
  seekAncestor/fork region (tip - 25)                      870 us  ->  885 us
  forkEntry (50 block fork)                                823 us  ->  851 us
  branchDiff (50 block fork)                               837 us  ->  887 us
  getBranchIncreasing (last 100 blocks)                    974 us  ->  983 us

Adds Chainweb.Test.BlockHeaderDBBranch, which compares the new
implementation against the generic one for identical queries (streamed
items, reported count and Eos) over random trees, over random chains with
forks of random length at random heights, and over a large deterministic
grid on a 200 block trunk with a 150 block fork.
branchEntries db k l mir mar lower upper f
-- If the upper bounds are empty the result is empty, too. In that case
-- the lower bounds are not even resolved.
| HS.null upper = f (return (0, Eos True))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this test is necessary here. Uppers empty case will be caught later

-- Resolve the upper bounds. Keys that aren't in the database are
-- ignored.
uppers <- resolveBounds db _getUpperBound upper
if HS.null uppers

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not necessary here neither. Will be caught later.

iterSeek it (RankedBlockHash (int r) h)
iterValue it >>= \case
Just v
| view blockHash (_getRankedBlockHeader v) == h ->

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?
What are the cases where blockHash doesn't match with the DB key ?

Comment thread src/Chainweb/TreeDB.hs
| otherwise = do
lowers <- liftIO $ getEntriesHs db _getLowerBound lowerBounds

let mar = getMax $ fromJuste $

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I miss something ..
But why applying Just on every element; and then doing fromJuste ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM, we need this... because it's needed by foldMap ..

But I think we should manage properly the "Nothing" case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants