Abstract: We test the performance of Bitcoin Core by successfully conducting 35 initial block downloads (IBDs) and recording the amount of time the node takes to synchronize with the network. We used software releases in the period spanning from 2012 to 2019. The results show a considerable and consistent improvement in the performance of the software, but also a high degree of variance. Even with the latest computer hardware, older versions of Bitcoin struggled to get past the pickup in transaction volume which occured in the 2015 to 2016 period. Therefore we conclude that without the software enhancements, an initial synchronization today could be almost impossible.
Figure 1 – Bitcoin Initial Block Download Time (Days) – Average Of 3 Attempts
(Source: BitMEX Research)
(Notes: Synchronization up to block 602,707. Further details in the notes below)
Overview
To test the performance of Bitcoin Core during the initial synchronization, we successfully conducted 35 initial block downloads (IBDs) and recorded the amount of time each attempt took. The results are shown in Figure 1 above and illustrate that there was a significant improvement in speed when Bitcoin Core 0.12.0 was released in February 2016, due to the upgrade from OpenSSL to libsecp256k1 for signature verification. Libsecp256k1 was built specifically for Bitcoin. Since then, the improvements in speed were much slower and due to the high variance in IBD times, the improvements are only clearly visible after multiple attempts. However, even after Bitcoin Core 0.12.0 was released in February 2016, a small gradual improvement in performance is still visible after each software release from Bitcoin Core 0.13.0 to Bitcoin Core 0.19.0.1.
Of course, IBD time is only one metric, and there are plenty of other angles and considerations that one can use to evaluate the performance and capabilities of Bitcoin Core. While the IBD time may not be the perfect or complete measure of overall software performance, it is highly resource-intensive and therefore potentially a good metric to benchmark.
This report follows on from two previous experiments:
- In November 2018 Jameson Lopp conducted a similar exercise, however that analysis focused on independent implementations, while this analysis focuses on older versions of Bitcoin Core (or just “Bitcoin”, as some of the older software pre-dates the name “Bitcoin Core”).
- Sjors Provoost also conducted this experiment in July 2017, although Sjors provided data for fewer synchronization attempts.
Full Results and Raw Data
Figure 2 – Bitcoin Initial Block Download Time (Days)
(Source: BitMEX Research)
(Notes: Synchronization up to block 602,707, further details in the notes below)
System Specification & Other Notes
MacBook Pro (64 bit)
|
Linux VPS (64 bit)
| |
| OS |
macOS Mojave (10.14)
|
Ubuntu 18.04.3
|
| Processor |
6 Core Intel i9 2.9GHz
|
8 Core Intel Xeon
|
| Memory |
32GB
|
32GB
|
| Storage |
1 TB Flash Storage
|
640GB Flash Storage
|
| Internet Downstream Bandwidth |
62Mb/s
|
2,000Mb/s
|
| Internet Upstream Bandwidth |
20Mb/s
|
400Mb/s
|
| IBD ended at height |
602,707
|
602,707
|
| Bitcoin.conf settings |
assumevalid=0 dbcache=24000 maxmempool=500 | |
Full Table of Results
| Client | Client release date |
Sync Time (Hours)
|
Machine
|
| Bitcoin Core 0.19.0.1 |
24/11/2019
|
11.4
|
MacBook Pro
|
| Bitcoin Core 0.18.1 |
20/07/2019
|
10.4
|
MacBook Pro
|
| Bitcoin Core 0.17.0 |
03/10/2018
|
17.7
|
MacBook Pro
|
| Bitcoin Core 0.16.0 |
28/02/2018
|
18.5
|
MacBook Pro
|
| Bitcoin Core 0.15.0 |
14/07/2017
|
21.1
|
MacBook Pro
|
| Bitcoin Core 0.14.0 |
08/03/2017
|
16.4
|
MacBook Pro
|
| Bitcoin Core 0.13.0 |
17/08/2016
|
24.7
|
MacBook Pro
|
| Bitcoin Core 0.12.0 |
17/02/2016
|
15.8
|
MacBook Pro
|
| Bitcoin Core 0.11.2 |
10/11/2015
|
53.3
|
MacBook Pro
|
| Bitcoin Core 0.10.0 |
12/02/2015
| 81.2 |
MacBook Pro
|
| Bitcoin Core 0.9.0 |
18/03/2014
|
85.1
|
MacBook Pro
|
| Bitcoin Core 0.8.6 |
09/12/2013
|
Abandoned
|
MacBook Pro
|
| Bitcoin Core 0.19.0.1 |
24/11/2019
|
13.6
|
Linux
|
20/07/2019
|
15.9
|
Linux
| |
03/10/2018
|
13.3
|
Linux
| |
28/02/2018
|
18.8
|
Linux
| |
14/07/2017
|
17.9
|
Linux
| |
08/03/2017
|
25.1
|
Linux
| |
17/08/2016
|
15.8
|
Linux
| |
17/02/2016
|
14.8
|
Linux
| |
10/11/2015
| 46.0 |
Linux
| |
12/02/2015
| 77.2 |
Linux
| |
18/03/2014
| 78.9 |
Linux
| |
09/12/2013
| 98.5 |
Linux
| |
24/11/2019
| 14.0 |
Linux
| |
20/07/2019
|
13.7
|
Linux
| |
03/10/2018
|
16.0
|
Linux
| |
28/02/2018
|
18.2
|
Linux
| |
14/07/2017
|
17.9
|
Linux
| |
08/03/2017
|
17.0
|
Linux
| |
17/08/2016
| 21.9 |
Linux
| |
17/02/2016
| 17.1 |
Linux
| |
10/11/2015
| 44.1 |
Linux
| |
12/02/2015
| 82.2 |
Linux
| |
18/03/2014
| 82.1 |
Linux
| |
09/12/2013
| 72.6 |
Linux
|
(Source: BitMEX Research)
Analysis of the Results
As Figure 2 above illustrates, even when conducting the IBD with the same software and with a machine with the same specification, there is considerable variance in the reported times.
Figure 3 – IBD time vs Client Release Date (Days) – Average Time of 3 Attempts
(Source: BitMEX Research)
(Note: For the Bitcoin 0.8.6 client, the results above are an average of only 2 attempts)
Figure 3 above indicates that the performance of the software improved incrementally with each software release, with the exception of the strong performance of Bitcoin Core 0.12.0. However, despite the apparent clear trend in the above chart, the large variance and in IBD times on each attempt could indicate there is considerable uncertainty. One may need more sample data before drawing strong conclusions about improvements in performance since 2016. It is possible the variation is primarily caused by issues in the Bitcoin P2P network or the internet connection and therefore a good area of further study may be to compare the re-scan speed, the time taken to fully verify the blockchain once it has already been downloaded.
Bitcoin Core 0.12.0 performs well in the above analysis. This may be because Bitcoin Core 0.12.0 has libsecp256k enabled, but does not validate signatures for transaction inputs where the witness is segregated (Segregated Witness). Therefore Bitcoin Core 0.12.0 does not validate all the signatures in the blockchain post August 2017, giving the client somewhat of an “unfair advantage”. However this advantage may also apply to Bitcoin Core 0.13.0, despite this node not appearing to be an outlier. Of course all the versions prior to Bitcoin Core 0.12.0 have that same “unfair” advantage, but this is dwarfed by the disadvantages of using OpenSSL.
Syncing The Client Up To Its Release Date
The below chart (Figure 4) illustrates the time it takes to synchronize a client, up until the block height on the date the software was released.
Figure 4 – IBD Time Up To Client Release Date (Days)
(Source: BitMEX Research)
(Note: Data for the nodes running on Linux only. Bitcoin Core 0.19.0.1 only synced up to height 602,707)
The chart shows that the trend was reasonably flat from Bitcoin Core 0.8.6 to Bitcoin Core 0.14.0, at that point the scalability improvements could not match the impact of time progressing and the blockchain increasing in height, and the chart shows an upward trend. Unfortunately the rate of software improvement has been reduced in recent years, perhaps as the low-hanging fruit improvements have already been made. Higher transaction volume may have also contributed to this. Future scalability improvements may be a lot more challenging, and even if the 4 million unit blockweight limit is maintained, IBD times may continue to increase going forwards, despite further software upgrades and moderate increases in hardware performance.
The Failed IBD Attempts
We did successfully compile and run versions of Bitcoin prior to 0.8.6, however, the synchronization became slow when the node reached the 2015 to 2016 period. The pre-0.8.6 nodes, such as 0.7.0, did successfully get past the apparent hardfork in 2013, by manually changing the lock limit, however 2015 proved too challenging due to the increased transaction volume, and the node stopped processing blocks. We tried restarting the node, which did help push it forwards, but then it only got stuck again. We then even tried running Bitcoin Core 0.7.0 on our brand new local machine, with 64 GB of RAM and 8 Intel i9 processors, however the node was still unable to get past 2016. With many of the scaling parameters involved being non-linear, one cannot simply throw more hardware at the problem.
On occasions when the nodes got stuck on a block and we re-started, we abandoned the synchronization after 4 restart attempts. For Bitcoin Core 0.8.6 on the MacBook Pro, the synchronization was abandoned when the leading block was in 2016. Although this is slightly disappointing, no restarts were required for the remaining 35 successful synchronizations.
Conclusion
Other than the fact that the BitMEX IT department should be more cautious when issuing BitMEX Research with MacBook Pros, the data illustrates the significant scalability enhancements which have been delivered over the last seven years. The transition to libsecp256k being the most significant improvement. The large reductions in IBD times and the inability of old nodes to fully synchronize indicates that if it were not for these scalability enhancements, by now Bitcoin would be essentially dead, even if users had the highest specification hardware available. The data also shows that technological innovation is unlikely to keep up with the growing blockchain going forward and that IBD times will increase.