Chapter 18 of the book The Blocksize War is published below. The full book is available on Amazon. As a reminder, 50% of any profits from physical book sales will be donated to Médecins Sans Frontières, a charity that provides medical assistance to people affected by conflict, epidemics, disasters, or exclusion from healthcare.*
* Note: Applies to sales up until the earlier of i. the death of the author and ii. January 2031
The Blocksize War – Chapter 18 – New York Agreement
On May 22, 2017, there was a meeting in New York arranged by Barry Silbert, of Digital Currency Group (DCG), aimed at resolving the conflict. Jihan Wu was in attendance. This resulted in yet another agreement. The following document referred to as the New York Agreement (NYA), was published. It read:
We agree to immediately support the following parallel upgrades to the bitcoin protocol, which will be deployed simultaneously and based on the original Segwit2Mb proposal:
Activate Segregated Witness at an 80% threshold, signaling at bit 4
Activate a 2 MB hard fork within six months
We are also committed to the research and development of technical mechanisms to improve signaling in the bitcoin community, as well as to put in place communication tools, in order to more closely coordinate with ecosystem participants in the design, integration, and deployment of safe solutions that increase bitcoin capacity.
We welcome all companies, miners, developers, and users to join us and help prepare bitcoin for the future.
The group of signed companies represents a critical mass of the bitcoin ecosystem. As of May 25, this group represents:
58 companies located in 22 countries
83.28% of hashing power
5.1 billion USD monthly on chain transaction volume
20.5 million bitcoin wallets
Separately, as of May 24, the following companies have committed to provide technical and engineering support to test and support the upgrade software, as well as to assist companies with preparing for the upgrades:
Abra | BitClub Network | Bitcoin.com | BitFury | BitGo | Bitmain | BitPay | Blockchain | Bloq | BTCC | Circle | Ledger | RSK Labs | Xapo
If you wish to dedicate technical and engineering support from your team, please let us know and we will include you in the list above.
The agreement was based on an earlier proposal in March 2017, from Bitcoin developer and researcher Sergio Lerner. The idea was to do both SegWit and a non-witness hardfork blocksize increase to 2 MB. Speaking to people close to Barry, this was said to be a compromise, with some people wanting SegWit and others wanting a hardfork, therefore now everyone could get what they wanted. It was explained to me that Barry was concerned that the network had reached a stalemate and that something had to be done to move forwards. The agreement itself was said to have been drafted by DCG employee Meltem Demirors. This document had given Jihan the opportunity to save face, a methodology of dealing with the potential threat of the UASF, just like the Litecoin agreement a month earlier.
The number and significance of the signatories to the agreement was very impressive. In total, there were 58 signatories including Gavin Andresen, Roger Ver’s Bitcoin.com, Jihan Wu’s Bitmain and Brian Armstrong’s Coinbase. Although many (33 of the signatories) were DCG portfolio companies, there was widespread support outside of this group too, from many of the exchanges and mining pools. As such, many observers in the space concluded that the blocksize issue had now finally been resolved and that a successful implementation of the agreement was almost inevitable. This agreement was a major coup and achievement for the larger blockers, at a time when they were weak. It looked like a major turning point in the conflict. The larger blockers were almost down and out, but now they were suddenly in the ascendancy again.
However, the large blockers were not totally happy with the agreement. After all, it included SegWit, something they were not too keen on. But it did appear to do something they wanted, which was to “fire Bitcoin Core”. This is how Roger Ver justified his support. He claimed that, although he did not like the agreement itself, at least it would get rid of Bitcoin Core. Some large blockers appeared suspicious of the agreement, and those on the more extreme end of the large blocker camp opposed it. The agreement committed to activate SegWit before the hardfork, with the hardfork activating “within six months”. The agreement said the hardfork and softfork would be “deployed simultaneously”, but that the softfork would activate first. Some large blockers were concerned that step one could occur and then the signatories could bail on the agreement, before enacting step two.
As for the small blockers, there was no representation of them in New York, and their views were not reflected in the agreement whatsoever. There were no sentences with double meanings or conflicting statements; it appeared to have been written entirely by the large blocker side. Notably, the agreement made no reference to the idea that Bitcoin users control the protocol or that support of the users was necessary before changing the protocol rules. The NYA did not even pay lip service to the idea that users should have a say. It was pitched as the large corporates in the space imposing the rules on the users in a top-down manner. If Bitcoin were to operate like that, it would undermine the core value proposition of the currency. The NYA signatories did not appear to consider it important to lobby and persuade users before activating the fork. Instead, it felt like a threat or an ultimatum.
As well as undermining the core value proposition of Bitcoin, this was also bad tactics. Bitcoin users wanted to feel in control and did not like being told what to do. Therefore, the NYA felt like a repeat of Bitcoin XT, Bitcoin Classic and Bitcoin Unlimited. They had made the same mistakes again, except this time with much more industry support. One thing of note, however, is that several very significant and growing names in the industry were not signatories of the NYA. Most notable by its absence was Bitfinex, perhaps the most economically significant company at the time. Local Bitcoins (the largest P2P exchange at the time), Poloniex, BitMEX and mining pool Slush were also notably absent.
The easiest section of the agreement for the small blockers to pick apart was where it said “Activate Segregated Witness at an 80 percent threshold, signalling at bit 4”. This made little sense, as SegWit activated using bit 1, not bit 4. Miners could flag using bit 4, however this would not actually activate SegWit. Apparently, based on speaking to somebody in attendance at the meeting, Jihan had insisted on this point, perhaps because he had been pressured for months to flag support for SegWit using bit 1 and he did not want to cave into the small blocker demands. However, activating SegWit using this bit was not possible and therefore it was not exactly clear what would happen.
Almost immediately after the NYA was published, on May 22, 2017, Bitcoin and mining software developer James Hillard proposed a solution to this bit 4 dilemma, called BIP 91:
I would like to propose an implementation that accomplishes the first part of the Barry Silbert proposal independently from the second:
“Activate Segregated Witness at an 80% threshold, signaling at bit 4”
The goal here is to minimize chain split risk and network disruption while maximizing backwards compatibility and still providing for rapid activation of segwit at the 80% threshold using bit 4.
By activating segwit immediately and separately from any HF we can scale quickly without risking a rushed combined segwit+HF that would almost certainly cause widespread issues.
James proposed that SegWit would activate in two phases, with two softforks. A first softfork would activate using the 80 percent miner signalling threshold mentioned in the NYA and would make signalling for SegWit via bit 1 compulsory. This would then activate the final softfork, SegWit itself. This also made the upgrade compatible with BIP 148 (the UASF), because this also made signalling for SegWit via bit 1 mandatory. In actual fact, BIP91 was co-written by both James Hillard and Shaolinfry, the author of BIP 148. Eventually, a client was released, called Segsignal, which was Bitcoin Core patched to support BIP 91. BIP 91 was quite an aggressive and quick upgrade mechanism, activating if 269 blocks flagged support in 336 block signalling windows, an 80 percent threshold.
As for the client implementing the NYA in full, it quickly emerged that this would be called BTC1 and the lead developer would be Jeff Garzik, who had proposed to Satoshi to increase the blocksize limit a few weeks after it was first imposed in 2010. Jeff was asked to make BTC1 compatible with BIP 91 and activate SegWit via bit 1, arguing that, from a technical standpoint, activation via bit 4 was a nonsense. Initially, Jeff refused, although his reasoning appeared somewhat muddled.
On May 29, an email from the CEO of BitGo, Mike Belshe, was leaked. It contained a plan and timeline for the NYA. It should be noted that, although BitGo was listed in the NYA as one of the companies providing technical support for the upgrade, they were not signatories to the agreement themselves. The agreement even stated at the very bottom: “Note: BitGo was erroneously included in this initial published list. This has since been corrected.” From speaking to some BitGo staff, it is my understanding that they had asked for the company to be removed, thinking that the custodian and payment processor should instead stay neutral and support both sides of the split. In any event, the leaked email contained a proposed timeline, including the alpha release of the software, the launch of the testnet and then for signalling to begin by July 21.
Many in the Bitcoin community were furious to discover that development and plans for the NYA client were taking place in secret and not on a public mailing list. Bitcoin was supposed to be an open system, subject to open scrutiny. A secret mailing list driving the rollout of consensus change was considered antithetical to Bitcoin. However, the reason it was conducted in private was clear: if this was done in public, the small blockers would no doubt pick holes in what they were doing, making their proposal look weak. SegWit was extremely complicated and an alternative group was trying to roll it out with a very limited understanding of it. With the benefit of hindsight, conducting much of the process behind closed doors was a mistake, as the lack of scrutiny appeared to cause more errors.
At the end of May 2017, the pressure on Jeff caused by the incompatible SegWit activation, via bit 4, was immense. The small blockers and the Dragons’ Den participants realised that this was a major flaw in the BTC1 client and, if they persuaded Jeff to make this change and adopt BIP 91, SegWit may finally activate on Bitcoin. After activation occurred, the small blockers could then work on stopping the second part of the NYA, the hardfork. Social media was full of comments about how disruptive and uncooperative Jeff was being by refusing to adopt BIP 91, and his inbox was presumably full of similar demands and accusations from across the technical community.
Finally, on June 5, 2017, Jeff caved in under the pressure and included BIP 91 in his BTC1 client. The small blockers had now got their way; the NYA client now implemented the UASF. Inside the Dragons’ Den, there were excited celebrations.
The activation logic for the hardfork also changed at this point; the hardfork in the BTC1 client was now said to be scheduled for activation three months after SegWit activated (if, indeed, SegWit was to ever activate), rather than six months later. However, I was unable to understand how the code implemented the activation logic, and there was considerable confusion around the specifics of the hardfork activation timing. This three-month period was half the six-month period mentioned in the NYA, which was originally implemented in early versions of the BTC1 client. From speaking to some BTC1 insiders, I ascertained that the reason for this change was apparently to give people less time to back out of the hardfork once SegWit had activated, which would help ensure the hardfork occurred. This seemed to me to be the wrong approach that could actually make the hardfork phase more difficult to pull off, as now proponents of the hardfork had a shorter period in which to persuade users to upgrade (three months instead of six).
As for the BTC1 client, it appeared to be full of bugs, even after implementing BIP 91. Jeff may not have understood SegWit and made some mistakes with the peer-to-peer layer of SegWit, which required further input to fix. Also, remarkably, as someone pointed out in an email on June 14, the BTC1 client did not even implement the hardfork blocksize limit increase. The client kept the four million-unit weight cap, which would have prevented a hardfork and ensured the blocksize limit never increased. It appears as if Jeff never understood the new SegWit limit: remember, he had thought SegWit had two limits. Only after this was pointed out, post the release of the first version of BTC1, did it actually implement a hardfork. It was proving too difficult for the NYA supporters to double something that they did not understand.
Jeff was also under considerable pressure to add replay protection to the BTC1 client, however he opposed this. The idea was not that BTC1 would create a new coin, but that there was to only be one coin after the upgrade, and for the original rules chain to somehow die due to overwhelming support for the new chain. The Ethereum split of 2016 was ancient history now and most large blockers seemed to have forgotten about it. Therefore, in the mind of the NYA supporters, replay protection was not necessary. However, this was a repeat of the same arguments which had been going around and around in circles for years; some people thought the original rules chain may prevail and therefore replay protection was necessary. On June 14, Sergio Lerner, upon whose proposal the NYA was based, threw his weight behind replay protection. By this point, Jeff was under tremendous pressure from all sides:
There are two group of people which have two different visions for Bitcoin. None of these visions is “wrong”. One group values more things like decentralization, lack of government, censorship resistance, anonymity. This group thinks that Bitcoin will transform our world in 20-30 years. To reach this goal, it’s of utter importance to stick to those values. There is no rush. The other group values more things like reaching one billion users in the next 5 years, or serving real unbanked users today, even if that requires a political agreement now. Both visions have their merits. But they are incompatible. Replay protection gives a chance to each of these “bitcoiners” to fully push their own vision. Both visions can co-exists.(sic)
On June 16, 2017, the miners had yet another roundtable meeting. Almost all the major mining pools were in attendance. At the meeting, the miners agreed to support the NYA.
The first release of BTC1 did not have wipeout protection and, as was explained earlier in this book, pushing for a contentious hardfork without wipeout protection is like going to war having deliberately tied one’s hands behind one’s back. By May 2017, after almost two years of fighting for harforks, Jihan Wu had finally come to this realisation and had pushed for wipeout protection. Jihan had supported wipeout protection as early as May 12, 2017:
Since it is a very significant consensus rule change which has been debated for 4 years, we can add another consensus rule at and ONLY at the forking height, the size of the block HAS TO be bigger than 1,000,000 Byte. It is a very simple and straight forward way to protect re-org.
As the pressure on Jeff mounted, and it appeared that the NYA hardfork might be more contentious than its promoters had originally thought, he agreed to add wipeout protection. Jihan had encouraged Jeff to add this feature into BTC1 and, on June 20, Jeff complied. Rather than merely allowing blocks to be larger than 1 MB after the hardfork, BTC1 now required the first block to be more than 1 MB, a basic form of wipeout protection.
Requested by Bitmain and BU to implement an anti-wipeout feature; myself and other WG members concurred via the wider rationale of creating a more predictable network upgrade.
…
The traditional HF sequence is that the network fork occurs *on or after* the HF rule change, when the miner creates the first block larger than 1M. It is proposed to tighten this such that the HF block *must be* larger than 1M on the block when the rule change occurs. This makes the event more predictable by guaranteeing the fork occurs specifically at block X.
This feature was implemented poorly and, on July 11, 2017, the BTC1 testnet split into two. It appeared that someone mined testnet blocks 50 times faster than expected, activating the hardfork early. Then, due to a bug in how the ‘greater than 1 MB’ rule was implemented, the newer versions of BTC1, which required the first block to be more than 1 MB, split onto a different chain from the older BTC1 client. This was because the first block was not more than 1 MB as there were insufficient transactions. This bug and split was again exploited by the small blockers to indicate that BTC1 was a weak and buggy client. The BTC1 team defended themselves by indicating that this is what the testnet was for. However, this did illustrate a point: BTC1 was attempting to change the consensus rules of Bitcoin in a rushed and arbitrary schedule and, in this short time, had released multiple clients, which were already incompatible with themselves due to bugs and other last minute changes.
By early July, around 80 to 95 percent of miners (by hash power) were including the letters “NYA” in the blocks they mined, and the NYA looked like it was in a commanding position. However, almost none of the users were running BTC1 or even Segnet (the first part of the NYA only); adoption was near zero. At the same time, the major exchanges I spoke to were not running BTC1, Segnet or the BIP 148 client; they were just running Bitcoin Core. The outlook therefore seemed highly uncertain.
In fact, the miners and mining pools I spoke to were not running BTC1 either, despite the flags for the NYA. In mid-July 2017, I spoke to people at two of the mining pools who signed both the NYA and the new mining roundtable agreement in June 2017. Just to explain, if a miner was running BTC1, they should by default at this point flag both bit 4 and bit 1; once the first softfork activated, bit 1 would become mandatory and activate SegWit. These miners told me that they did not trust the BTC1 client and were therefore running Segnet or Bitcoin Core. Those running Bitcoin Core were manually including the NYA flag and bit 4 and/or bit 1. I was informed that this was top secret and that this was shared with me in confidence; in public, it was vital to show support for BTC1 and the NYA, the miners explained.
On July 20, Jihan tweeted that Bitmain was running BTC1; except, by this point, he also said that Bitmain had modified the software to remove the bit 1 flag. Jihan appeared keen to hold out until the last possible moment before activating SegWit, ensuring that it did not activate before voting for it became mandatory.
Bitmain is running btc1 software but we modify to only vote on bit 4 only at this stage.
The flagging was clearly a mess, miners were flagging all over the place, misleading users about what they were actually running. Pools were signalling support for the NYA without running BTC1, for example. Bitmain was also one of the worst offenders when it came to false flagging. For instance, Antpool (Bitmain’s mining pool) actually flagged bit 4 before the alpha release of BTC1, indicating that it was using a client that didn’t yet exist. Even in July 2017, Bitmain was still flagging support for Bitcoin Unlimited, when this node had not implemented BTC1 or SegWit, and therefore their flags were contradictory.
A few days later, towards the end of July 2017, Bitmain finally flagged for bit 1, just a few days before the deadline. The small blockers were ecstatic. After a grueling campaign, more than 10 months after SegWit was released, the largest miner in the space had finally flagged support for it. Most small blockers had believed that this would never happen. SegWit activation now finally looked likely.
The 80 percent threshold for BIP 91 was then finally achieved, and it locked in on July 21, 2017. According to this new temporary softfork rule, miners were then required to flag support for SegWit from July 26, 2017. As that date approached, some Bitcoiners were concerned about the elevated risk. If not all miners complied, there could be network issues. However, the day came and all the miners complied, flagging bit 1. There was no chain-split and SegWit was finally locked in, and it then activated on the Bitcoin network. With this insane mess and complexity of interlocking deadlines and activation mechanisms, it was almost impossible to keep track of what was going on. It seemed like every few days there was some sort of activation mechanism or deadline. If you have read this far into this book, it might be apparent to you that I was pretty obsessed with this conflict, to say the least; however, at this point, even I found it challenging to keep track of what was going on.
BIP 91 activated right before the BIP 148 deadline of August 1, 2017, leaving just a tiny, five-day gap between when BIP 91 made flagging for SegWit mandatory and when the UASF would have done the same. This close proximity appeared to indicate that the threat of a UASF had worked. The fact that the UASF succeeded was a miraculous achievement. A modern-day David vs Goliath victory. The UASF was never implemented in a release of Bitcoin Core, and some of the most influential Bitcoin Core developers had openly opposed it.
In the true spirit of Bitcoin and its pseudonymous creator, Satoshi, the UASF didn’t emerge from an official, closed-door, roundtable meeting among the big players. Instead, it was released in the open and promoted by a pseudonymous developer, Shaolinfry. Somehow, one pseudonymous individual, a few grassroots hardcore followers and a few hats commissioned by Samson Mow, had gone head-to-head with a multi-billion dollar enterprise (Bitmain) supported by many other well capitalised businesses, and won. I remember thinking that something like that could only happen in Bitcoin. It was the battlefield in Bitcoin that was unique and therefore one can get some quite peculiar outcomes.
The BIP 91 upgrade was far from perfect from the smaller blocker point of view; it was too rushed, the short, 336-block voting windows were too small (just 2.33 days long), mandatory miner signalling was risky and, crucially, only the miners were running BIP 91, which made the upgrade incredibly precarious. It could have resulted in miners and the users being left on different chains. Regardless, however messy and dangerous the process was, the small blockers had got their way: SegWit had activated, and the UASF appeared to force Jihan’s hands. They could now focus on stopping phase two of the NYA, the hardfork.
The large blockers were furious by these developments. One of the prominent large blockers mentioned to me in person that the “stupid UASF hats had worked” and forced miners to activate SegWit. He was particularly frustrated by the fact that this had all happened before August 1; had it occurred after that date, the large blocker narrative that the NYA had caused SegWit to activate, not the UASF, would have been more compelling. The large blockers felt the most extreme parts of the small blocker movement had taken control of Bitcoin and activated something they were extremely unhappy about. They appeared disillusioned with Bitcoin at this point, and were not even particularly bothered by phase two of the NYA.
It seemed to me that perhaps Jihan had potentially missed a trick here. He could have waited for the August 1 deadline to pass, then activated SegWit on Bitcoin. This would have made it even more difficult for the small blockers to support the BIP 148 chain, since SegWit would have activated on the other chain anyway. Jihan could then argue he had defeated the UASF. However, Jihan chose not to do this and the reason appeared to be that the UASF worked. Jihan was scared of a chain-split, possibly overestimated the power of his opponents, and gave in to the pressure.
With SegWit activated, the small blockers were very much free to move forwards on their chain, Bitcoin. However, there was still the issue that many significant companies in the space, in particular US-based and VC-backed ones, had committed to a hardfork blocksize increase. They did not want this to be a new, alternative coin; they wanted this to be Bitcoin. Getting these companies to back down looked likely to be extremely difficult. The blocksize war therefore continued. This hardfork was due to happen in just three months, and the battlefield was heating up.