BitMEX Downtime Postmortem

Traders,

Today, from 8:08:01 UTC to 8:20:38 UTC, our trading engine was suspended due to an inconsistency on a user’s deposits.

BitMEX is serious about the security of user’s funds and of the exchange as a whole. Each one of our systems runs extensive, continuous internal audits. At 8:08:01 UTC, blockchain data received by the trading engine indicated that a user’s deposit had a problem. A transaction that has previously had over 20 confirmations had suddenly gone back into “Unconfirmed” status. This caused a mismatch between the expected deposited funds in the system and the “actual” funds.

We source blockchain data not only from a locally-run full node, but also from several full nodes across the internet. The inconsistency on a remote copy of the blockchain disappeared within seconds, but an internal audit failure causes a complete shutdown on BitMEX until we are sure that it is safe to proceed.

At 8:20:38 UTC, our team determined that the transaction was in fact confirmed, and that the failed audit was merely a temporary data issue. At that point, we decided that it was okay to proceed and unsuspended the trading engine.

We are sorry for any inconvenience this caused. We are working to ensure that this type of hiccup cannot happen again. As always, our policy is to err on the side of user safety.

If you have any questions, please talk to us in chat or email us at support@bitmex.com.