关于 2019 年 2 月 19 日系统服务轻微受阻的公告

在北京时间 2 月 19 日下午 13:31 ,BitMEX 曾经历了大约 1 分钟的服务轻微受阻,导致所有交易引擎运作一度暂停。

问题是出于内部市场数据分发组件之间数据传输的持续时间。这曾经是定期更新的一部分,旨在提高平台的整体韧性。 我们已确定根本原因并通过内部流程进行修复以防止事件再次发生。 此外,我们正在继续重新设计我们的市场数据分布架构,以消除在这种情况下对交易引擎的任何潜在影响。

对于给您带来的不便,我们深表歉意。 如果您有任何疑问,请联系客服。

欢迎转载,请注明文章来自

BitMEX (www.bitmex.com)

Notice of Minor System Outage 19 February 2019

On 19 February at 05:31 UTC, BitMEX experienced a minor outage for approximately 1 minute whereby all trading engine operations were suspended.

This issue occurred due to a sustained period of data transfers between the internal market data distribution components. This was part of a regularly scheduled update to improve the overall resiliency of the platform. The root cause has been identified and a fix via internal processes has been put in place to prevent a recurrence. Additionally, we are continuing to re-work our market data distribution architecture to eliminate any potential impact to the trading engine in such a scenario.

We apologise for the inconvenience. Should you have any questions, please contact customer support.

有关 2019 年 2 月 8 日 API 超时的公告

今天在北京时间下午 13:40 和15:11 之间,由于 API 层的资源争用,有部分对 BitMEX REST API 的请求经历了缓慢的 API 响应, 最终导致 API 超时。通过我们的内部警报机制检测后,我们确定了原因,并在几分钟内减轻了直接影响。 目前没有持续的问题,在此期间对交易引擎或用户数据没有影响。
我们已经确定了针对该问题的根本原因的修复,并且正在将其作为优先事项进行处理。 一旦修复生效,我们将发布另一个公告跟进。 我们还提高了系统监控的灵敏度,以便更快地检测和解决潜在的类似问题。 对由此造成的任何不便,我们深表歉意。

欢迎转载,请注明文章来自

BitMEX (www.bitmex.com)

Notice of API Timeouts 8 February 2019

Between 05:40 and 07:11 UTC today, a subset of the requests to the BitMEX REST API experienced slow API responses and eventual API timeouts due to resource contention at the API layer. Upon detection via our internal alerting mechanisms we identified the cause and mitigated the immediate impact within a few minutes. There is currently no ongoing issue and there was no impact to the trading engine or user data during this time.

Fixes for the underlying root cause of the issue have been identified and are being worked on as a priority. We will follow up with another announcement once these are live. We have also increased the sensitivity of our system monitoring to detect and resolve potential similar issues much sooner. We apologise for any inconvenience this may have caused.

更新:解决上周问题的根由

针对上周的问题发布,昨天我们成功发布了内部市场数据分布组件的重新订阅逻辑的改进版。这解决了上周问题的根由和设立了额外的安全机制,以防止对当时所部署的交易引擎造成影响,我们预计上周的问题不会再次发生。

 

 

欢迎转载,请注明文章来自

BitMEX (www.bitmex.com)

Update: fix for root cause of last week’s issue

In response to last week’s post, yesterday we successfully released an enhancement to our internal market data distribution component’s re-subscription logic. This addresses the root cause of the previous week’s issue and along with the additional safety mechanism to prevent impact to the trading engine deployed at the time, we don’t anticipate a reoccurrence of last week’s issue.

1 月 9 日两次个别的服务受阻

BitMEX 在 1 月 9 日曾出现了两次个别的服务受阻。

北京时间时间 10:44:10,WebSocket API 性能曾下降了一分钟,有 7% 的客户命令发送失败。其后的连接在 1% 的命令失败率下运行,直到服务器在北京时间 10:47:00 回复正常

在此期间,客户可能也发现某些市场数据 REST 端点的响应时间有所增加。这是由于在短时间
内 API 服务器陆续重启所致。

此外,在北京时间 13:48:10 和北京时间 14:10:10 ,由于引擎繁忙,BitMEX 曾出现了约 30 秒的服务受阻,当时所提交至交易引擎的委托均被卸载了。在此期间,客户也会发现 WebSocket API 没有更新。服务受阻的原因是定期进行的市场数据分布组件引发了资料重放问题。

在这些服务受阻期间的数据均未丢失,并且已经部署了用于防止类似情况影响交易引擎的附加安
全机制。我们已确定了根本原因所在,目前正在进行永久性修复,以防止问题再次发生。随后将会有相关更新。

对您造成的不便,我们深表歉意。如果您有任何问题,请联系客服。

 

 

欢迎转载,请注明文章来自

BitMEX (www.bitmex.com)

Two unrelated minor outages on 9 January

On 9 January BitMEX experienced two unrelated minor outages.

At 02:44:10 UTC the WebSocket API saw a degradation in performance for a minute where 7% of commands sent by clients failed. Connections continued undergoing a 1% failure rate of commands until servers recovered at 02:47:00 UTC.

Clients may have also seen an increase in response times for some market data REST endpoints up during this period. This was due to a rolling restart of the API servers that occurred in too tight of a timeframe.  

In addition, at 05:48:10 UTC and 06:10:10 UTC, BitMEX experienced minor outages for approximately 30 seconds whereby requests to the trading engine were load-shed as the engine was busy. During these times, clients would have observed a lack of updates over the WebSocket API for the same reason. The outages were due to data replay complications during a regularly scheduled market data distribution component restart.  

There was no data loss during these events and an additional safety mechanism to prevent a similar situation from impacting the trading engine has already been deployed. The root causes have also been identified and we are currently working on permanent fixes to prevent a recurrence. Updates will follow in the future.

We apologise for the inconvenience. If you have any questions, please contact customer support.

报告:服务受阻, 2018 年 12 月 28 日

12 月 28 日北京时间 02:38 , BitMEX 经历了大约 1 分钟的轻微停机,导致交易引擎无法使用。在此期间,市场数据更新无法通过 Websocket API 来发布。

 

此外,在从北京时间 02:13 到 02:36 的 23 分钟内,交易相关数据的 REST API 查询( HTTP GET 的只读请求,而不是委托管理请求)耗时比平常时间长了许多。

 

我们已确定造成这两个问题的原因,并正在积极修复中以防止再次发生。我们将在未来就进一步相关情况进行更新。

 

对于给您带来的不便,我们深表歉意。如果您有任何其他疑问,可以联系客户支持。

 

 

 

欢迎转载,请注明文章来自

BitMEX (www.bitmex.com)

Postmortem: Downtime, 27 December 2018

On 27 December at 18:38 UTC, BitMEX experienced a minor outage for approximately 1 minute whereby the trading engine was unavailable. During this time, market data updates were not published over the Websocket API.

Additionally, REST API queries for trading related data (read-only HTTP GET requests, not order management requests) took longer to return than usual over a period of 23 minutes from 18:13 UTC to 18:36 UTC.

The root causes of both issues have been identified and we are working on permanent fixes to prevent a recurrence. Updates will follow in the future.

We apologise for the inconvenience. If you have any questions, please contact customer support.