Postmortem: Downtime, July 14, 2017

Traders,

On July 14, 2017, we suffered a minor downtime as a runaway ZFS snapshot process froze up disk I/O on the trading engine. No data was lost. While the outage was relatively minor and required only a host reboot, we took additional time to re-verify data, clean up ZFS snapshots, and fix the underlying issue.

We apologize for the disruption.

If you are interested in our recent migration to ZFS, please see this post.