Buffer-Controlled Writes to HDDs for SSD-HDD Hybrid Storage Server

BCW is based on the finding that HDDs are usually underutilized in an SSD-HDD hybrid storage server while SSDs are suffering from high write pressure.

The SSD delivers outstanding peak performance:

  • Peak write requests per second > 𝟏𝟎𝑲𝑹𝑷𝑺
  • Data written per day > 𝟑𝑻𝑩

However, the long tail latency is also unneglectable:

  • 99th-percentile write latency > 𝟏𝟎𝒎𝒔
  • 99.9th-percentile write latency > 𝟓𝟎𝒎𝒔

The queue length of an SSD can be as large as more than 100.

HDD can reach 𝝁𝒔-level write latency, especially for small size requests:

  • 35us for 4KB writes
  • 66us for 16KB writes
  • 180us for 64KB writes

Due to the HDD buffers, the HDDs repeatly performs fast/mid/slow writes:

  • 𝑭ast write (low-latency): 35𝝁𝒔
  • 𝑴id write (mid-latency): 55𝝁𝒔
  • 𝑺low write (high-latency spike): 12ms

The pattern looks like this: FFFF MMMS MMMS MMMS …

The latency of HDD buffered writes can be lower than SSD write latency and is predictable! Thus, the solution is to redirect writes to HDDs when profitable. This paper compared multiple redirect strategies.