action #66709


Storage server for OSD and monitoring

Added by nicksinger about 4 years ago. Updated almost 4 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


We have raising storage needs for OSD and it's infrastructure. For this we want to use a part of our budget for a storage machine. I'm aiming at roughly 10k€ for this machine.

Current proposal (based on ):

  • Chassis: SuperMicro 216BE1C-R920LPB
    • I don't think we need a redundant HBA as offered in SuperMicro 216BE2C-R920LPB.
  • CPU: AMD EPYC 7232P
  • CPU: AMD EPYC 7252
    • We plan to do storage without encryption. Therefore a low end CPU should be sufficient.
    • UPDATE 1: The 7252 has double the cache of the 7232P for just 30€ more
  • RAM: 4 x Micron MTA18ASF2G72PDZ-3G2 = 64 GB, 4 x 16 GB
    • Reads in linux are heavily cached by RAM. I don't think we can have enough RAM but it gets expensive quite fast.
  • Controller: Broadcom 9300-8i
  • Controller: Broadcom 9300-8e
    • Given that the chassis does not support NVMe's over the backplane a small HBA controller should be sufficient.
    • Broadcom 9500-8i could be a viable alternative (PCI-E 4.0 instead of 3.0, NVMe support - therefore most likely higher speeds in general)
    • RAID capabilities are not needed since we should do them in software
    • UPDATE 1: For expandability it might make sense to use an HBA with external interfaces so we could connect storage chassis in the future if we plan to extend.
  • Network: Intel X550-T2
    • To even remotely make use of the SSD performance, we need at least 10GBit/s to serve clients fast enough.
    • 2 Ports for redundancy or direct connection to OSD
  • M.2 NVMe: Samsung PM981 MZVLB256HAHQ
    • For the OS itself
    • Utilizing the M.2 slot allows us to fully use all 24 hotswapable ports in front of the machine for storage disks
  • SSDs: Micron 5300 PRO MTFDDAK960TDS (960GB per disk) - 181,56€/disk

This configuration ends up at 3.606,87€ 3.836,87€ (UPDATE 1) (excluding the mentioned SSD). Now for the disks I'm still unsure. Either we go full enterprise with rated 24/7 disks w/ 5y warranty:

  • HDDs: Seagate Enterprise ST2000NX0243 (2TB per disk) - 257,47€/disk

Alternative we could use consumer hardware with ~2y warranty but resulting in much lower prices:

  • Seagate BarraCuda ST4000LM024 (4TB per disk) - 150€/disk

As you can see it cuts quite some initial costs if we go with consumer HDDs (especially with 2.5"). However, it might increase our running costs if the disks fail more regularly (and we need to buy new ones instead of replacing them as part of the warranty)


Related issues 3 (0 open3 closed)

Related to openQA Project - coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old resultsResolvedokurz2020-03-18

Related to openQA Infrastructure - action #69577: Handle installation of the new "Storage Server"Resolvednicksinger2020-08-04

Related to openQA Infrastructure - action #93683: osd-deployment failed due to not reachable by saltResolvednicksinger2021-06-09


Also available in: Atom PDF