action #135923
closed[qe-sap][tools]test fails in hana_install because NFS is too slow - Move NFS to OSD size:M
0%
Description
Observation¶
openQA test in scenario sle-15-SP4-SAP-DVD-Updates-x86_64-qam-sles4sap_scc_gnome_hana_cli@64bit-sap-qam fails in
hana_install
Test suite description¶
Testsuite maintained at https://gitlab.suse.de/qa-maintenance/qam-openqa-yml. Maintainer: qa-css team, QA and QAM for HA/SAP.
Test SAP HANA installation on SLES4SAP.
Reproducible¶
Fails since (at least) Build 20230915-1
Expected result¶
Last good: 20230914-1 (or more recent)
Acceptance Criteria:¶
- AC1: The commands accessing data on the NFS server do not time out.
Suggestions¶
- We can't rely on the NFS share from OSD to be mounted on workers so consider to use the NFS server provided by OSD within tests
- Find out what this "HANA2 directory" contains. Does it's content change for each SLE build?
- Suggest to ticket reporter to ask Eng-Infra if another VM next to OSD can provide better performance or if it shares all resources with osd hence would bring no benefit
Further details¶
Always latest result in this scenario: latest
HANA and NetWeaver tests need their install media either mounted or mounted and copied over for the test run to proceed. Until the migration to the PRG2, the jobs that were assigned to prg_office worker class were connected to qesap-nfs.qa.suse.cz and the jobs in NUE1 where assigned to a now retired NFS Server in NUE1-SRV2.
Sadly, the connection between PRG_office and PRG2 is not adequate enough and all tests that require an NFS server fail consistently due to timeouts
To fix this, I propose we use the NFS Server on OSD so that the data doesn't have to be bottlenecked by the external connecton.
The current size of the data is at a bit less than 500GB.
Updated by apappas 7 months ago
I am trying with a connection to nue2 https://openqa.suse.de/tests/12168011#
Updated by okurz 7 months ago
- Related to action #135938: [qe-sap] test fails in hana_install copying from NFS server qesap-nfs.qa.suse.cz with timeout added
Updated by apappas 7 months ago
- Subject changed from [sap]test fails in hana_install because NFS is too slow - Move NFS to OSD to [qe-sap][tools]test fails in hana_install because NFS is too slow - Move NFS to OSD
After a 2 week investigation, we found that our NFS servers do not have the bandwidth to respond to the increased load that the new increased concurrent jobs impose. To combat this I am requesting that at least the HANA2 directory be moved to the OSD NFS server.
The requirements are 351G of space and the ability to host 4 connections per SAP incident and SAP aggregate build.
Updated by okurz 7 months ago
- Assignee set to okurz
- Target version set to Ready
So a you asked in Slack as well I stated there that we can't rely on the NFS share from OSD to be mounted on workers however you can use the NFS server provided by OSD within tests. So this HANA2 directory, what does it contain? Does it's content change for each SLE build? I suggest you also ask Eng-Infra if another VM next to OSD can provide better performance or if it shares all resources with osd hence would bring no benefit