openQA Project (public) » openQA Tests (public) » Containers and images

We are still waiting for an account. As soon as we get some more info, we'll be able to create the sub-tasks for this.

Reopening the ticket. The task has been asked by PM (François X.), cause there is customer interest about this.
The test is described in this confluence page: https://confluence.suse.com/display/qasle/BCI+on+IBM+Cloud+validation+proposal

We don't need to run this as part of ALL BCI Containers validation, we can limit it to the NodeJS container.

Kickoff meeting 03.07.2024:

• The IBM cloud has high environment complexity, because it does not allow you to directly interact with the container engine
• Test complexity requires us to define only a limited test scope - To be clarified: Which containers to test? (Suggestion: BCI-Base only)
• The environment has been sporadic, the existing test environment stopped working after some weeks of testing
• Action item: Determine, if we can obtain the container logs (this was not possible with the free account)
• To clarify: Only test released images. We cannot block our release pipeline for IBM images, due to the test environment (too flanky due to high complexity, not easy to interact)

ph03nix wrote in #note-8:

Kickoff meeting 03.07.2024:

• The IBM cloud has high environment complexity, because it does not allow you to directly interact with the container engine
• Test complexity requires us to define only a limited test scope - To be clarified: Which containers to test? (Suggestion: BCI-Base only)
• The environment has been sporadic, the existing test environment stopped working after some weeks of testing
• Action item: Determine, if we can obtain the container logs (this was not possible with the free account)
• To clarify: Only test released images. We cannot block our release pipeline for IBM images, due to the test environment (too flanky due to high complexity, not easy to interact)

@jlausuch question to clarify:

1. We can only test a minimal test of BCI containers due to the high text complexity. We cannot test all containers. Which ones do we need to test? Our suggestion are the following containers:

• bci-base
• bci-busybox

We think because all other containers rely on them, this is a good start. Plus, we already test all containers internally on s390x, so that a lifeness-test of the BCI containers provides us with sufficient test coverage to be confident, that BCI would run in the IBM cloud.

1. Is testing BCI images after the release sufficient? We would highly recommend to not include IBM cloud into our main BCI test pipeline, because this will mean a considerable increase in the testing latency, possible up to 1-2 weeks because of the high complexity of the test environment.

ph03nix wrote in #note-8:

Kickoff meeting 03.07.2024:

• The IBM cloud has high environment complexity, because it does not allow you to directly interact with the container engine

The high complexity of the environment and the inability to interact with the container engine is because this environment is intended to be ultra-secure as a confidential computing platform for container based applications.

• Test complexity requires us to define only a limited test scope - To be clarified: Which containers to test? (Suggestion: BCI-Base only)

I agree BCI base is good to start with.

• The environment has been sporadic, the existing test environment stopped working after some weeks of testing

There was an issue with the IBM provided account for SLE BCI testing that was identified by IBM Cloud. This was resolved in 2023.

• Action item: Determine, if we can obtain the container logs (this was not possible with the free account)

Starting in 2023, SUSE is paying a nominal yearly fee for the logging service which stores logs for 30 days.

• To clarify: Only test released images. We cannot block our release pipeline for IBM images, due to the test environment (too flanky due to high complexity, not easy to interact)

This environment is not flaky but rather an ultra-secure confidential computing environment which makes interacting in normal ways difficult.

ph03nix wrote in #note-9:

ph03nix wrote in #note-8:

Kickoff meeting 03.07.2024:

• The IBM cloud has high environment complexity, because it does not allow you to directly interact with the container engine
• Test complexity requires us to define only a limited test scope - To be clarified: Which containers to test? (Suggestion: BCI-Base only)
• The environment has been sporadic, the existing test environment stopped working after some weeks of testing
• Action item: Determine, if we can obtain the container logs (this was not possible with the free account)
• To clarify: Only test released images. We cannot block our release pipeline for IBM images, due to the test environment (too flanky due to high complexity, not easy to interact)

@jlausuch question to clarify:

1. We can only test a minimal test of BCI containers due to the high text complexity. We cannot test all containers. Which ones do we need to test? Our suggestion are the following containers:

• bci-base
• bci-busybox

We think because all other containers rely on them, this is a good start. Plus, we already test all containers internally on s390x, so that a lifeness-test of the BCI containers provides us with sufficient test coverage to be confident, that BCI would run in the IBM cloud.

I agree bci-base is a good container for testing.

1. Is testing BCI images after the release sufficient? We would highly recommend to not include IBM cloud into our main BCI test pipeline, because this will mean a considerable increase in the testing latency, possible up to 1-2 weeks because of the high complexity of the test environment.

Yes. The testing of bci-base in IBM Cloud is to verify that the container properly starts on a non-SLE container host. Successful test(s) will enable the documentation team to update the support status for IBM Hyper Protect Platform listed in https://documentation.suse.com/container/all/single-html/Container-guide/#sec-sle-container-sle-host-support

mfriesenegger wrote in #note-11:

ph03nix wrote in #note-8:

Kickoff meeting 03.07.2024:

• The IBM cloud has high environment complexity, because it does not allow you to directly interact with the container engine

The high complexity of the environment and the inability to interact with the container engine is because this environment is intended to be ultra-secure as a confidential computing platform for container based applications.

• Test complexity requires us to define only a limited test scope - To be clarified: Which containers to test? (Suggestion: BCI-Base only)

I agree BCI base is good to start with.

• The environment has been sporadic, the existing test environment stopped working after some weeks of testing

There was an issue with the IBM provided account for SLE BCI testing that was identified by IBM Cloud. This was resolved in 2023.

• Action item: Determine, if we can obtain the container logs (this was not possible with the free account)

Starting in 2023, SUSE is paying a nominal yearly fee for the logging service which stores logs for 30 days.

• To clarify: Only test released images. We cannot block our release pipeline for IBM images, due to the test environment (too flanky due to high complexity, not easy to interact)

This environment is not flaky but rather an ultra-secure confidential computing environment which makes interacting in normal ways difficult.

Thank you for the clarification. Given this input we can start working and will re-evaluate the stability statement. I certainly hope the past issues have been resolved, thank you!