QA » openQA Project » openQA Tests

In some cases the interactive "# $command" fails but function "retrieve_mail_pop3()" returns "success".
This may induce test case fake "PASS".

Two negative testing examples for your reference:

1. revise code ".../lib/utils.pm" as following on purpose

sub retrieve_mail_pop3 {
    ...
    script_run_interactive(
        "telnet localhost pop3",
        ...
            {
276                # prompt => qr/\+OK/m,                -> comment out this line
277                prompt => qr/\+error message/m,       -> using unmatching string on purpose           
                      string => "pass $pw\n",
            },

Run test case "usr_lib_dovecot_pop3" and check the log,
the command "# telnet" reports "-ERR Disconnected for inactivity. Connection closed by foreign host."
but function "retrieve_mail_pop3" returns "success".

FYI:
http://10.67.19.89/tests/629#step/usr_lib_dovecot_pop3/36

2 . revise code ".../lib/utils.pm" as following on purpose

...
256     script_run_interactive(
257         #"telnet localhost pop3",             -> comment out this line
258         "telnet localhost err-protocol",           -> using a error protocol on purpose   

Run test case "usr_lib_dovecot_pop3" and check the log,
the command "# telnet" reports "telnet: err-proptocol: bad port"
but function "retrieve_mail_pop3" returns "success" and this makes test case "PASS", it is "FAIL" indeed.

FYI:
http://10.67.19.89/tests/630#step/usr_lib_dovecot_pop3/36

Merged PR#6978

@vsvecova: Telnet return code 1, which should be 0 because all operations succeeded. Need further investigation, or we could create a new poo.