QA » openQA Project » openQA Tests » qe-yam

This is still in progress.
So far, I have researched and study around gems and OBS and what can be done there. I did some experiments to build an rpm but i figured that we need to have the project as a gem anyway, even if we package it as rpm later. So i have created an account in rubygems.org(details in the maillist). I did some small changes in the repo, build and publish. The process is quite easy and it can be easily integrated in the CI IMO. The downside, as i see it, with gem is that we will have to handle the version of the dependencies manually or use one of the rbenv || rvm.
Having the gem, there are a few ways to create builds in OBS. Next step is to try one or two of them as POC. One thing for sure with the OBS is that is more complicated and it will require more work for the libyui-client builds. In additional with the gem file built as described before falls in this list of disadvantages but also advantages.

Agreed with @ybonatakis to try to have the skeleton for the ci with the most simple path working, so at least we have something and for the other findings or potential paths, document it here.

The area of exploration was widely and the results of this research is incomplete. I will create a new ticket to address some part of the work that I think it needs to be done in order to accomplish release flow in the same way as other yast modules. Nevertheless the findings there are still unknowns (see below).

Builds:

There were two core approaches. gem and rpm.

Gem

• easy to do packaging for gem.
  bash gem build libyui_client.gemspec
  
• publish in rubygems.org
  bash gem push libyui_client-<v>.gem
  rpm
  there are many different routes.
  i tried:
  
• (After having the gem published)setup obs, built package local and publish
  
• for setup see docs
  
• steps i did to build (approximately)
  bash # from obs project repo # this get the gem and creates the basic spec for the rpm gem2rpm -o libyui_client.spec --fetch libyui_client # builds require changelogs osc vc # final build osc build --local # and if everything is fine osc add libyui_client-0.4.0.gem libyui_client.changes libyui_client.spec osc commit
  All the above works but it does not make easy the CI and release process. And in additional it is not how yast guys are doing it. However those are some of the findings:
  
• They have ready scripts that they run inside a container and does all the job. This includes rake test, coverage, static analysis etc. and at the end it builds a rpm package. The image contains everything for this job. An image that we can use is yast-ruby.
  
• The important part of that image is that contains all it takes to run the script yast-travis-ruby.
  
• To make it work we need a Dockerfile in the libyui_client repo and a folder package. The folder must hold the rpm spec and the changelog file.
  
• We need a spec file under package directory. I provide a minimal one which builds but with errors. I generate this with gem2rpm and editted to include some things. One thing that it is required is rubygem(%rb_default_ruby_abi:yast-rake). But in general we have to learn a few things regarding the packaging as it seems that we will have to maintain this manual. But i would\ not expect it to being changed very often. in the same directory there is also the changelogs which it is required by the build.
  
• In theory having the above files in place, we could just copy the .travis configuration and have our CI. In practice we have to do modifications because the script and the test which it runs using specific gems. In particular they use yast/rake and yast/rspec instead of the normal one respectively. I tried to adjust the project to install yast/rake but there was an err\ or Could not find a valid gem 'yast/rake' (>= 0) in any repository
  
• We need to update the tests (dependencies, path imports) and move them under test folder. Run the test with yast/rake. They need also to have suffix _spec.rb or _test.rb. in the Ruby community, they tend to use spec and spec_helper.rb when using rspec. But it is a matter of taste and test and test_helper.rb should work just fine. Botton line, it does not matter what co\ nvension we will use. Again the docker will execute the script and the later will run those test. Nothing more to do here.
  

Instead of use rake and define our job the convenience with yast/rake is that provides plenty of jobs for more of the ci things. to try local we need to install yast2-devtools and ruby2.7-rubygem-yast-rake (TW)

test@localhost:~/libyui_client> rake --tasks                                                                                                                                                                                                                                                                                                                                                
rake build_dependencies:install           # Install the packages required for building                                                                                                                                                                                                                                                                                                      
rake build_dependencies:list              # Print the packages required for building                                                                                                                                                                                                                                                                                                        
rake check:changelog                      # Checking for new IDs (bugzilla,fate,...) in *.changes file                                                                                                                                                                                                                                                                                      
rake check:committed                      # check if everything is commited to git repository                                                                                                                                                                                                                                                                                               
rake check:doc                            # Check for errors in documentation and minimal coverage (supported: rubydoc)                                                                                                                                                                                                                                                                     
rake check:license                        # Check the copyright+license headers in files                                                                                                                                                                                                                                                                                                    
rake check:osc                            # Check for installed osc client and its configuration                                                                                                                                                                                                                                                                                            
rake check:pot                            # Check translatable strings for common mistakes                                                                                                                                                                                                                                                                                                  
rake check:rubocop[options]               # Run Rubocop in parallel                                                                                                                                                                                                                                                                                                                         
rake check:rubocop:auto_correct[options]  # Run Rubocop in parallel in the auto correct mode                                                                                                                                                                                                                                                                                                
rake check:spelling                       # Run spell checker (by default for *.md and *.html files)                                                                                                                                                                                                                                                                                        
rake check:syntax                         # Check syntax of all Ruby (*.rb) files                                                                                                                                                                                                                                                                                                           
rake console                              # Runs console with preloaded module directories                                                                                                                                                                                                                                                                                                  
rake install                              # Install to system                                                                                                                                                                                                                                                                                                                               
rake osc:build[osc_options]               # Build package locally                                                                                                                                                                                                                                                                                                                           
rake osc:commit                           # Commit package to devel project in build service if sources are correct and b...                                                                                                                                                                                                                                                                
rake osc:sr                               # Create submit request from updated devel project to target project if version...                                                                                                                                                                                                                                                                
rake osc:sr:force                         # Create submit request from devel project to target project without any other ...                                                                                                                                                                                                                                                                
rake package                              # Prepare sources for rpm build                                                                                                                                                                                                                                                                                                                   
rake pot                                  # Collect translatable strings and create POT files                                                                                                                                                                                                                                                                                               
rake run[client]                          # Run given client                                                                                                                                                                                                                                                                                                                                
rake server[port]                         # Start an HTTP server providing dynamically generated source code tarball                                                                                                                                                                                                                                                                        
rake tarball                              # Build tarball of git repository                                                                                                                                                                                                                                                                                                                 
rake test:unit                            # Runs unit tests                                                                                                                                                                                                                                                                                                                                 
rake version:bump                         # Increase the last part of version in package/*.spec files                                                                                                                                                                                                                                                                                       

https://github.com/yast/yast-rake

For the CI I tried a travis and buddy.works. I ignore the jenkins intentionally.

*** Jenkins

• the configuration can be expensive, especially for a small project like libyui_client.
  *** Travis
  
• in the other hand fits perfect in an open source project and it is what yast uses too. easy and flexible
  *** Buddy.works
  
• easy and fast. Many plugins without conflicts(see jenkins). Amazing debugging feature of the pipeline connecting to the console of the container.
  
• I have leave this attached for now to review. Let me know when you done to disable it. it is configured to run the tests and build the gem for now. the job which pushes to rubygem is disabled.
  

Some results from the execution of the CI:

https://travis-ci.org/github/qa-sle-yast/libyui_client/builds/725902206 fails with robocop ruby 2.3

repo with chages and experiments i used:

https://github.com/qa-sle-yast/libyui_client/pull/25

riafarov will organize meeting to settle down the name and continue from there.