Repo was moved to our organization, so we can establish CI for it: https://github.com/qa-sle-yast/libyui_client
As a test developer I want to install and update libyui client gem same way as other dependencies.
Part of the ticket to confirm the name of the gem.
We have option to build it using OBS/IBS or public pool.
#1 Updated by riafarov about 1 year ago
- Subject changed from [y] Establish build and releases of ruby libyui client to [y][timeboxed:20h] Establish build and releases of ruby libyui client
- Description updated (diff)
- Category changed from Infrastructure to Spike/Research
- Status changed from New to Workable
#3 Updated by ybonatakis about 1 year ago
- Status changed from Workable to In Progress
This is still in progress.
So far, I have researched and study around gems and OBS and what can be done there. I did some experiments to build an rpm but i figured that we need to have the project as a gem anyway, even if we package it as rpm later. So i have created an account in rubygems.org(details in the maillist). I did some small changes in the repo, build and publish. The process is quite easy and it can be easily integrated in the CI IMO. The downside, as i see it, with gem is that we will have to handle the version of the dependencies manually or use one of the rbenv || rvm.
Having the gem, there are a few ways to create builds in OBS. Next step is to try one or two of them as POC. One thing for sure with the OBS is that is more complicated and it will require more work for the libyui-client builds. In additional with the gem file built as described before falls in this list of disadvantages but also advantages.
#5 Updated by ybonatakis about 1 year ago
- Due date changed from 2020-09-22 to 2020-09-08
- Status changed from In Progress to Feedback
The area of exploration was widely and the results of this research is incomplete. I will create a new ticket to address some part of the work that I think it needs to be done in order to accomplish release flow in the same way as other yast modules. Nevertheless the findings there are still unknowns (see below).
There were two core approaches. gem and rpm.
- easy to do packaging for gem.
bash gem build libyui_client.gemspec
- publish in rubygems.org
bash gem push libyui_client-<v>.gem
there are many different routes.
- (After having the gem published)setup obs, built package local and publish
- for setup see docs
- steps i did to build (approximately)
bash # from obs project repo # this get the gem and creates the basic spec for the rpm gem2rpm -o libyui_client.spec --fetch libyui_client # builds require changelogs osc vc # final build osc build --local # and if everything is fine osc add libyui_client-0.4.0.gem libyui_client.changes libyui_client.spec osc commit
All the above works but it does not make easy the CI and release process. And in additional it is not how yast guys are doing it. However those are some of the findings:
- They have ready scripts that they run inside a container and does all the job. This includes rake test, coverage, static analysis etc. and at the end it builds a rpm package. The image contains everything for this job. An image that we can use is
- The important part of that image is that contains all it takes to run the script
- To make it work we need a Dockerfile in the libyui_client repo and a folder
package. The folder must hold the rpm spec and the changelog file.
- We need a spec file under package directory. I provide a minimal one which builds but with errors. I generate this with gem2rpm and editted to include some things. One thing that it is required is rubygem(%rb_default_ruby_abi:yast-rake). But in general we have to learn a few things regarding the packaging as it seems that we will have to maintain this manual. But i would\
not expect it to being changed very often. in the same directory there is also the changelogs which it is required by the build.
- In theory having the above files in place, we could just copy the .travis configuration and have our CI. In practice we have to do modifications because the script and the test which it runs using specific gems. In particular they use
yast/rspecinstead of the normal one respectively. I tried to adjust the project to install
yast/rakebut there was an err\ or Could not find a valid gem 'yast/rake' (>= 0) in any repository
- We need to update the tests (dependencies, path imports) and move them under test folder. Run the test with
yast/rake. They need also to have suffix
_test.rb. in the Ruby community, they tend to use spec and spec_helper.rb when using rspec. But it is a matter of taste and test and test_helper.rb should work just fine. Botton line, it does not matter what co\ nvension we will use. Again the docker will execute the script and the later will run those test. Nothing more to do here.
Instead of use rake and define our job the convenience with
yast/rake is that provides plenty of jobs for more of the ci things. to try local we need to install yast2-devtools and ruby2.7-rubygem-yast-rake (TW)
test@localhost:~/libyui_client> rake --tasks rake build_dependencies:install # Install the packages required for building rake build_dependencies:list # Print the packages required for building rake check:changelog # Checking for new IDs (bugzilla,fate,...) in *.changes file rake check:committed # check if everything is commited to git repository rake check:doc # Check for errors in documentation and minimal coverage (supported: rubydoc) rake check:license # Check the copyright+license headers in files rake check:osc # Check for installed osc client and its configuration rake check:pot # Check translatable strings for common mistakes rake check:rubocop[options] # Run Rubocop in parallel rake check:rubocop:auto_correct[options] # Run Rubocop in parallel in the auto correct mode rake check:spelling # Run spell checker (by default for *.md and *.html files) rake check:syntax # Check syntax of all Ruby (*.rb) files rake console # Runs console with preloaded module directories rake install # Install to system rake osc:build[osc_options] # Build package locally rake osc:commit # Commit package to devel project in build service if sources are correct and b... rake osc:sr # Create submit request from updated devel project to target project if version... rake osc:sr:force # Create submit request from devel project to target project without any other ... rake package # Prepare sources for rpm build rake pot # Collect translatable strings and create POT files rake run[client] # Run given client rake server[port] # Start an HTTP server providing dynamically generated source code tarball rake tarball # Build tarball of git repository rake test:unit # Runs unit tests rake version:bump # Increase the last part of version in package/*.spec files
For the CI I tried a travis and buddy.works. I ignore the jenkins intentionally.
- the configuration can be expensive, especially for a small project like libyui_client.
- in the other hand fits perfect in an open source project and it is what yast uses too. easy and flexible
- easy and fast. Many plugins without conflicts(see jenkins). Amazing debugging feature of the pipeline connecting to the console of the container.
- I have leave this attached for now to review. Let me know when you done to disable it. it is configured to run the tests and build the gem for now. the job which pushes to rubygem is disabled.
Some results from the execution of the CI:
https://travis-ci.org/github/qa-sle-yast/libyui_client/builds/725902206 fails with robocop ruby 2.3
repo with chages and experiments i used:
#6 Updated by ybonatakis about 1 year ago
Project name voting https://github.com/qa-sle-yast/libyui_client/issues/22