Actions
action #35709
closed[functional][y][hard] Improve auto-investigation in case the yast installer gets stuck
Status:
Resolved
Priority:
Normal
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA (private) - Milestone 17
Start date:
2018-04-30
Due date:
2018-07-03
% Done:
0%
Estimated time:
8.00 h
Difficulty:
hard
Description
Observation¶
openQA test in scenario sle-15-Installer-DVD-x86_64-proxyscc_upgrade_sles12sp3+ha+geo+sdk+we@64bit fails in
installation_overview getting stuck and we do not know what exactly is going on. As apparently this was not reproduced manually so far the according product bug bsc#1082148 was closed as RESOLVED WONTFIX.
Acceptance criteria¶
- AC1: We have a more generic post_fail_hook approach for the installer at least identifying where processes are stuck if any
Suggestion¶
locilka made nice suggestions in the aforementioned bug: "So I also propose to enhance that functionality to try to dig deeper, such as strace (you might need to detach it too) on the command that got stuck. Yes, getting a clue which one it is might be tough though. Maybe pstree helps. Maybe something more hipster-like, I don't know.". So I suggest the following:
- Read the full comment stream in bsc#1082148 to get the complete picture
- Take a look what we do with gdb in lib/y2snapper_common.pm when snapper gets stuck
- Research if the interweb has any better suggestions of what to do when "unknown process gets stuck in linux"
- Incorporate something into the yast installer post_fail_hook
- If possible extend the generic failure_investigation approach used in other test modules as well
Further details¶
Always latest result in this scenario: latest
Updated by JERiveraMoya over 6 years ago
- Subject changed from [functional][y] Improve auto-investigation in case the yast installer gets stuck to [functional][y][hard] Improve auto-investigation in case the yast installer gets stuck
- Difficulty set to hard
Updated by mloviska over 6 years ago
There are no symbols for gdb in installer.
GNU gdb (GDB; openSUSE Tumbleweed) 8.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
`/lib/modules/4.16.13-1-default/modules.symbols': can't read symbols: File format not recognized.
Attaching to process 3348
[New LWP 3350]
[New LWP 3351]
[New LWP 3352]
[New LWP 4092]
Missing separate debuginfo for /mounts/mp_0001/usr/bin/ruby.ruby2.5
Try: zypper install -C "debuginfo(build-id)=3df8ed7d3a686c9dae6bc0e931dd7f5948c82c43"
Missing separate debuginfo for /usr/lib64/libruby2.5.so.2.5
Try: zypper install -C "debuginfo(build-id)=c38e4b70f5d7bab2f77c2249712a6ac7be38ffca"
Missing separate debuginfo for /lib64/libc.so.6
Try: zypper install -C "debuginfo(build-id)=a91e2eb5538e0a2b00334adee0d86dceb4c9d115"
Missing separate debuginfo for /lib64/libpthread.so.0
Try: zypper install -C "debuginfo(build-id)=d49253aaac06cdc3c7f7e5292940246f755d795b"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Missing separate debuginfo for /lib64/libdl.so.2
Try: zypper install -C "debuginfo(build-id)=fc18fe0db7ca8a40641bd9636ab06e8e7de5e5c8"
Missing separate debuginfo for /lib64/libcrypt.so.1
Try: zypper install -C "debuginfo(build-id)=77c4837fdf83bb3ecdeffda673c0bd079537b0ca"
Missing separate debuginfo for /lib64/libm.so.6
Try: zypper install -C "debuginfo(build-id)=4be53bc951148cd205831c48f7f26a1510d610e1"
Missing separate debuginfo for /lib64/ld-linux-x86-64.so.2
Try: zypper install -C "debuginfo(build-id)=35e8536c5628e8e06b202db02a7ddb53f9973fb4"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/enc/encdb.so
Try: zypper install -C "debuginfo(build-id)=4926cd228e248d990327dd4dae54f5109073510c"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/enc/trans/transdb.so
Try: zypper install -C "debuginfo(build-id)=24d03d08ef036e9121052f173b944d4780fdb1ee"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/stringio.so
Try: zypper install -C "debuginfo(build-id)=3a0bcb44914c053939af487a56f67254f9dada7f"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/socket.so
Try: zypper install -C "debuginfo(build-id)=97a8e62058c1601361492fd8a557bcb20ab90b81"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/io/wait.so
Try: zypper install -C "debuginfo(build-id)=1ca5f2f5b7e4b1cdd809387e97bd5178a4f3e629"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/vendor_ruby/2.5.0/x86_64-linux-gnu/yastx.so
Try: zypper install -C "debuginfo(build-id)=d5ea08400c91cebd87a13d018bebaca821265b2c"
Missing separate debuginfo for /usr/lib64/liby2.so.4
Try: zypper install -C "debuginfo(build-id)=a57265d10df9af3966840f33748939dc35baadde"
Missing separate debuginfo for /usr/lib64/libycp.so.5
Try: zypper install -C "debuginfo(build-id)=f73c35cca4204af7820cd00aafbf6479a352e848"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2scr.so.2
Try: zypper install -C "debuginfo(build-id)=a1c7756a39efd2ccc27ad18ab306a10bf7f8eac9"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2wfm.so.2
Try: zypper install -C "debuginfo(build-id)=68f9b4427c140f324bcd667178ef617c10eaa4e7"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2UI.so.2
Try: zypper install -C "debuginfo(build-id)=e62636b74d290d7878daa2b13bfdaf39eb19abf4"
Missing separate debuginfo for /usr/lib64/libstdc++.so.6
Try: zypper install -C "debuginfo(build-id)=e749f79183b31dd9a719a40abb8476d68cc5c157"
Missing separate debuginfo for /lib64/libgcc_s.so.1
Try: zypper install -C "debuginfo(build-id)=7c81b6c9a2f488bad2ba0787b8dbe135377bda7f"
Missing separate debuginfo for /usr/lib64/liby2util.so.5
Try: zypper install -C "debuginfo(build-id)=93b8771d8627b37b82162b19c31accdeec58e331"
Missing separate debuginfo for /lib64/libowcrypt.so.1
Try: zypper install -C "debuginfo(build-id)=965567d950f8e3d986d13c1ce59ff332352878eb"
Missing separate debuginfo for /usr/lib64/libycpvalues.so.6
Try: zypper install -C "debuginfo(build-id)=638e80571c7e6a929deeb0caebf673af19f121d2"
Missing separate debuginfo for /usr/lib64/libscr.so.3
Try: zypper install -C "debuginfo(build-id)=b188b51c2d4cf0e528961f870e83638a40d767d5"
Missing separate debuginfo for /usr/lib64/libyui.so.8
Try: zypper install -C "debuginfo(build-id)=c32f24ac9dab3e5527fe73c9a2e9343e3859c066"
Missing separate debuginfo for /lib64/libutil.so.1
Try: zypper install -C "debuginfo(build-id)=96f8805c56f4789175720480851c4bd1058a427d"
Missing separate debuginfo for /lib64/libnss_compat.so.2
Try: zypper install -C "debuginfo(build-id)=13a05c53d9028e3837202a217075617db7f01743"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2lang_perl.so
Try: zypper install -C "debuginfo(build-id)=c0a8f5f0c859482c96e1242bec4be2027b80c95b"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/CORE/libperl.so
Try: zypper install -C "debuginfo(build-id)=b4423b0c0c9f08f12819a539146fed936a7766ad"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2lang_ruby.so
Try: zypper install -C "debuginfo(build-id)=1859ca42a4ad44a7fa597b821b1719c4bd226f82"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/vendor_ruby/2.5.0/x86_64-linux-gnu/yast/builtinx.so
Try: zypper install -C "debuginfo(build-id)=90eba7d3c4569c2bfe092d42b497da799644c0ac"
Missing separate debuginfo for /usr/lib64/yui/libyui-qt.so.8
Try: zypper install -C "debuginfo(build-id)=e1658e40fe8ba385441f7ab7d89fa5ebb09bd05f"
Missing separate debuginfo for /usr/lib64/libX11.so.6
Try: zypper install -C "debuginfo(build-id)=ca26e3699b99cd86b673f3ecd3208ee34fbeac39"
Missing separate debuginfo for /usr/lib64/libQt5X11Extras.so.5
Try: zypper install -C "debuginfo(build-id)=65b8883226274989327a262060b72479eb03b382"
Missing separate debuginfo for /usr/lib64/libQt5Widgets.so.5
Try: zypper install -C "debuginfo(build-id)=48fbe0c3e9fa2a6f1a3378a135d224fe3c83538e"
Missing separate debuginfo for /usr/lib64/libQt5Gui.so.5
Try: zypper install -C "debuginfo(build-id)=942ea56275207385d60c1bfc4ad91d7bd7e9d049"
Missing separate debuginfo for /usr/lib64/libQt5Core.so.5
Try: zypper install -C "debuginfo(build-id)=f654f18de4034398180fa5a74169c733141d844b"
Missing separate debuginfo for /usr/lib64/libxcb.so.1
Try: zypper install -C "debuginfo(build-id)=ffe46eccec8ecb876c330168b4f1518ef1cd2a8c"
Missing separate debuginfo for /usr/lib64/libGL.so.1
Try: zypper install -C "debuginfo(build-id)=2759f8fb75165f2e6d9f444bcb80795bb37a0626"
Missing separate debuginfo for /usr/lib64/libpng16.so.16
Try: zypper install -C "debuginfo(build-id)=7c45d1a665fd6afa3af0e9479223582ad0cf5055"
Missing separate debuginfo for /usr/lib64/libharfbuzz.so.0
Try: zypper install -C "debuginfo(build-id)=ea58dd6cdcc366e241d1233303db58d28b54b65e"
Missing separate debuginfo for /lib64/libz.so.1
Try: zypper install -C "debuginfo(build-id)=0b25ab8abacc59fa90a1e65c6ad8b99827bfb57b"
Missing separate debuginfo for /usr/lib64/libsystemd.so.0
Try: zypper install -C "debuginfo(build-id)=0dcbf447abf9b9d395fbdd87e1370fad531301a0"
Missing separate debuginfo for /usr/lib64/libicui18n.so.61.1
Try: zypper install -C "debuginfo(build-id)=051aaa5eb5bc1028233c115c9271f572e47589cd"
Missing separate debuginfo for /usr/lib64/libicuuc.so.61.1
Try: zypper install -C "debuginfo(build-id)=b6793076d9934661552e6461a2461454bdf319ca"
Missing separate debuginfo for /usr/lib64/libpcre2-16.so.0
Try: zypper install -C "debuginfo(build-id)=c46fada9a2de1a104ea0ef659685257ae6607ede"
Missing separate debuginfo for /usr/lib64/libdouble-conversion.so.1
Try: zypper install -C "debuginfo(build-id)=47419b6c54f6c019e1138413d38f0adf77a3d1a5"
Missing separate debuginfo for /usr/lib64/libglib-2.0.so.0
Try: zypper install -C "debuginfo(build-id)=4a760c4136bad2ae6cd768123fad52c3e270e3aa"
Missing separate debuginfo for /usr/lib64/libXau.so.6
Try: zypper install -C "debuginfo(build-id)=1d80d5179803eff82258a53098728b4f5867f971"
Missing separate debuginfo for /usr/lib64/libGLX.so.0
Try: zypper install -C "debuginfo(build-id)=b8900b72d300616625a2837284ce06dba0ac723a"
Missing separate debuginfo for /usr/lib64/libGLdispatch.so.0
Try: zypper install -C "debuginfo(build-id)=fc3b67636404dba754e770e55708cbeeed09751f"
Missing separate debuginfo for /usr/lib64/libfreetype.so.6
Try: zypper install -C "debuginfo(build-id)=79e4a707e95e0a3365ef037d327130b37426f418"
Missing separate debuginfo for /usr/lib64/libgraphite2.so.3
Try: zypper install -C "debuginfo(build-id)=8a202bbc4090555bb569a058574c1037a6c85568"
Missing separate debuginfo for /lib64/librt.so.1
Try: zypper install -C "debuginfo(build-id)=ff14c7229a755a70a2dc3d719be5932a649e1b7c"
Missing separate debuginfo for /usr/lib64/liblzma.so.5
Try: zypper install -C "debuginfo(build-id)=ac00c35ed10681155443a629ee8dc40799c06c33"
Missing separate debuginfo for /usr/lib64/liblz4.so.1
Try: zypper install -C "debuginfo(build-id)=dcfb9d0f85699c66a66617cb7617162e381c0a4d"
Missing separate debuginfo for /usr/lib64/libcap.so.2
Try: zypper install -C "debuginfo(build-id)=a301ba649aae6f92e4b6928b08b7a2e378071f5e"
Missing separate debuginfo for /usr/lib64/libgcrypt.so.20
Try: zypper install -C "debuginfo(build-id)=d042c00a2473ad5735efaa6b36e6c95ae5e739b5"
Missing separate debuginfo for /usr/lib64/libicudata.so.61.1
Try: zypper install -C "debuginfo(build-id)=fa37342db945c8851de7119c4df5d1077eaa77e6"
Missing separate debuginfo for /usr/lib64/libpcre.so.1
Try: zypper install -C "debuginfo(build-id)=dbdaff3d2b2d22214ecef5b34841fbed23f919ff"
Missing separate debuginfo for /usr/lib64/libbz2.so.1
Try: zypper install -C "debuginfo(build-id)=92ceeed2a3ac9794cfb52c57189294449427226b"
Missing separate debuginfo for /usr/lib64/libgpg-error.so.0
Try: zypper install -C "debuginfo(build-id)=7a36538ce6cb2906a849ef83b8e8cbdd24a72ff3"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/platforms/libqxcb.so
Try: zypper install -C "debuginfo(build-id)=d43ec70ee291bdf2b686995b93e0c0ffb306de95"
Missing separate debuginfo for /usr/lib64/libQt5XcbQpa.so.5
Try: zypper install -C "debuginfo(build-id)=fc8c15a891c03330e9e9f779e56ce4a19ae3faef"
Missing separate debuginfo for /usr/lib64/libfontconfig.so.1
Try: zypper install -C "debuginfo(build-id)=3e1629b9dbc4a75d70f633c40a5d0a101c65a71c"
Missing separate debuginfo for /usr/lib64/libQt5DBus.so.5
Try: zypper install -C "debuginfo(build-id)=ac3fb78be6ebb87312bc9c312ca29376134cfbd6"
Missing separate debuginfo for /usr/lib64/libX11-xcb.so.1
Try: zypper install -C "debuginfo(build-id)=6b712172ef35e0cee012f1552570fcdbd8c9feed"
Missing separate debuginfo for /usr/lib64/libXi.so.6
Try: zypper install -C "debuginfo(build-id)=fa4216e2faa0ce31444ac07829d46d14a8345b34"
Missing separate debuginfo for /usr/lib64/libSM.so.6
Try: zypper install -C "debuginfo(build-id)=991d12dc7d98a107087d9c5af53100cc1c526398"
Missing separate debuginfo for /usr/lib64/libICE.so.6
Try: zypper install -C "debuginfo(build-id)=25f005f0c0123cb6cca5fceff8db5eecbfc9e800"
Missing separate debuginfo for /usr/lib64/libXrender.so.1
Try: zypper install -C "debuginfo(build-id)=f1e82dced961b81ffdabc2fd063a737df3a71210"
Missing separate debuginfo for /usr/lib64/libxcb-xkb.so.1
Try: zypper install -C "debuginfo(build-id)=c08a0f41d628d25696865b8ea3a3d4c7a201fc13"
Missing separate debuginfo for /usr/lib64/libxcb-render-util.so.0
Try: zypper install -C "debuginfo(build-id)=d76deece1b062a2597eeeaf9dd263e492e3f6e67"
Missing separate debuginfo for /usr/lib64/libxcb-render.so.0
Try: zypper install -C "debuginfo(build-id)=0ce63b313e85e37f34eee992f8d7c4846e5525c9"
Missing separate debuginfo for /usr/lib64/libxcb-sync.so.1
Try: zypper install -C "debuginfo(build-id)=340dc7e162a844730dde7662223816e519d37ba4"
Missing separate debuginfo for /usr/lib64/libxcb-xfixes.so.0
Try: zypper install -C "debuginfo(build-id)=3af44aac63ac9f2d7193073798c59062f754d7a8"
Missing separate debuginfo for /usr/lib64/libxcb-xinerama.so.0
Try: zypper install -C "debuginfo(build-id)=95a2fef5fa243e7dc3bc66565fcdf0e82f5374a2"
Missing separate debuginfo for /usr/lib64/libxcb-randr.so.0
Try: zypper install -C "debuginfo(build-id)=f16c8349aa6517a1b113bfc6a2adb1bd44c88e0b"
Missing separate debuginfo for /usr/lib64/libxcb-image.so.0
Try: zypper install -C "debuginfo(build-id)=ee3acff966e47a5a6b4994b74a596274521a0bc5"
Missing separate debuginfo for /usr/lib64/libxcb-shm.so.0
Try: zypper install -C "debuginfo(build-id)=c071ae646952154ad728d39ee49c1bab34bd8baa"
Missing separate debuginfo for /usr/lib64/libxcb-keysyms.so.1
Try: zypper install -C "debuginfo(build-id)=85258e86ff17d9d93723491fba6397e1eb2e6e16"
Missing separate debuginfo for /usr/lib64/libxcb-icccm.so.4
Try: zypper install -C "debuginfo(build-id)=52220df25d52e6fea3997c80adfca78e8aea14fa"
Missing separate debuginfo for /usr/lib64/libxcb-shape.so.0
Try: zypper install -C "debuginfo(build-id)=8eade985713129bca5fa3ddae54caa58059d21a8"
Missing separate debuginfo for /usr/lib64/libxkbcommon-x11.so.0
Try: zypper install -C "debuginfo(build-id)=53fa5e1bf061f0a1d938e32fb8f5bbffeab2c8c6"
Missing separate debuginfo for /usr/lib64/libxkbcommon.so.0
Try: zypper install -C "debuginfo(build-id)=713a85232c4bdb509a873043ee863e382b6dad6f"
Missing separate debuginfo for /usr/lib64/libexpat.so.1
Try: zypper install -C "debuginfo(build-id)=d23bb128a5625e05982518a09c71702738db2f1c"
Missing separate debuginfo for /usr/lib64/libdbus-1.so.3
Try: zypper install -C "debuginfo(build-id)=2669e812f2596e5d0983e70ef2d787cb1f031106"
Missing separate debuginfo for /usr/lib64/libXext.so.6
Try: zypper install -C "debuginfo(build-id)=77d84b793aa0a8846f46b204a4225a1456274c72"
Missing separate debuginfo for /usr/lib64/libuuid.so.1
Try: zypper install -C "debuginfo(build-id)=a98948807b1c0f9bccf7af6aa0e3d610cfa7ddde"
Missing separate debuginfo for /usr/lib64/libxcb-util.so.1
Try: zypper install -C "debuginfo(build-id)=01d0aad7f59c3a5d2a6cedbc1f09acfcb1792e65"
Missing separate debuginfo for /usr/lib64/libXcursor.so.1
Try: zypper install -C "debuginfo(build-id)=29c80ca2af6bd0e9878a1f01afb0c4a528f12640"
Missing separate debuginfo for /usr/lib64/libXfixes.so.3
Try: zypper install -C "debuginfo(build-id)=236dddb1a51dac1428e369f4057220542266eb7d"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/xcbglintegrations/libqxcb-glx-integration.so
Try: zypper install -C "debuginfo(build-id)=9280246a76e5702acb96fa8decf6c9739dc9668b"
Missing separate debuginfo for /usr/lib64/libxcb-glx.so.0
Try: zypper install -C "debuginfo(build-id)=d2742c9ec652d8464f163dc38c9c19be33763a74"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/xcbglintegrations/libqxcb-egl-integration.so
Try: zypper install -C "debuginfo(build-id)=aec8e36bd6a44253565659dcbafb044b81dab793"
Missing separate debuginfo for /usr/lib64/libEGL.so.1
Try: zypper install -C "debuginfo(build-id)=78e269e19cebb874be94460d7666cc51d75ddbb5"
Missing separate debuginfo for /usr/lib64/libEGL_mesa.so.0
Try: zypper install -C "debuginfo(build-id)=727cde0e81ce12d58b18a7158dc7d6fe25aa3883"
Missing separate debuginfo for /usr/lib64/libxcb-dri2.so.0
Try: zypper install -C "debuginfo(build-id)=9684b18b002c16566a690e8d4c6bd445fb1a69f3"
Missing separate debuginfo for /usr/lib64/libxcb-dri3.so.0
Try: zypper install -C "debuginfo(build-id)=b67053bb643abd2a5848fb2f5551e7cb77ab870d"
Missing separate debuginfo for /usr/lib64/libxcb-present.so.0
Try: zypper install -C "debuginfo(build-id)=ec1da1f40db98b36e10843bedfaed567ac1c197d"
Missing separate debuginfo for /usr/lib64/libxshmfence.so.1
Try: zypper install -C "debuginfo(build-id)=954963a25d0fdb3a8cbd0b3f2b1616b2caaeddcc"
Missing separate debuginfo for /usr/lib64/libwayland-client.so.0
Try: zypper install -C "debuginfo(build-id)=632b347616eacba711596020fe35a8e506b15b97"
Missing separate debuginfo for /usr/lib64/libgbm.so.1
Try: zypper install -C "debuginfo(build-id)=1538b47f0aca5727142b1caa765ab00c1196ad82"
Missing separate debuginfo for /usr/lib64/libwayland-server.so.0
Try: zypper install -C "debuginfo(build-id)=e5fdad8ffed17e570916c98d3548e5e8e9129057"
Missing separate debuginfo for /usr/lib64/libdrm.so.2
Try: zypper install -C "debuginfo(build-id)=0d120ed25825453e351e1c3aaf8d8eee9afbeadb"
Missing separate debuginfo for /usr/lib64/libglapi.so.0
Try: zypper install -C "debuginfo(build-id)=3de6654799a1be3e4bdfd47eae897ccfb0bfb89f"
Missing separate debuginfo for /usr/lib64/libffi.so.7
Try: zypper install -C "debuginfo(build-id)=35fce3521b13419f0ce3ad612fe495a4e802fb57"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so
Try: zypper install -C "debuginfo(build-id)=a02f8842eb31dee0facff44f968859fda81198dc"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_system.so.2
Try: zypper install -C "debuginfo(build-id)=1997a09b15c239d0f9f548b90599f01992177509"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/cgi/escape.so
Try: zypper install -C "debuginfo(build-id)=8f181d166af95a12ad3d3cbd7f39855ebddc087b"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/strscan.so
Try: zypper install -C "debuginfo(build-id)=c631cce895ed6546d65f5b287053ca296af3cff4"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_ini.so.2
Try: zypper install -C "debuginfo(build-id)=8e51c8eb9346a2c8f5d73bef1b61ec3821636e53"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/pathname.so
Try: zypper install -C "debuginfo(build-id)=c14df070c0bf25b0187bf66253c657195480b9c0"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/imageformats/libqgif.so
Try: zypper install -C "debuginfo(build-id)=38fe0734a50357f053abf6e427a471f7fc3296c6"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/imageformats/libqico.so
Try: zypper install -C "debuginfo(build-id)=82d4fd019d3c06b841f1da29781bc231cfd6c8fe"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/qt5/plugins/imageformats/libqjpeg.so
Try: zypper install -C "debuginfo(build-id)=403c9a40fae82de23d3264f93f01d0234dd33fbc"
Missing separate debuginfo for /usr/lib64/libjpeg.so.8
Try: zypper install -C "debuginfo(build-id)=4e98563e427ec6068e9ecc3174ca8a4884871494"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/etc.so
Try: zypper install -C "debuginfo(build-id)=a163af4fc38aa2c8eab5fa5e878c6d7b3f0aba53"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_xml.so.2
Try: zypper install -C "debuginfo(build-id)=795871b0380aaf017cface629a825704bca0170b"
Missing separate debuginfo for /usr/lib64/libxml2.so.2
Try: zypper install -C "debuginfo(build-id)=268f563a2138dc5077c0813194b5be7a9e464a19"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/vendor_ruby/2.5.0/x86_64-linux-gnu/storage.so
Try: zypper install -C "debuginfo(build-id)=edbb310d8e5a9ad7346bb212facafdbd86dc66ad"
Missing separate debuginfo for /usr/lib64/libstorage-ng.so.1
Try: zypper install -C "debuginfo(build-id)=5fda583066a773c275a574e723fab5f12cb3ca34"
Missing separate debuginfo for /usr/lib64/libjson-c.so.4
Try: zypper install -C "debuginfo(build-id)=a4b9022645e3c6aa2452769f099987a7af565dd0"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/psych.so
Try: zypper install -C "debuginfo(build-id)=f9c3875b8447633cc37b129690496ab39ddb105c"
Missing separate debuginfo for /usr/lib64/libyaml-0.so.2
Try: zypper install -C "debuginfo(build-id)=a5d1cd800e59d00e20a758239610c9214c5b0cf0"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2Pkg.so.2
Try: zypper install -C "debuginfo(build-id)=c7fed5d616dd4ca170648711274a55fe009f6046"
Missing separate debuginfo for /usr/lib64/libzypp.so.1702
Try: zypper install -C "debuginfo(build-id)=e075d4e418644a822c0a3f1773b352b3b9971b1a"
Missing separate debuginfo for /usr/lib64/librpm.so.8
Try: zypper install -C "debuginfo(build-id)=c717114d8bd38aff25e50469382668b6d79df66d"
Missing separate debuginfo for /usr/lib64/libcurl.so.4
Try: zypper install -C "debuginfo(build-id)=a198eced2d9b531436c96f76470d76f7af166c5c"
Missing separate debuginfo for /usr/lib64/libcrypto.so.1.1
Try: zypper install -C "debuginfo(build-id)=07a13ac81c84780a5e6e5569e4430c41c2bb4b74"
Missing separate debuginfo for /usr/lib64/libboost_thread.so.1.66.0
Try: zypper install -C "debuginfo(build-id)=53ad1f06daef1943578943e9f3973a7b41f66304"
Missing separate debuginfo for /usr/lib64/libgpgme.so.11
Try: zypper install -C "debuginfo(build-id)=41464a5eae5d729d5192a7edcc3657b510dda1fe"
Missing separate debuginfo for /usr/lib64/libudev.so.1
Try: zypper install -C "debuginfo(build-id)=0d223f40ccd63fc5bf81f37bf22eba2629e99e2c"
Missing separate debuginfo for /usr/lib64/libproxy.so.1
Try: zypper install -C "debuginfo(build-id)=4c0d2e57adbe0a9b9e493c7af10da8fb96c03976"
Missing separate debuginfo for /usr/lib64/librpmio.so.8
Try: zypper install -C "debuginfo(build-id)=69c1f44ad9c6fef345042e371a39d12779973638"
Missing separate debuginfo for /usr/lib64/libpopt.so.0
Try: zypper install -C "debuginfo(build-id)=ed66614dd7c5e8841e72156a84a60d23a8ccde46"
Missing separate debuginfo for /lib64/libacl.so.1
Try: zypper install -C "debuginfo(build-id)=73290eb25ca84151cfd051deffe0a67e2386ed44"
Missing separate debuginfo for /usr/lib64/liblua5.3.so.5
Try: zypper install -C "debuginfo(build-id)=f63c68f37c5f11e439b6464dba85692df8514267"
Missing separate debuginfo for /usr/lib64/libnghttp2.so.14
Try: zypper install -C "debuginfo(build-id)=8358666f75463e611218e73ffbde11bc4db0c09d"
Missing separate debuginfo for /usr/lib64/libidn2.so.0
Try: zypper install -C "debuginfo(build-id)=007a5305d719262ac51b77731593dd9970ae46b4"
Missing separate debuginfo for /usr/lib64/libssh.so.4
Try: zypper install -C "debuginfo(build-id)=ffcb0cbda6a86dcfebfe8868c5b8d2fbf76f76e0"
Missing separate debuginfo for /usr/lib64/libpsl.so.5
Try: zypper install -C "debuginfo(build-id)=f2aa52668fd4866376200f88330a02822706524d"
Missing separate debuginfo for /usr/lib64/libssl.so.1.1
Try: zypper install -C "debuginfo(build-id)=4363a1da3fc5463bacad5a9888159361192e2434"
Missing separate debuginfo for /usr/lib64/libgssapi_krb5.so.2
Try: zypper install -C "debuginfo(build-id)=e0fa490e5a35b48c566dab950d0cd8db12075350"
Missing separate debuginfo for /usr/lib64/libldap_r-2.4.so.2
Try: zypper install -C "debuginfo(build-id)=e956f63d21048ae7def470b5888b48689a904fb2"
Missing separate debuginfo for /usr/lib64/liblber-2.4.so.2
Try: zypper install -C "debuginfo(build-id)=4f2d3d4d22765ca818369b5a1bb6cec5fe84e4ee"
Missing separate debuginfo for /usr/lib64/libboost_system.so.1.66.0
Try: zypper install -C "debuginfo(build-id)=142a763776250acbacd208f902a6fd4ba3640839"
Missing separate debuginfo for /usr/lib64/libassuan.so.0
Try: zypper install -C "debuginfo(build-id)=c675c2dde31f944491ef5fffb50846fb4dcddf9d"
Missing separate debuginfo for /usr/lib64/libmodman.so.1
Try: zypper install -C "debuginfo(build-id)=0ab580ff1e6aca6f78444ae92a23ef63cabde16b"
Missing separate debuginfo for /usr/lib64/libelf.so.1
Try: zypper install -C "debuginfo(build-id)=9b090a9176a6d8f491466c36af246a277fbe6d30"
Missing separate debuginfo for /lib64/libattr.so.1
Try: zypper install -C "debuginfo(build-id)=03e6a516d4bd1c49172c6979e60b10ee409b817e"
Missing separate debuginfo for /usr/lib64/libunistring.so.2
Try: zypper install -C "debuginfo(build-id)=797e9996d81d5fa361291d4f23443f17a05be239"
Missing separate debuginfo for /usr/lib64/libkrb5.so.3
Try: zypper install -C "debuginfo(build-id)=eb9879e30cd1ca77539c5f00b433e3b861374ba5"
Missing separate debuginfo for /usr/lib64/libk5crypto.so.3
Try: zypper install -C "debuginfo(build-id)=3815a903ae94a07e3c27678e49ba8ca7e74b1a1f"
Missing separate debuginfo for /lib64/libcom_err.so.2
Try: zypper install -C "debuginfo(build-id)=db487310e6799efb68bd20471fa3210daa13b42f"
Missing separate debuginfo for /usr/lib64/libkrb5support.so.0
Try: zypper install -C "debuginfo(build-id)=1c83ddf39702baecf80e9fe3422d9112a0956180"
Missing separate debuginfo for /lib64/libresolv.so.2
Try: zypper install -C "debuginfo(build-id)=701fa9bbc2c8e2ad23a7d8e3a75820ea8187b32e"
Missing separate debuginfo for /usr/lib64/libsasl2.so.3
Try: zypper install -C "debuginfo(build-id)=6d8ed86cbcc108428b2debc1c14f38ed9dd747c6"
Missing separate debuginfo for /usr/lib64/libkeyutils.so.1
Try: zypper install -C "debuginfo(build-id)=3b34f0d61c742cf1956c1af133ae23af792e3a0b"
Missing separate debuginfo for /lib64/libselinux.so.1
Try: zypper install -C "debuginfo(build-id)=9df04087df5dad57730f2b547bae38d7708b2e27"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/digest.so
Try: zypper install -C "debuginfo(build-id)=31e27608c89b0897f180832429742061aeabe7c3"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/date_core.so
Try: zypper install -C "debuginfo(build-id)=eb24684b43156b6b3329964adf9a74490ed38fd3"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_hwprobe.so.2
Try: zypper install -C "debuginfo(build-id)=a8d698e6b4203068b9cbf4ad5c7ced9a45bbf644"
Missing separate debuginfo for /usr/lib64/libhd.so.21
Try: zypper install -C "debuginfo(build-id)=c43b640cfddeeed20130616033343d0e25569f65"
Missing separate debuginfo for /usr/lib64/libx86emu.so.1
Try: zypper install -C "debuginfo(build-id)=de92351e066f32b58c1e0c92b5a730f80ca88c51"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/gems/2.5.0/gems/ruby-augeas-0.5.0/lib/_augeas.so
Try: zypper install -C "debuginfo(build-id)=4b9c81d305ed925ef5e696b956c673c765e731af"
Missing separate debuginfo for /usr/lib64/libaugeas.so.0
Try: zypper install -C "debuginfo(build-id)=cec49256b497da20a3a3b165d810cce3d2845bd4"
Missing separate debuginfo for /usr/lib64/libfa.so.1
Try: zypper install -C "debuginfo(build-id)=1d93a280da31b2ae8d38c3fe5c48fa60697a91f0"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_modules.so.2
Try: zypper install -C "debuginfo(build-id)=2e3dc7e07184402e253f3a7d0dab5319951c8196"
Missing separate debuginfo for /usr/lib/perl5/vendor_perl/5.26.2/x86_64-linux-thread-multi/auto/YaST/YCP/libYCP.so
Try: zypper install -C "debuginfo(build-id)=6656d304cffc53f50ec6b5a8c847a7320931d0e5"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/auto/Data/Dumper/Dumper.so
Try: zypper install -C "debuginfo(build-id)=aff109edfba8f5f7c2f00d687b4ca3496c451249"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/digest/sha2.so
Try: zypper install -C "debuginfo(build-id)=c02d32584eff41285ff9089a8fcc07a11595ed43"
Missing separate debuginfo for /mounts/mp_0001/usr/lib64/ruby/2.5.0/x86_64-linux-gnu/fcntl.so
Try: zypper install -C "debuginfo(build-id)=3cae930d8dcd2ad9c42396b30ca2469313e43826"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_anyagent.so.2
Try: zypper install -C "debuginfo(build-id)=c216b64bfbfe5eeae563a803bfa6a03af247ba80"
Missing separate debuginfo for /usr/lib64/yui/libyui-qt-graph.so.8
Try: zypper install -C "debuginfo(build-id)=498925bda172081a5090f6ae60feef5df680423f"
Missing separate debuginfo for /usr/lib64/libgvc.so.6
Try: zypper install -C "debuginfo(build-id)=0e0bb11706b7395a101ef5ca956a83af52c7062f"
Missing separate debuginfo for /usr/lib64/libcgraph.so.6
Try: zypper install -C "debuginfo(build-id)=3b784c9bd7fc20ea8531db9ca0260e0c683e7b1b"
Missing separate debuginfo for /usr/lib64/libltdl.so.7
Try: zypper install -C "debuginfo(build-id)=041b64d0f405b68a93e23211dca65af3db256048"
Missing separate debuginfo for /usr/lib64/libcdt.so.5
Try: zypper install -C "debuginfo(build-id)=f7b3c9be7c6d6ed2978c93cb26cec2928f8cf0ba"
Missing separate debuginfo for /usr/lib64/libpathplan.so.4
Try: zypper install -C "debuginfo(build-id)=6bfe7e3ce311eba9c5fe623962ac877c54c6ceaa"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/auto/Sys/Hostname/Hostname.so
Try: zypper install -C "debuginfo(build-id)=7182b30259425979e21d686c74c425dbbe475589"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/auto/Fcntl/Fcntl.so
Try: zypper install -C "debuginfo(build-id)=e7f3b806e8a0feed6cfefe542caa4762edb79b3b"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/auto/POSIX/POSIX.so
Try: zypper install -C "debuginfo(build-id)=7fd8209df2802c059c7d02dd62eaadcd0e6a3a39"
Missing separate debuginfo for /usr/lib/perl5/5.26.2/x86_64-linux-thread-multi/auto/Encode/Encode.so
Try: zypper install -C "debuginfo(build-id)=5e3d1d74dae4c39786e809f26087e0df332a8a5b"
Missing separate debuginfo for /usr/lib/perl5/vendor_perl/5.26.2/x86_64-linux-thread-multi/auto/Locale/gettext/gettext.so
Try: zypper install -C "debuginfo(build-id)=deaeef7a2f36ae650f269a765dad8e487bfcb4d6"
Missing separate debuginfo for /usr/lib64/YaST2/plugin/libpy2ag_crack.so.2
Try: zypper install -C "debuginfo(build-id)=0bea53571d8d2f12469ffc43526d1e4f2d4ef1b3"
Missing separate debuginfo for /usr/lib64/libcrack.so.2
Try: zypper install -C "debuginfo(build-id)=ad34f0daa653cc6155e1873c945b9458f0625d0d"
0x00007f369e1812ad in zypp::base::logger::getStream(char const*, zypp::base::logger::LogLevel, char const*, char const*, int) () from /usr/lib64/libzypp.so.1702
Updated by mloviska over 6 years ago
I have tried to setup gdb according to https://en.opensuse.org/openSUSE:YaST_debugging. Installer does not even start whenever I have tried to use Y2GDB as boot arg.
Regarding GDB symbols:
jreidinger mloviska: missing symbols is probably expected as in insts-sys there is no sources
Updated by mloviska over 6 years ago
Signals are being trapped by YaST, therefore I am not able to force core dump.
Tested signals (SEGV,QUIT,TERM,SIGKILL)
Core dump settings:
1) core pattern = core
2) ulimit set to unlimited
Updated by okurz over 6 years ago
I think the gdb stack trace can be still helpful even without additional debug symbols. If we see a scenario where we reproducibly get a stack trace from gdb we might then try to even inject debug symbols, e.g. over a DUD that is added to the installer. You could maybe silence all the warnings about the missing debug symbols in the output.
Also, did you try strace? You could attach to the installer process with strace which can help to find out when it's seemingly stuck if it is actually in a deadlock in all processes waiting for some external event or if some thread are in a busy-wait.
Also, do you know about cat /proc/<pid>/stack? This could help as well by getting the stack trace of the installer process (or some processes), especially as the installer calls many external tools so it might be even more likely that the problem is in an external tool, e.g. mount. Thinking about it, I don't even know if we even call the ps axf command or similar in a post_fail_hook in the installer or only on console tests later in the installed system. Seeing a process overview could help a lot as well.
Updated by mloviska over 6 years ago
Except of /proc//stack I am uploading also mounts, mountinfo, maps, status. Maybe even task directory might be useful. It is not an issue to attach strace to the installer process, however I do see many benefits doing it in post_fail_hook. There is an option to pass strace information in Y2DEBUG. For instance Y2DEBUG=1 strace -eopen -ostrace.log /usr/lib/YaST2/bin/y2base lan qt. At this point I am trying to somehow figure out how to run it on installer process. Currently we are not running ps/pstree command in y2logsstep::post_fail_hook.
According to Yast team gdb data is not so much helpful when it is used to gather information about ruby.
Updated by mloviska over 6 years ago
pstree during installation.
init-+-bash-+-pstree
| `-tee
|-2*[bash]
|-dbus-daemon
|-2*[gpg-agent]
|-haveged
|-icewm
|-init---inst_setup---yast2---YaST2.First-Sta---YaST2.call-+-Xorg---{Xorg}
| `-y2start-+-ag_udev_persist---{ag_udev_persist}
| `-4*[{y2start}]
|-klogd
|-nscd---8*[{nscd}]
|-rsyslogd---4*[{rsyslogd}]
|-sh
|-udevd
|-wickedd
|-wickedd-auto4
|-wickedd-dhcp4
|-wickedd-dhcp6
|-wickedd-nanny
`-wpa_supplicant
Updated by okurz over 6 years ago
mloviska wrote:
There is an option to pass strace information in Y2DEBUG. For instance
Y2DEBUG=1 strace -eopen -ostrace.log /usr/lib/YaST2/bin/y2base lan qt.At this point I am trying to somehow figure out how to run it on installer process
strace -f -p $(pidof y2base) should work
Updated by mloviska over 6 years ago
That does not lead to any useful information. We need to start the trace before post_fail_hook is called.
http://dhcp151.suse.cz/tests/3469/file/partitioning_togglehome-installer_trace.log
Updated by okurz over 6 years ago
I think it can still provide valuable information as stated in #35709#note-6 "to find out when it's seemingly stuck if it is actually in a deadlock in all processes waiting for some external event or if some thread are in a busy-wait."
If a process is in a "busy-wait" you will see that a lot of stuf is going on, if it's deadlocked then you see something like just the single line you provided.
Updated by mloviska over 6 years ago
- Status changed from Workable to Feedback
Updated by okurz over 6 years ago
- Target version changed from Milestone 17 to Milestone 17
Updated by riafarov over 6 years ago
- Status changed from Feedback to Resolved
Updated by okurz over 6 years ago
- Due date changed from 2018-06-19 to 2018-07-03
- Status changed from Resolved to In Progress
As discussed in the review, I assume it is more usable if gdb also collects the stack trace from all threads, e.g. take a look here https://github.com/okurz/scripts/blob/master/gdb-thread-all-bt
so this is what we can try in the next sprint. If it turns out for a specific reason that the gdb batch mode can not work in this situation I guess we are also ok like it is.
One more thing: How feasible do you think it would be to only call the extended investigation in case the error is not already obvious from parsing the y2log? E.g. if the installer just shows a popup that a package dependency can not be provided all further screenshots as well as logs are not helping and just distracting from the already known issue.
Updated by mloviska over 6 years ago
I would say we are risking some case when the strace/gdb would be possibly helpful but it won't be executed, because it was excluded. Maybe it is more a responsibility of the reviewer to include only the useful data to bugzilla records.
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5256
Updated by riafarov over 6 years ago
- Precedes action #37126: [functional][y] collect strace output for the installer to debug sporadic issues added
Updated by mloviska over 6 years ago
Verification run in osd
https://openqa.suse.de/tests/1772210/file/select_patterns_and_packages-installer_gdb.log
Updated by mloviska over 6 years ago
yast2 log parser was not affected by post_fail_hook code update:
https://openqa.suse.de/tests/1772944#step/select_patterns_and_packages/173
http://dhcp151.suse.cz/tests/3601#step/select_patterns_and_packages/262
Updated by mloviska over 6 years ago
- Status changed from In Progress to Resolved
Updated by okurz over 6 years ago
ok, for the time being I expect that the additional data can help some investigation and also your work serves as a great example of what we can apply in other cases as well when we see problems elsewhere where we e.g. want to understand where a process is stuck. I expect in the future we still want to not always execute all steps as they cost time, storage and a bit of confusing logs when we do not always need them but I guess it makes sense if we collect some feedback first based on the current state before we plan this.
Actions