Actions
action #105618
closed[Alerting] CPU Load alert size:S
Added by okurz over 2 years ago. Updated about 2 years ago.
Start date:
2022-01-27
Due date:
% Done:
0%
Estimated time:
Description
Observation¶
5 Minutes Average has a value of 64.389 , that's a lot, right?
Suggestions¶
- NOT postgres vaccuming
- A surge of scheduled jobs
- Consider increasing the threshold if we have a good hypothesis to accept what acceptable situation led to the alert
- mkittler observed a gap in monitoring data for the same time period. Could point to the same underlying problem
Files
| poo#105618.logs (4.03 MB) poo#105618.logs | nicksinger, 2022-01-27 16:24 |
Updated by okurz over 2 years ago
- Copied to action #105621: [Alerting] Failed systemd services alert added
Updated by nicksinger over 2 years ago
I've updated the link to include a more precise timestamp. What comes to my attention is how our metrics look in general at this point in time. We have missing data for a lot of metrics and the CPU graph shows that shortly after we receive data again we have >90% system utilization. "IO-wait" is quite low and storage graphs don't seem to be extra ordinarily high so this indicates we didn't see storage related problems here. However, shortly before we start missing metrics the RAM usage skyrocketed. After it cooled down, RAM was fine too again (higher but this could be explained by piled up minion jobs).
Updated by nicksinger over 2 years ago
- File poo#105618.logs poo#105618.logs added
I found something interesting in the journal with journalctl --since "2022-01-26 15:15:00" --until "2022-01-26 15:20:00":
-- Logs begin at Sat 2022-01-22 04:23:25 CET, end at Thu 2022-01-27 17:18:43 CET. --
Jan 26 15:15:01 openqa systemd[1]: Started Session 10654 of user geekotest.
Jan 26 15:15:01 openqa systemd[1]: Started Session 10657 of user geekotest.
Jan 26 15:15:01 openqa cron[27986]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa systemd[1]: Started Session 10656 of user geekotest.
Jan 26 15:15:01 openqa cron[27984]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa systemd[1]: Created slice User Slice of UID 17307.
Jan 26 15:15:01 openqa CRON[27996]: (geekotest) CMD (([ "$(df --output=pcent /results/testresults | sed '1d;s/[^0-9]//g')" -ge 88 ] && (! pgrep --full --list-full 'find /results/testresults' >/dev/null) && find /results/testresults -type f -iname '*.ogv' -mtime +28 -delete))
Jan 26 15:15:01 openqa systemd[1]: Starting User Runtime Directory /run/user/17307...
Jan 26 15:15:01 openqa systemd[1]: Started Session 10658 of user geekotest.
Jan 26 15:15:01 openqa systemd[1]: Started Session 10660 of user geekotest.
Jan 26 15:15:01 openqa systemd[1]: Started Session 10655 of user geekotest.
Jan 26 15:15:01 openqa cron[27990]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa CRON[27986]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:01 openqa cron[27987]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa systemd[1]: session-10654.scope: Succeeded.
Jan 26 15:15:01 openqa cron[27988]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa CRON[28019]: (geekotest) CMD ((cd /var/lib/openqa/share/factory/repo/ && for i in fixed/SLE-15-*-s390x-GM-*1/ ; do ln -sf $i ; done))
Jan 26 15:15:01 openqa cron[27989]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:15:01 openqa systemd[1]: Finished User Runtime Directory /run/user/17307.
Jan 26 15:15:01 openqa systemd[1]: Starting User Manager for UID 17307...
Jan 26 15:15:01 openqa CRON[28027]: (geekotest) CMD ((cd /var/lib/openqa/share/factory/repo/ && for i in fixed/SLE-12-{,SP?-}Server-DVD-s390x-GM-DVD1/ ; do ln -sf $i ; done))
Jan 26 15:15:01 openqa CRON[28022]: (geekotest) CMD ((cd /var/lib/openqa/share/factory/repo/ && for i in fixed/SLE-12-SP?-SDK-POOL-{x86_64,aarch64,ppc64le,s390x}-BuildGM-Media1/ ; do ln -sf $i ; done))
Jan 26 15:15:01 openqa systemd[28030]: pam_unix(systemd-user:session): session opened for user cvdupdate by (uid=0)
Jan 26 15:15:02 openqa CRON[27988]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:02 openqa systemd[1]: session-10660.scope: Succeeded.
Jan 26 15:15:02 openqa CRON[27989]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:02 openqa systemd[1]: session-10655.scope: Succeeded.
Jan 26 15:15:02 openqa CRON[27987]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:02 openqa systemd[1]: session-10658.scope: Succeeded.
Jan 26 15:15:02 openqa systemd[28030]: Queued start job for default target Main User Target.
Jan 26 15:15:02 openqa systemd[28030]: Reached target Paths.
Jan 26 15:15:02 openqa systemd[28030]: Reached target Timers.
Jan 26 15:15:02 openqa systemd[28030]: Starting D-Bus User Message Bus Socket.
Jan 26 15:15:02 openqa systemd[28030]: Listening on D-Bus User Message Bus Socket.
Jan 26 15:15:02 openqa systemd[28030]: Reached target Sockets.
Jan 26 15:15:02 openqa systemd[28030]: Reached target Basic System.
Jan 26 15:15:02 openqa cron[27985]: pam_unix(crond:session): session opened for user cvdupdate by (uid=0)
Jan 26 15:15:02 openqa systemd[28030]: Reached target Main User Target.
Jan 26 15:15:02 openqa CRON[28114]: (cvdupdate) CMD (/home/cvdupdate/.local/bin/cvdupdate update)
Jan 26 15:15:02 openqa systemd[28030]: Startup finished in 815ms.
Jan 26 15:15:02 openqa CRON[27984]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:02 openqa systemd[1]: Started User Manager for UID 17307.
Jan 26 15:15:02 openqa systemd[1]: Started Session 10659 of user cvdupdate.
Jan 26 15:15:02 openqa systemd[1]: session-10657.scope: Succeeded.
Jan 26 15:15:05 openqa CRON[27985]: pam_unix(crond:session): session closed for user cvdupdate
Jan 26 15:15:05 openqa systemd[1]: session-10659.scope: Succeeded.
Jan 26 15:15:05 openqa postfix/pickup[22426]: C240A343F2: uid=17307 from=<cvdupdate>
Jan 26 15:15:05 openqa postfix/cleanup[28431]: C240A343F2: message-id=<20220126141505.C240A343F2@localhost>
Jan 26 15:15:05 openqa postfix/qmgr[1688]: C240A343F2: from=<cvdupdate@localhost>, size=1127, nrcpt=1 (queue active)
Jan 26 15:15:05 openqa postfix/local[28433]: C240A343F2: to=<cvdupdate@localhost>, orig_to=<cvdupdate>, relay=local, delay=0.26, delays=0.2/0.06/0/0.01, dsn=2.0.0, status=sent (delivered to mailbox)
Jan 26 15:15:05 openqa postfix/qmgr[1688]: C240A343F2: removed
Jan 26 15:15:10 openqa systemd[1]: session-10656.scope: Succeeded.
Jan 26 15:15:10 openqa CRON[27990]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:15:15 openqa systemd[1]: Stopping User Manager for UID 17307...
Jan 26 15:15:15 openqa systemd[28030]: Stopped target Main User Target.
Jan 26 15:15:15 openqa systemd[28030]: Stopped target Basic System.
Jan 26 15:15:15 openqa systemd[28030]: Stopped target Paths.
Jan 26 15:15:15 openqa systemd[28030]: Stopped target Sockets.
Jan 26 15:15:15 openqa systemd[28030]: Stopped target Timers.
Jan 26 15:15:15 openqa systemd[28030]: dbus.socket: Succeeded.
Jan 26 15:15:15 openqa systemd[28030]: Closed D-Bus User Message Bus Socket.
Jan 26 15:15:15 openqa systemd[28030]: Reached target Shutdown.
Jan 26 15:15:15 openqa systemd[28030]: systemd-exit.service: Succeeded.
Jan 26 15:15:15 openqa systemd[28030]: Finished Exit the Session.
Jan 26 15:15:15 openqa systemd[28030]: Reached target Exit the Session.
Jan 26 15:15:15 openqa systemd[1]: user@17307.service: Succeeded.
Jan 26 15:15:15 openqa systemd[1]: Stopped User Manager for UID 17307.
Jan 26 15:15:15 openqa systemd[1]: Stopping User Runtime Directory /run/user/17307...
Jan 26 15:15:15 openqa systemd[1298]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[25535]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[16764]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[1297]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[1]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[1116]: run-user-17307.mount: Succeeded.
Jan 26 15:15:15 openqa systemd[1200]: run-user-17307.mount: Succeeded.
Jan 26 15:15:16 openqa systemd[1]: user-runtime-dir@17307.service: Succeeded.
Jan 26 15:15:16 openqa systemd[1]: Stopped User Runtime Directory /run/user/17307.
Jan 26 15:15:16 openqa systemd[1]: Removed slice User Slice of UID 17307.
Jan 26 15:19:01 openqa systemd-journald[29174]: Journal started
Jan 26 15:19:01 openqa systemd-journald[29174]: System Journal (/var/log/journal/84f4f4f356b525388b60f0ae547597e0) is 40.9G, max 80.0G, 39.0G free.
Jan 26 15:19:01 openqa kernel: sshd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=-1000
Jan 26 15:19:01 openqa kernel: CPU: 8 PID: 1279 Comm: sshd Not tainted 5.3.18-59.40-default #1 SLE15-SP3
Jan 26 15:19:01 openqa kernel: Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Jan 26 15:19:01 openqa kernel: Call Trace:
Jan 26 15:19:01 openqa kernel: dump_stack+0x66/0x8b
Jan 26 15:19:01 openqa kernel: dump_header+0x4a/0x200
Jan 26 15:19:01 openqa kernel: oom_kill_process+0xe8/0x130
Jan 26 15:19:01 openqa kernel: out_of_memory+0x105/0x510
Jan 26 15:19:01 openqa kernel: __alloc_pages_slowpath+0x9f3/0xd20
Jan 26 15:19:01 openqa kernel: __alloc_pages_nodemask+0x2cb/0x320
Jan 26 15:19:01 openqa kernel: pagecache_get_page+0xa5/0x2c0
Jan 26 15:19:01 openqa kernel: filemap_fault+0x381/0x9b0
Jan 26 15:19:01 openqa kernel: ? xas_load+0x9/0x80
Jan 26 15:19:01 openqa kernel: ? xas_find+0x177/0x1b0
Jan 26 15:19:01 openqa kernel: ? filemap_map_pages+0xa3/0x3d0
Jan 26 15:19:01 openqa kernel: ext4_filemap_fault+0x2c/0x40 [ext4]
Jan 26 15:19:01 openqa kernel: __do_fault+0x31/0xb0
Jan 26 15:19:01 openqa kernel: __handle_mm_fault+0xdfe/0x1260
Jan 26 15:19:01 openqa kernel: handle_mm_fault+0xc4/0x200
Jan 26 15:19:01 openqa kernel: __do_page_fault+0x2ce/0x500
Jan 26 15:19:01 openqa kernel: do_page_fault+0x30/0x110
Jan 26 15:19:01 openqa kernel: async_page_fault+0x3e/0x50
Jan 26 15:19:01 openqa kernel: RIP: 0033:0x557a912b2815
Jan 26 15:19:01 openqa kernel: Code: Bad RIP value.
Jan 26 15:19:01 openqa kernel: RSP: 002b:00007ffc4c702be0 EFLAGS: 00010203
Jan 26 15:19:01 openqa kernel: RAX: 0000000000000001 RBX: 0000000000000064 RCX: 00007fc165a922c6
Jan 26 15:19:01 openqa kernel: RDX: 0000000000000000 RSI: 0000557a921a6b50 RDI: 0000000000000009
Jan 26 15:19:01 openqa kernel: RBP: 0000557a9157c820 R08: 0000000000000000 R09: 0000000000000000
Jan 26 15:19:01 openqa kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000557a921a6b50
Jan 26 15:19:01 openqa kernel: R13: 0000000000000063 R14: 000000000000018c R15: 0000000000000001
Jan 26 15:19:01 openqa kernel: Mem-Info:
Jan 26 15:19:01 openqa kernel: active_anon:4366804 inactive_anon:336196 isolated_anon:0
active_file:8296 inactive_file:8624 isolated_file:332
unevictable:6872 dirty:17356 writeback:0 unstable:0
slab_reclaimable:110044 slab_unreclaimable:48237
mapped:782865 shmem:781828 pagetables:113940 bounce:0
free:64596 free_pcp:0 free_cma:0
Jan 26 15:19:01 openqa kernel: Node 0 active_anon:17467216kB inactive_anon:1344784kB active_file:43112kB inactive_file:24704kB unevictable:27488kB isolated(anon):0kB isolated(file):1072kB mapped:3131460kB dirty:69424kB writeback:0kB shmem:3127312kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 112640kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Jan 26 15:19:01 openqa kernel: Node 0 DMA free:11812kB min:40kB low:52kB high:64kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15908kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 15:19:01 openqa kernel: lowmem_reserve[]: 0 2976 24003 24003 24003
Jan 26 15:19:01 openqa kernel: Node 0 DMA32 free:109736kB min:26808kB low:29856kB high:32904kB active_anon:2446572kB inactive_anon:226320kB active_file:7100kB inactive_file:4596kB unevictable:0kB writepending:12820kB present:3129268kB managed:3063732kB mlocked:0kB kernel_stack:3016kB pagetables:57952kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 15:19:01 openqa kernel: lowmem_reserve[]: 0 0 21026 21026 21026
Jan 26 15:19:01 openqa kernel: Node 0 Normal free:136836kB min:212484kB low:234012kB high:255540kB active_anon:15020644kB inactive_anon:1118464kB active_file:17260kB inactive_file:36568kB unevictable:27488kB writepending:56604kB present:22020096kB managed:17345388kB mlocked:27488kB kernel_stack:17800kB pagetables:397808kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 15:19:01 openqa kernel: lowmem_reserve[]: 0 0 0 0 0
Jan 26 15:19:01 openqa kernel: Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11812kB
Jan 26 15:19:01 openqa kernel: Node 0 DMA32: 2436*4kB (UME) 2407*8kB (UME) 2024*16kB (UME) 1055*32kB (UME) 224*64kB (UME) 2*128kB (M) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 109736kB
Jan 26 15:19:01 openqa kernel: Node 0 Normal: 13361*4kB (UME) 5070*8kB (UME) 2677*16kB (UME) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 136836kB
Jan 26 15:19:01 openqa kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 26 15:19:01 openqa kernel: 814777 total pagecache pages
Jan 26 15:19:01 openqa kernel: 13318 pages in swap cache
Jan 26 15:19:01 openqa kernel: Swap cache stats: add 1916152, delete 1902840, find 38055382/38633622
Jan 26 15:19:01 openqa kernel: Free swap = 0kB
Jan 26 15:19:01 openqa kernel: Total swap = 499708kB
Jan 26 15:19:01 openqa kernel: 6291339 pages RAM
Jan 26 15:19:01 openqa kernel: 0 pages HighMem/MovableOnly
Jan 26 15:19:01 openqa kernel: 1185082 pages reserved
Jan 26 15:19:01 openqa kernel: 0 pages cma reserved
Jan 26 15:19:01 openqa kernel: 0 pages hwpoisoned
Jan 26 15:19:01 openqa kernel: Tasks state (memory values in pages):
Jan 26 15:19:01 openqa kernel: [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Jan 26 15:19:01 openqa kernel: [ 532] 0 532 3103 341 69632 802 0 haveged
Jan 26 15:19:01 openqa kernel: [ 540] 0 540 30694 956 266240 578 -1000 systemd-udevd
Jan 26 15:19:01 openqa kernel: [ 619] 0 619 85559 6663 221184 0 -1000 multipathd
Jan 26 15:19:01 openqa kernel: [ 692] 0 692 13177 232 131072 90 -1000 auditd
Jan 26 15:19:01 openqa kernel: [ 696] 0 696 11125 0 122880 91 0 rpc.idmapd
Jan 26 15:19:01 openqa kernel: [ 712] 0 712 27239 533 245760 215 0 cupsd
Jan 26 15:19:01 openqa kernel: [ 713] 499 713 16904 1004 176128 111 -900 dbus-daemon
Jan 26 15:19:01 openqa kernel: [ 720] 0 720 27680 1246 249856 191 0 systemd-logind
Jan 26 15:19:01 openqa kernel: [ 722] 0 722 11660 720 139264 299 0 wickedd-auto4
Jan 26 15:19:01 openqa kernel: [ 724] 0 724 11661 733 135168 299 0 wickedd-dhcp4
Jan 26 15:19:01 openqa kernel: [ 726] 0 726 11661 569 139264 257 0 wickedd-dhcp6
Jan 26 15:19:01 openqa kernel: [ 767] 0 767 11697 867 135168 240 0 wickedd
Jan 26 15:19:01 openqa kernel: [ 796] 0 796 11667 100 143360 258 0 wickedd-nanny
Jan 26 15:19:01 openqa kernel: [ 1215] 0 1215 27550 254 249856 264 0 rebootmgrd
Jan 26 15:19:01 openqa kernel: [ 1217] 0 1217 6270 443 94208 127 0 rsyncd
Jan 26 15:19:01 openqa kernel: [ 1219] 0 1219 71029 1460 598016 9123 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1221] 0 1221 50734 733 425984 4841 0 salt-minion
Jan 26 15:19:01 openqa kernel: [ 1243] 475 1243 30914 381 155648 88 0 chronyd
Jan 26 15:19:01 openqa kernel: [ 1244] 0 1244 11607 464 126976 96 0 xinetd
Jan 26 15:19:01 openqa kernel: [ 1248] 0 1248 849306 13417 815104 8496 0 telegraf
Jan 26 15:19:01 openqa kernel: [ 1260] 493 1260 13891 810 155648 94 0 rpcbind
Jan 26 15:19:01 openqa kernel: [ 1272] 492 1272 23380 548 221184 12416 0 rpc.statd
Jan 26 15:19:01 openqa kernel: [ 1279] 0 1279 18586 820 184320 194 -1000 sshd
Jan 26 15:19:01 openqa kernel: [ 1297] 0 1297 22224 604 204800 265 0 systemd
Jan 26 15:19:01 openqa kernel: [ 1298] 1001 1298 22224 949 217088 260 0 systemd
Jan 26 15:19:01 openqa kernel: [ 1305] 26 1305 835822 16905 421888 256 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1316] 0 1316 16924 498 167936 6775 0 rpc.mountd
Jan 26 15:19:01 openqa kernel: [ 1323] 497 1323 11969 469 139264 181 -500 nrpe
Jan 26 15:19:01 openqa kernel: [ 1329] 0 1329 82871 763 679936 671 0 smbd
Jan 26 15:19:01 openqa kernel: [ 1335] 1001 1335 33594 60 282624 853 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 1336] 0 1336 33594 60 282624 853 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 1438] 26 1438 25762 390 217088 301 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1462] 0 1462 81362 56 647168 675 0 smbd-notifyd
Jan 26 15:19:01 openqa kernel: [ 1463] 0 1463 81364 48 634880 667 0 cleanupd
Jan 26 15:19:01 openqa kernel: [ 1475] 26 1475 835855 693769 6184960 259 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1477] 26 1477 835855 303007 5476352 261 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1478] 26 1478 835822 4590 266240 270 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1479] 26 1479 835997 556 266240 322 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1480] 26 1480 26368 329 225280 271 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1481] 26 1481 835929 347 245760 311 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1500] 0 1500 82998 468 667648 643 0 lpqd
Jan 26 15:19:01 openqa kernel: [ 1516] 0 1516 35826 1214 327680 321 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1587] 0 1587 9589 291 114688 67 0 agetty
Jan 26 15:19:01 openqa kernel: [ 1591] 0 1591 9589 447 118784 68 0 agetty
Jan 26 15:19:01 openqa kernel: [ 1676] 0 1676 303623 8625 729088 6812 0 salt-minion
Jan 26 15:19:01 openqa kernel: [ 1686] 0 1686 22489 528 180224 142 0 master
Jan 26 15:19:01 openqa kernel: [ 1688] 51 1688 27139 265 176128 127 0 qmgr
Jan 26 15:19:01 openqa kernel: [ 1706] 0 1706 10895 516 126976 168 0 cron
Jan 26 15:19:01 openqa kernel: [ 1720] 0 1720 53485 718 446464 6148 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1721] 0 1721 71890 300 450560 6525 0 salt-minion
Jan 26 15:19:01 openqa kernel: [ 1727] 0 1727 91888 1802 585728 8969 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1730] 0 1730 77405 5694 626688 9620 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1733] 0 1733 297356 6874 708608 5545 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1735] 0 1735 269436 17090 741376 1831 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1736] 0 1736 70985 1279 577536 9113 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1738] 0 1738 165704 3077 634880 9570 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1739] 0 1739 265103 12755 704512 1785 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1748] 0 1748 265131 12531 712704 1872 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1749] 0 1749 265103 12605 708608 1812 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1750] 0 1750 265194 12480 708608 1899 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1752] 0 1752 89910 2489 593920 8904 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 1753] 0 1753 265024 12675 704512 1686 0 salt-master
Jan 26 15:19:01 openqa kernel: [ 3217] 26 3217 836302 75133 1130496 359 0 postgres
Jan 26 15:19:01 openqa kernel: [ 3218] 26 3218 836304 3315 303104 443 0 postgres
Jan 26 15:19:01 openqa kernel: [ 3237] 26 3237 836498 3493 315392 373 0 postgres
Jan 26 15:19:01 openqa kernel: [ 5348] 26 5348 867579 243570 5758976 1426 0 postgres
Jan 26 15:19:01 openqa kernel: [ 17168] 26 17168 853622 154280 2170880 2202 0 postgres
Jan 26 15:19:01 openqa kernel: [ 29178] 0 29178 170234 1330 1191936 186 -250 systemd-journal
Jan 26 15:19:01 openqa kernel: [ 1200] 17254 1200 22226 559 217088 253 0 systemd
Jan 26 15:19:01 openqa kernel: [ 1201] 17254 1201 34092 166 290816 980 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 16332] 17254 16332 13822 495 135168 824 0 screen
Jan 26 15:19:01 openqa kernel: [ 16333] 17254 16333 12789 641 147456 1246 0 bash
Jan 26 15:19:01 openqa kernel: [ 28183] 30 28183 35956 623 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28184] 30 28184 35958 856 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28187] 30 28187 40336 5110 348160 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28188] 30 28188 35948 622 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28189] 30 28189 35958 654 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28191] 30 28191 35958 607 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28192] 30 28192 35960 895 315392 288 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28193] 30 28193 35961 608 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28194] 30 28194 35961 1023 315392 288 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28196] 30 28196 35958 859 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28197] 30 28197 35955 921 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28198] 30 28198 35950 983 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28199] 30 28199 35958 926 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28202] 30 28202 35957 906 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28203] 30 28203 35956 917 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28204] 30 28204 35956 741 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28206] 30 28206 35956 908 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28207] 30 28207 35956 836 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28208] 30 28208 35954 798 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28211] 30 28211 35954 875 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28212] 30 28212 35952 783 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28213] 30 28213 35959 916 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28215] 30 28215 35957 627 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28216] 30 28216 35957 911 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28217] 30 28217 35961 907 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28218] 30 28218 35955 914 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28220] 30 28220 35954 910 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28221] 30 28221 35960 633 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28222] 30 28222 37101 1889 323584 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28223] 30 28223 35955 825 315392 291 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28224] 30 28224 35960 912 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28225] 30 28225 35960 925 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28226] 30 28226 35961 910 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28227] 30 28227 35958 900 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28228] 30 28228 35957 907 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28229] 30 28229 35959 903 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28230] 30 28230 35959 725 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28232] 30 28232 35956 910 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28233] 30 28233 35958 725 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28234] 30 28234 35951 704 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28235] 30 28235 35954 898 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28236] 30 28236 35957 812 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28238] 30 28238 35963 617 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28239] 30 28239 35951 903 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28240] 30 28240 35959 912 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28241] 30 28241 35959 909 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28242] 30 28242 35961 914 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28243] 30 28243 35955 895 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28244] 30 28244 35959 635 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28245] 30 28245 35959 708 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28246] 30 28246 35957 1042 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28247] 30 28247 35957 744 315392 317 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28248] 30 28248 35948 844 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28249] 30 28249 35960 619 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28250] 30 28250 35955 825 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28251] 30 28251 35963 911 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28253] 30 28253 35959 907 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28254] 30 28254 35957 615 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28255] 30 28255 35959 873 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28256] 30 28256 35960 915 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28257] 30 28257 35967 829 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28258] 30 28258 35978 922 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28259] 30 28259 35985 1056 315392 272 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28261] 30 28261 35958 854 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28262] 30 28262 35971 831 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28263] 30 28263 35955 892 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28265] 30 28265 35955 911 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28266] 30 28266 35956 1002 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28267] 30 28267 35953 908 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28268] 30 28268 35956 913 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28269] 30 28269 35955 803 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28270] 30 28270 35955 892 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28271] 30 28271 35959 630 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28272] 30 28272 35955 634 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28273] 30 28273 35958 911 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28274] 30 28274 35954 900 315392 293 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28275] 30 28275 35956 858 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28276] 30 28276 35953 717 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28277] 30 28277 35958 715 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28278] 30 28278 35967 848 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28279] 30 28279 35957 714 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28280] 30 28280 35953 867 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28282] 30 28282 35953 833 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28283] 30 28283 35956 788 315392 288 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28285] 30 28285 35952 880 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28315] 30 28315 35960 696 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28318] 30 28318 35958 764 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28319] 30 28319 35959 605 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28320] 30 28320 35954 679 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28354] 30 28354 35960 702 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28355] 30 28355 35959 860 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28411] 30 28411 35959 705 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28583] 30 28583 35957 837 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 28801] 30 28801 35957 851 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30814] 30 30814 35977 904 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30815] 30 30815 35959 793 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30817] 30 30817 35954 908 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30819] 30 30819 35947 890 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30820] 30 30820 35959 823 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30826] 30 30826 35956 588 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30827] 30 30827 35959 624 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30828] 30 30828 35961 820 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30829] 30 30829 35957 587 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30831] 30 30831 35956 958 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30832] 30 30832 35953 875 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30834] 30 30834 35956 826 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30841] 30 30841 35958 898 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30842] 30 30842 35956 891 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30843] 30 30843 35976 905 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30844] 30 30844 35956 588 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30846] 30 30846 35974 609 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30847] 30 30847 35955 915 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30848] 30 30848 35959 853 315392 292 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30850] 30 30850 35953 890 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30852] 30 30852 35957 894 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30854] 30 30854 35954 909 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30855] 30 30855 35977 920 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30856] 30 30856 35954 911 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30877] 30 30877 35964 917 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30878] 30 30878 35955 614 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30879] 30 30879 35958 613 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30881] 30 30881 35962 907 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30882] 30 30882 35952 932 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30884] 30 30884 35958 833 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30885] 30 30885 35958 710 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30886] 30 30886 36148 1115 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30887] 30 30887 35960 918 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30888] 30 30888 35960 603 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30889] 30 30889 35957 917 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30890] 30 30890 35969 852 315392 287 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30891] 30 30891 35952 913 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30892] 30 30892 40342 5258 348160 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30893] 30 30893 35969 909 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30894] 30 30894 35961 888 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30895] 30 30895 35946 800 315392 334 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30896] 30 30896 35960 861 315392 338 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30897] 30 30897 35953 667 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30898] 30 30898 35957 915 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30899] 30 30899 35956 894 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30900] 30 30900 35952 640 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30901] 30 30901 35946 891 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30902] 30 30902 35958 1037 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30903] 30 30903 35950 863 315392 297 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30904] 30 30904 35957 900 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30905] 30 30905 35952 1030 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30906] 30 30906 35960 873 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30907] 30 30907 35960 985 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30908] 30 30908 35972 911 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30927] 30 30927 35956 887 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30928] 30 30928 35959 908 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30929] 30 30929 35960 597 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30931] 30 30931 35958 914 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30932] 30 30932 35953 646 315392 272 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30933] 30 30933 35960 921 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30934] 30 30934 35965 911 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30936] 30 30936 35967 876 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30937] 30 30937 35958 847 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30938] 30 30938 35951 911 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30939] 30 30939 35960 798 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30940] 30 30940 35958 914 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30942] 30 30942 35952 904 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30944] 30 30944 35981 924 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30945] 30 30945 35956 911 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30946] 30 30946 35962 926 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30947] 30 30947 35946 907 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30949] 30 30949 35972 910 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30950] 30 30950 35959 841 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30951] 30 30951 35959 822 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30952] 30 30952 35947 907 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30953] 30 30953 35968 852 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30955] 30 30955 35955 902 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30956] 30 30956 35972 704 315392 292 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30957] 30 30957 35978 949 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30958] 30 30958 35956 992 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30959] 30 30959 35948 843 315392 324 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30960] 30 30960 35957 890 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30961] 30 30961 35973 918 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30962] 30 30962 35954 906 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30963] 30 30963 35960 880 315392 287 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30965] 30 30965 35970 716 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30966] 30 30966 35952 1034 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30967] 30 30967 35956 602 315392 291 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30969] 30 30969 35960 814 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30970] 30 30970 35961 862 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30971] 30 30971 35968 907 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30973] 30 30973 35946 913 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30974] 30 30974 35960 908 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30975] 30 30975 35953 896 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30976] 30 30976 35952 861 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30977] 30 30977 35967 909 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30978] 30 30978 35967 917 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30982] 30 30982 35956 1048 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30983] 30 30983 35961 763 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30984] 30 30984 35948 924 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30985] 30 30985 35957 913 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30986] 30 30986 35977 902 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30987] 30 30987 35965 725 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30988] 30 30988 35951 848 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30989] 30 30989 35967 788 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 30990] 30 30990 35972 914 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 20862] 30 20862 35957 860 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 20864] 30 20864 35948 813 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 20930] 30 20930 35956 645 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21477] 30 21477 35951 813 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21481] 30 21481 35958 821 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21482] 30 21482 35954 911 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21485] 30 21485 35957 771 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21486] 30 21486 35956 908 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21487] 30 21487 35953 890 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21488] 30 21488 35956 914 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21516] 30 21516 35970 770 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21517] 30 21517 35953 900 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21518] 30 21518 35955 901 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21519] 30 21519 35955 632 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21520] 30 21520 35955 716 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21521] 30 21521 35956 821 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21523] 30 21523 35954 883 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21524] 30 21524 35953 889 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21525] 30 21525 35959 635 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21526] 30 21526 35956 903 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21527] 30 21527 35958 911 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21528] 30 21528 35959 914 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21529] 30 21529 35953 911 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21530] 30 21530 35954 906 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21531] 30 21531 35953 608 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21532] 30 21532 35980 881 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21533] 30 21533 35957 904 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21534] 30 21534 35956 905 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21537] 30 21537 35961 939 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21538] 30 21538 35953 820 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21539] 30 21539 35977 618 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21543] 30 21543 35959 829 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21544] 30 21544 35957 903 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21545] 30 21545 35956 915 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21547] 30 21547 36015 708 315392 290 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21548] 30 21548 35954 639 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21549] 30 21549 35952 920 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21550] 30 21550 35953 895 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21551] 30 21551 35958 1038 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21552] 30 21552 35955 718 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21554] 30 21554 35955 917 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21555] 30 21555 35971 904 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21556] 30 21556 35956 853 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21557] 30 21557 35956 818 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21558] 30 21558 35960 912 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21559] 30 21559 35954 862 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21560] 30 21560 35952 911 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21561] 30 21561 35954 899 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21562] 30 21562 35958 811 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21563] 30 21563 35955 854 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21564] 30 21564 35971 844 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21565] 30 21565 35958 908 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21566] 30 21566 35954 919 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21567] 30 21567 35965 909 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21568] 30 21568 35961 813 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21569] 30 21569 35955 915 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21571] 30 21571 35956 857 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21572] 30 21572 35958 908 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21573] 30 21573 35958 803 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21574] 30 21574 35958 637 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21575] 30 21575 35954 910 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21579] 30 21579 35952 750 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21580] 30 21580 35956 872 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21581] 30 21581 35952 930 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21582] 30 21582 35960 621 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21584] 30 21584 35952 898 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21585] 30 21585 35957 659 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21586] 30 21586 35959 931 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21587] 30 21587 35956 937 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21588] 30 21588 35955 896 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21589] 30 21589 35982 895 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21590] 30 21590 35954 729 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21591] 30 21591 35959 928 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21592] 30 21592 35957 735 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21593] 30 21593 35962 664 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21594] 30 21594 35954 932 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21595] 30 21595 35987 948 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21596] 30 21596 35976 766 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21597] 30 21597 35959 861 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21599] 30 21599 35960 843 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21600] 30 21600 35961 911 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21601] 30 21601 35954 933 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21602] 30 21602 35954 649 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21603] 30 21603 35969 819 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21604] 30 21604 35984 936 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21605] 30 21605 35955 820 315392 298 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21606] 30 21606 35957 645 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21607] 30 21607 35958 924 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21608] 30 21608 35960 844 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21609] 30 21609 35958 922 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21610] 30 21610 36054 947 315392 299 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21613] 30 21613 35948 907 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21614] 30 21614 35956 936 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21615] 30 21615 35958 931 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21616] 30 21616 35948 878 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21617] 30 21617 35958 1051 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21618] 30 21618 35954 934 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21619] 30 21619 35956 661 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21620] 30 21620 35954 938 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21621] 30 21621 35954 917 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21622] 30 21622 35955 934 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21623] 30 21623 35976 937 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21624] 30 21624 35955 742 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21625] 30 21625 35954 888 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21626] 30 21626 35955 941 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21627] 30 21627 35970 700 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21628] 30 21628 35948 639 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21629] 30 21629 35953 855 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21630] 30 21630 35959 924 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21631] 30 21631 35954 934 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21632] 30 21632 35954 892 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21633] 30 21633 35954 936 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21634] 30 21634 35954 798 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21635] 30 21635 35957 741 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21636] 30 21636 35967 936 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21637] 30 21637 35961 883 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21638] 30 21638 35958 785 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21639] 30 21639 35956 928 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21640] 30 21640 35957 611 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21641] 30 21641 35962 941 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21642] 30 21642 35955 644 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21643] 30 21643 35955 937 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21644] 30 21644 35976 921 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21651] 30 21651 35956 756 315392 313 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21652] 30 21652 35982 939 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21653] 30 21653 35956 924 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21654] 30 21654 35958 914 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21655] 30 21655 35962 920 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21657] 30 21657 35981 939 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21658] 30 21658 35960 666 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21660] 30 21660 35962 862 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21661] 30 21661 35970 936 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21662] 30 21662 35954 918 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21664] 30 21664 35972 899 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21665] 30 21665 35954 1048 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21666] 30 21666 35956 918 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21667] 30 21667 35960 926 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21668] 30 21668 35956 671 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21669] 30 21669 35962 633 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21670] 30 21670 35953 934 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21671] 30 21671 35959 931 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21673] 30 21673 35956 876 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21674] 30 21674 35948 895 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21675] 30 21675 35948 1052 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21676] 30 21676 35954 922 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21678] 30 21678 35957 927 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21679] 30 21679 35956 879 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21680] 30 21680 35954 907 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21681] 30 21681 35958 664 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21682] 30 21682 35955 658 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21685] 30 21685 35955 786 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21686] 30 21686 35970 824 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21687] 30 21687 35950 930 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21688] 30 21688 35958 934 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21689] 30 21689 35948 921 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21691] 30 21691 35956 935 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21692] 30 21692 35946 907 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21693] 30 21693 35954 1057 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21694] 30 21694 35950 927 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21695] 30 21695 35950 657 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21696] 30 21696 35946 649 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21697] 30 21697 35952 664 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21698] 30 21698 35954 895 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21699] 30 21699 35957 668 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21701] 30 21701 35958 656 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21702] 30 21702 35953 885 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21703] 30 21703 35959 934 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21704] 30 21704 35967 935 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21705] 30 21705 35970 941 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21706] 30 21706 35962 932 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21708] 30 21708 35968 937 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21709] 30 21709 35961 919 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21710] 30 21710 35955 897 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21711] 30 21711 35954 629 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21714] 30 21714 35960 666 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21715] 30 21715 35958 925 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21716] 30 21716 35960 629 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21717] 30 21717 35957 831 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21721] 30 21721 35960 854 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21722] 30 21722 35955 755 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21723] 30 21723 35954 660 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21724] 30 21724 35955 1057 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21726] 30 21726 35958 822 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21727] 30 21727 35958 859 315392 269 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21728] 30 21728 35955 867 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21729] 30 21729 35958 932 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21730] 30 21730 35957 842 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21731] 30 21731 35954 935 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21732] 30 21732 35957 928 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21733] 30 21733 36344 1306 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21734] 30 21734 35957 1059 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21735] 30 21735 35959 748 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21736] 30 21736 35958 646 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21737] 30 21737 35966 851 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21738] 30 21738 35960 905 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21739] 30 21739 35954 655 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21740] 30 21740 35963 821 315392 341 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21741] 30 21741 35972 935 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21742] 30 21742 35972 873 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21743] 30 21743 35948 876 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21744] 30 21744 35961 862 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21745] 30 21745 38461 3419 335872 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21746] 30 21746 35957 926 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21747] 30 21747 35954 727 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21748] 30 21748 35956 629 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21749] 30 21749 35959 917 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 21750] 30 21750 35971 871 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17972] 30 17972 35954 912 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17973] 30 17973 35954 703 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17974] 30 17974 35972 720 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17977] 30 17977 35983 920 315392 267 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17979] 30 17979 35960 896 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17980] 30 17980 35960 866 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17984] 30 17984 35954 903 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17986] 30 17986 35954 829 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17987] 30 17987 35954 870 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17988] 30 17988 35958 908 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17989] 30 17989 35948 859 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17990] 30 17990 35975 900 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17991] 30 17991 35971 922 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17992] 30 17992 35973 918 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17994] 30 17994 35970 842 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17995] 30 17995 35955 913 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17996] 30 17996 35968 897 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17997] 30 17997 35958 919 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17998] 30 17998 35987 648 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 17999] 30 17999 35954 902 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18000] 30 18000 35970 636 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18001] 30 18001 35954 721 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18002] 30 18002 35954 914 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18003] 30 18003 35958 918 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18004] 30 18004 35971 619 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18005] 30 18005 35960 902 315392 295 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18006] 30 18006 35954 715 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18007] 30 18007 35952 621 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18009] 30 18009 35957 1030 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18014] 30 18014 35987 952 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18015] 30 18015 35974 932 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18016] 30 18016 35954 921 315392 273 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18017] 30 18017 35961 897 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18018] 30 18018 35954 933 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18019] 30 18019 35961 865 315392 271 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18020] 30 18020 35958 751 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18021] 30 18021 35955 916 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18022] 30 18022 35952 868 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18023] 30 18023 35948 915 315392 270 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18024] 30 18024 35958 745 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18025] 30 18025 35948 626 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18026] 30 18026 35960 642 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18027] 30 18027 35960 938 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18028] 30 18028 35967 936 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18030] 30 18030 35955 761 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18031] 30 18031 35960 647 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18032] 30 18032 35957 932 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18033] 30 18033 35951 676 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18034] 30 18034 35952 1019 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18035] 30 18035 35960 935 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18036] 30 18036 35948 730 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18037] 30 18037 35973 922 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18038] 30 18038 35960 1061 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18040] 30 18040 35956 759 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18041] 30 18041 35960 933 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18042] 30 18042 35956 885 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18043] 30 18043 35952 933 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18044] 30 18044 35967 890 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18046] 30 18046 35961 923 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18047] 30 18047 35948 922 315392 264 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18048] 30 18048 35954 934 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18062] 30 18062 35954 1048 315392 268 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18063] 30 18063 35975 647 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18064] 30 18064 35981 883 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18065] 30 18065 35959 916 315392 256 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18066] 30 18066 35960 938 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18067] 30 18067 35950 996 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18068] 30 18068 35960 659 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18069] 30 18069 35960 933 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18071] 30 18071 35956 638 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18072] 30 18072 35954 939 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18073] 30 18073 35960 941 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18074] 30 18074 35974 660 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18075] 30 18075 35954 935 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18076] 30 18076 35962 925 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18077] 30 18077 35958 1062 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18078] 30 18078 35973 805 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18079] 30 18079 35961 859 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18080] 30 18080 35954 937 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18081] 30 18081 35948 922 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18082] 30 18082 35952 663 315392 258 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18083] 30 18083 35960 895 315392 263 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18084] 30 18084 35950 824 315392 261 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18085] 30 18085 35954 934 315392 259 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18087] 30 18087 35970 929 315392 265 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18088] 30 18088 35954 937 315392 260 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18089] 30 18089 35960 946 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18090] 30 18090 35956 899 315392 272 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18091] 30 18091 35985 774 315392 296 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18092] 30 18092 35952 938 315392 257 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 18093] 30 18093 35946 769 315392 266 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 451] 30 451 35960 892 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 455] 30 455 35958 895 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 456] 30 456 35958 833 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 475] 30 475 35955 844 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 476] 30 476 35954 771 315392 334 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 478] 30 478 35960 907 315392 272 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 479] 30 479 35955 620 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 503] 30 503 35952 881 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 504] 30 504 35953 885 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 506] 30 506 35954 612 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 507] 30 507 35956 912 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 508] 30 508 35971 644 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 509] 30 509 35957 913 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 510] 30 510 35953 908 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 519] 30 519 35971 624 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 520] 30 520 35973 638 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 521] 30 521 35952 642 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 522] 30 522 35986 918 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 523] 30 523 35951 909 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 524] 30 524 35955 791 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 525] 30 525 35954 919 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 526] 30 526 35954 918 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 527] 30 527 35955 619 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 528] 30 528 35957 632 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 529] 30 529 35967 599 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 530] 30 530 35943 853 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 531] 30 531 35975 917 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 534] 30 534 35956 865 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 541] 30 541 36011 918 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 542] 30 542 35956 893 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 543] 30 543 35955 918 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 544] 30 544 35954 720 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 545] 30 545 35950 899 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 546] 30 546 35955 911 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 547] 30 547 35960 764 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 548] 30 548 35973 869 315392 291 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 549] 30 549 35956 912 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 550] 30 550 35954 912 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 551] 30 551 35973 913 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 552] 30 552 35971 891 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 553] 30 553 35960 921 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 554] 30 554 35954 638 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 555] 30 555 35954 894 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 556] 30 556 35974 728 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 557] 30 557 35952 633 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 558] 30 558 35954 857 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 559] 30 559 35958 905 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 560] 30 560 35985 891 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 561] 30 561 35954 917 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 562] 30 562 35981 744 315392 272 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 564] 30 564 35984 931 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 565] 30 565 35952 912 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 566] 30 566 35952 924 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 567] 30 567 35954 916 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 568] 30 568 35961 911 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 569] 30 569 35956 841 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 570] 30 570 35973 655 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 571] 30 571 35961 643 315392 274 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 572] 30 572 35960 923 315392 275 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1307] 30 1307 35960 881 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1358] 30 1358 35960 822 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1359] 30 1359 35948 629 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1364] 30 1364 38537 3413 335872 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1365] 30 1365 35950 976 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1367] 30 1367 35977 898 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1368] 30 1368 35953 824 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1372] 30 1372 35952 894 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1373] 30 1373 35953 882 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1374] 30 1374 35954 901 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1375] 30 1375 35960 857 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1376] 30 1376 35960 679 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1377] 30 1377 35984 614 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1378] 30 1378 35954 899 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1379] 30 1379 35956 899 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1386] 30 1386 35960 912 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1387] 30 1387 35960 707 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1388] 30 1388 35950 853 315392 288 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1390] 30 1390 35956 913 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1391] 30 1391 35960 904 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1392] 30 1392 35960 920 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1393] 30 1393 35960 639 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1394] 30 1394 35956 908 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1395] 30 1395 35960 708 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1396] 30 1396 35952 616 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1397] 30 1397 35962 838 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1399] 30 1399 35955 913 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1400] 30 1400 38402 3353 331776 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1401] 30 1401 35954 901 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1406] 30 1406 35954 909 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1407] 30 1407 35962 922 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1408] 30 1408 35956 796 315392 368 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1409] 30 1409 35954 853 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1410] 30 1410 35960 918 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1411] 30 1411 35956 725 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1412] 30 1412 35973 863 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1413] 30 1413 35958 819 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1414] 30 1414 35954 911 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1415] 30 1415 35950 907 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1416] 30 1416 35950 875 315392 313 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1417] 30 1417 35960 912 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1419] 30 1419 35960 847 315392 286 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1420] 30 1420 35987 924 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1421] 30 1421 35974 901 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1422] 30 1422 35960 912 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1423] 30 1423 35958 800 315392 297 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1424] 30 1424 35961 911 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1425] 30 1425 35950 915 315392 276 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1426] 30 1426 35956 829 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1427] 30 1427 35960 912 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1428] 30 1428 35960 899 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1429] 30 1429 35961 893 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1430] 30 1430 35960 916 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1435] 30 1435 35973 728 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1436] 30 1436 35952 626 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1437] 30 1437 35960 922 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1439] 30 1439 35948 863 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1440] 30 1440 35958 916 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1441] 30 1441 37518 2208 327680 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1442] 30 1442 35960 915 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1443] 30 1443 35972 906 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1449] 30 1449 35973 922 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1450] 30 1450 35960 880 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1451] 30 1451 35960 621 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1452] 30 1452 35960 843 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1453] 30 1453 35960 823 315392 294 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1454] 30 1454 35952 816 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1455] 30 1455 35962 878 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1456] 30 1456 35956 634 315392 283 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1457] 30 1457 35954 628 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1458] 30 1458 35960 824 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1459] 30 1459 35958 905 315392 287 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1460] 30 1460 35982 922 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1461] 30 1461 35960 892 315392 285 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1464] 30 1464 35960 724 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1466] 30 1466 35960 641 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1469] 30 1469 35958 916 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1470] 30 1470 35955 902 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1471] 30 1471 36956 1876 323584 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1472] 30 1472 35960 897 315392 289 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1473] 30 1473 35960 624 315392 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1474] 30 1474 35954 630 315392 288 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1476] 30 1476 35958 906 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1482] 30 1482 35950 859 315392 278 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1483] 30 1483 35960 859 315392 280 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1484] 30 1484 36503 1467 319488 279 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1485] 30 1485 35986 928 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1486] 30 1486 35954 908 315392 282 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1487] 30 1487 35986 748 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1488] 30 1488 35957 872 315392 281 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1489] 30 1489 35958 922 315392 277 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 1490] 30 1490 35952 907 315392 284 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 11617] 0 11617 5707 418 90112 98 0 vsftpd
Jan 26 15:19:01 openqa kernel: [ 16419] 30 16419 35961 922 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16423] 30 16423 35972 923 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16424] 30 16424 35956 858 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16426] 30 16426 35957 913 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16427] 30 16427 35960 620 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16428] 30 16428 35962 908 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16429] 30 16429 35955 928 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16431] 30 16431 35968 931 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16432] 30 16432 35961 881 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16433] 30 16433 35961 918 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16434] 30 16434 35950 832 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16435] 30 16435 35961 925 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16436] 30 16436 35952 712 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16437] 30 16437 35971 462 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16438] 30 16438 35960 916 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16473] 30 16473 35951 919 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16474] 30 16474 35967 890 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16475] 30 16475 35979 931 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16476] 30 16476 35952 903 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16478] 30 16478 35944 876 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16479] 30 16479 35961 723 315392 255 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16480] 30 16480 35955 1057 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16483] 30 16483 35960 742 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16484] 30 16484 35981 946 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16485] 30 16485 35960 935 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16486] 30 16486 35948 656 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16487] 30 16487 35954 736 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16488] 30 16488 35960 932 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16489] 30 16489 35972 841 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16490] 30 16490 35960 907 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16491] 30 16491 35946 886 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16522] 30 16522 35961 933 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16523] 30 16523 35952 919 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16524] 30 16524 35976 942 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16525] 30 16525 35950 870 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16526] 30 16526 35954 848 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16527] 30 16527 35952 938 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16528] 30 16528 35959 962 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16529] 30 16529 35959 937 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16531] 30 16531 35950 903 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16532] 30 16532 35961 932 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16533] 30 16533 35960 832 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16534] 30 16534 35961 926 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16535] 30 16535 35969 653 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16537] 30 16537 35948 879 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16538] 30 16538 35956 932 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16539] 30 16539 35961 826 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16540] 30 16540 35952 651 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16541] 30 16541 35954 935 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16542] 30 16542 35959 852 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16543] 30 16543 35954 856 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16544] 30 16544 35959 636 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16546] 30 16546 35948 938 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16547] 30 16547 35985 929 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16550] 30 16550 35969 938 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16551] 30 16551 35960 890 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16552] 30 16552 35953 926 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16553] 30 16553 35962 934 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16554] 30 16554 35962 661 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16555] 30 16555 35960 663 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16556] 30 16556 35967 937 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16557] 30 16557 35976 930 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16580] 30 16580 35959 918 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16581] 30 16581 35962 812 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16582] 30 16582 35974 1062 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16583] 30 16583 35972 938 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16584] 30 16584 35970 651 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16585] 30 16585 35946 639 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16587] 30 16587 35975 940 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16588] 30 16588 35961 870 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16589] 30 16589 35948 925 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16591] 30 16591 35950 893 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16592] 30 16592 35951 759 315392 262 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16593] 30 16593 35955 660 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16594] 30 16594 35960 752 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16595] 30 16595 35955 638 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16596] 30 16596 35971 752 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16597] 30 16597 36495 1481 319488 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16598] 30 16598 35954 816 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16599] 30 16599 35948 933 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16600] 30 16600 35962 666 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16601] 30 16601 35960 730 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16602] 30 16602 35948 932 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16603] 30 16603 35948 664 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16604] 30 16604 35948 758 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16605] 30 16605 35970 900 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16606] 30 16606 35952 932 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16607] 30 16607 35978 875 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16608] 30 16608 35968 878 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16609] 30 16609 35948 910 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16610] 30 16610 35960 933 315392 253 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16611] 30 16611 35948 931 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 16612] 30 16612 35972 937 315392 254 0 httpd-prefork
Jan 26 15:19:01 openqa kernel: [ 12194] 26 12194 856681 563705 6602752 202 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16390] 1001 16390 76927 37552 655360 0 0 openqa-livehand
Jan 26 15:19:01 openqa kernel: [ 16392] 1001 16392 75112 35744 634880 0 0 openqa-schedule
Jan 26 15:19:01 openqa kernel: [ 16393] 1001 16393 78719 39283 675840 0 0 openqa-websocke
Jan 26 15:19:01 openqa kernel: [ 16394] 1001 16394 88013 40045 724992 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 16441] 1001 16441 84954 40005 712704 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 16444] 26 16444 838179 216394 5898240 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16464] 26 16464 836147 3723 368640 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16515] 26 16515 836441 42706 2310144 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16549] 26 16549 836147 3738 368640 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16574] 26 16574 856565 24086 958464 209 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16642] 26 16642 836430 135067 3260416 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 25523] 0 25523 27439 1034 258048 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 25535] 17305 25535 22226 868 204800 0 0 systemd
Jan 26 15:19:01 openqa kernel: [ 25536] 17305 25536 34117 1002 290816 146 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 25544] 17305 25544 27621 880 237568 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 25546] 17305 25546 13024 2127 147456 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 20118] 0 20118 27439 1046 253952 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 20132] 17305 20132 27439 583 245760 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 20154] 17305 20154 13056 2165 151552 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 22426] 51 22426 21960 391 172032 0 0 pickup
Jan 26 15:19:01 openqa kernel: [ 24011] 1001 24011 91550 44875 753664 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 24012] 26 24012 837560 104878 4796416 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 25537] 1001 25537 97313 51076 798720 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 25542] 26 25542 856099 307830 6328320 207 0 postgres
Jan 26 15:19:01 openqa kernel: [ 27075] 26 27075 836733 4321 425984 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 27484] 1001 27484 443312 389795 3530752 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 27487] 26 27487 836884 4907 421888 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 27500] 26 27500 859357 285356 5971968 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28087] 1001 28087 96164 49719 790528 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 28091] 26 28091 855449 292992 6307840 207 0 postgres
Jan 26 15:19:01 openqa kernel: [ 31334] 1001 31334 101507 55233 831488 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 31338] 26 31338 855415 251853 6246400 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 31559] 1001 31559 96142 49852 790528 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 31561] 26 31561 838868 246263 6082560 212 0 postgres
Jan 26 15:19:01 openqa kernel: [ 1099] 0 1099 27439 1037 258048 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 1116] 17285 1116 22226 1145 204800 0 0 systemd
Jan 26 15:19:01 openqa kernel: [ 1117] 17285 1117 34117 1008 290816 146 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 1133] 17285 1133 27439 453 249856 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 1135] 17285 1135 12994 2070 147456 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 2119] 17285 2119 26150 800 249856 0 0 sudo
Jan 26 15:19:01 openqa kernel: [ 2126] 0 2126 21796 668 217088 0 0 su
Jan 26 15:19:01 openqa kernel: [ 2129] 0 2129 13013 2098 139264 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 7190] 1001 7190 101020 54582 827392 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 7202] 26 7202 841415 337840 6561792 212 0 postgres
Jan 26 15:19:01 openqa kernel: [ 7355] 26 7355 836916 7686 516096 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 7686] 1001 7686 99144 52529 815104 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 7693] 26 7693 855303 284264 6369280 207 0 postgres
Jan 26 15:19:01 openqa kernel: [ 8465] 26 8465 836916 7562 503808 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 8615] 26 8615 836733 4354 401408 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 8845] 1001 8845 99286 52882 815104 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 8847] 26 8847 838116 253521 6107136 212 0 postgres
Jan 26 15:19:01 openqa kernel: [ 8951] 26 8951 836899 7501 495616 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 9088] 26 9088 836881 7146 503808 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 9476] 1001 9476 97574 49764 802816 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 9479] 26 9479 855088 211509 6135808 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 9786] 1001 9786 96388 49563 790528 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 9793] 26 9793 838633 258981 6221824 212 0 postgres
Jan 26 15:19:01 openqa kernel: [ 10948] 1001 10948 93831 47372 770048 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 10951] 26 10951 837828 157936 5574656 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 11004] 26 11004 836941 7296 516096 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 12297] 1001 12297 94207 47963 774144 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 12298] 26 12298 838222 229677 5836800 212 0 postgres
Jan 26 15:19:01 openqa kernel: [ 12524] 1001 12524 94282 47526 774144 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 12525] 26 12525 857728 178019 5820416 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 13324] 26 13324 836733 4337 417792 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 13798] 1001 13798 94013 47729 774144 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 13802] 26 13802 837717 142999 5525504 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 14203] 1001 14203 94493 47751 778240 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 14206] 26 14206 857506 214835 6107136 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 14771] 26 14771 836322 6739 434176 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16031] 1001 16031 93320 47126 753664 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 16035] 26 16035 837742 125545 5324800 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16261] 26 16261 836733 4327 417792 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 16742] 0 16742 27439 1034 249856 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 16764] 17258 16764 22229 1195 208896 0 0 systemd
Jan 26 15:19:01 openqa kernel: [ 16767] 17258 16767 34117 1008 290816 146 0 (sd-pam)
Jan 26 15:19:01 openqa kernel: [ 16786] 17258 16786 27439 689 241664 0 0 sshd
Jan 26 15:19:01 openqa kernel: [ 16791] 17258 16791 13037 2123 151552 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 16925] 17258 16925 26148 779 237568 0 0 sudo
Jan 26 15:19:01 openqa kernel: [ 16930] 0 16930 22310 651 221184 0 0 su
Jan 26 15:19:01 openqa kernel: [ 16933] 1001 16933 13072 2109 147456 0 0 bash
Jan 26 15:19:01 openqa kernel: [ 17026] 1001 17026 3239487 2616717 21155840 0 0 psql
Jan 26 15:19:01 openqa kernel: [ 17028] 26 17028 836286 480460 5505024 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 19225] 26 19225 836733 4321 409600 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 19325] 1001 19325 93766 47182 770048 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 19331] 26 19331 837570 148400 5451776 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 19896] 490 19896 13059 557 143360 87 0 vsftpd
Jan 26 15:19:01 openqa kernel: [ 19930] 40 19930 13065 409 139264 87 0 vsftpd
Jan 26 15:19:01 openqa kernel: [ 20177] 490 20177 13059 557 143360 87 0 vsftpd
Jan 26 15:19:01 openqa kernel: [ 20197] 40 20197 13065 412 139264 87 0 vsftpd
Jan 26 15:19:01 openqa kernel: [ 20204] 26 20204 836735 4335 401408 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 20210] 1001 20210 91506 45678 741376 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 20212] 26 20212 837679 159085 5488640 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 23608] 1001 23608 93186 46879 765952 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 23611] 26 23611 853927 118186 5074944 208 0 postgres
Jan 26 15:19:01 openqa kernel: [ 25144] 26 25144 836733 4338 397312 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 25479] 1001 25479 87976 43003 712704 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 25482] 26 25482 836884 4956 425984 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 25499] 26 25499 836367 13834 1257472 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 26319] 26 26319 836733 4321 397312 216 0 postgres
Jan 26 15:19:01 openqa kernel: [ 27962] 26 27962 836322 6738 434176 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28431] 51 28431 27134 744 176128 0 0 cleanup
Jan 26 15:19:01 openqa kernel: [ 28432] 51 28432 27110 669 180224 0 0 trivial-rewrite
Jan 26 15:19:01 openqa kernel: [ 28433] 0 28433 27119 739 176128 0 0 local
Jan 26 15:19:01 openqa kernel: [ 28894] 65534 28894 27367 1653 253952 81 0 rsyncd
Jan 26 15:19:01 openqa kernel: [ 28961] 1001 28961 88013 41728 708608 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 28962] 26 28962 836387 14683 1212416 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28973] 1001 28973 85088 40117 692224 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 28974] 1001 28974 85136 40287 692224 0 0 openqa
Jan 26 15:19:01 openqa kernel: [ 28975] 26 28975 836884 4814 425984 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28976] 26 28976 836884 4891 425984 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28977] 26 28977 836327 4332 401408 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28978] 26 28978 836327 4300 438272 213 0 postgres
Jan 26 15:19:01 openqa kernel: [ 28986] 1001 28986 2267 124 57344 0 0 timeout
Jan 26 15:19:01 openqa kernel: [ 28989] 1001 28989 3117 500 65536 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 28997] 1001 28997 3150 490 65536 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 28998] 1001 28998 3986 137 73728 0 0 sed
Jan 26 15:19:01 openqa kernel: [ 28999] 1001 28999 3150 511 65536 0 0 openqa-investig
Jan 26 15:19:01 openqa kernel: [ 29015] 1001 29015 3150 279 61440 0 0 openqa-investig
Jan 26 15:19:01 openqa kernel: [ 29016] 1001 29016 1233 87 57344 0 0 cat
Jan 26 15:19:01 openqa kernel: [ 29017] 1001 29017 3986 118 77824 0 0 sed
Jan 26 15:19:01 openqa kernel: [ 29019] 1001 29019 3150 273 61440 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 29020] 1001 29020 3150 333 61440 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 29021] 1001 29021 3150 323 61440 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 29022] 1001 29022 3150 360 61440 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 29024] 1001 29024 3150 115 61440 0 0 openqa-label-kn
Jan 26 15:19:01 openqa kernel: [ 29025] 1001 29025 35713 787 180224 0 0 curl
Jan 26 15:19:01 openqa kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=psql,pid=17026,uid=1001
Jan 26 15:19:01 openqa kernel: Out of memory: Killed process 17026 (psql) total-vm:12957948kB, anon-rss:10464292kB, file-rss:2576kB, shmem-rss:0kB
Jan 26 15:19:01 openqa systemd[1]: systemd-logind.service: Watchdog timeout (limit 3min)!
Jan 26 15:19:01 openqa systemd[1]: systemd-logind.service: Killing process 720 (systemd-logind) with signal SIGABRT.
Jan 26 15:19:01 openqa systemd[1]: systemd-udevd.service: Watchdog timeout (limit 3min)!
Jan 26 15:19:01 openqa systemd[1]: systemd-udevd.service: Killing process 540 (systemd-udevd) with signal SIGABRT.
Jan 26 15:19:01 openqa kernel: oom_reaper: reaped process 17026 (psql), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Jan 26 15:19:01 openqa systemd[1]: systemd-udevd.service: Main process exited, code=dumped, status=6/ABRT
Jan 26 15:19:01 openqa systemd[1]: systemd-udevd.service: Failed with result 'watchdog'.
Jan 26 15:19:01 openqa systemd[1]: systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
Jan 26 15:19:01 openqa systemd[1]: systemd-journald.service: Failed with result 'watchdog'.
Jan 26 15:19:01 openqa systemd[1]: systemd-udevd.service: Scheduled restart job, restart counter is at 1.
Jan 26 15:19:01 openqa systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
Jan 26 15:19:01 openqa systemd[1]: Stopping Flush Journal to Persistent Storage...
Jan 26 15:19:01 openqa systemd[1]: Stopped Rule-based Manager for Device Events and Files.
Jan 26 15:19:01 openqa systemd[1]: Starting Rule-based Manager for Device Events and Files...
Jan 26 15:19:01 openqa systemd[1]: systemd-journal-flush.service: Control process exited, code=exited, status=1/FAILURE
Jan 26 15:19:01 openqa systemd[1]: systemd-journal-flush.service: Failed with result 'exit-code'.
Jan 26 15:19:01 openqa systemd[1]: Stopped Flush Journal to Persistent Storage.
Jan 26 15:19:01 openqa systemd[1]: Stopped Journal Service.
Jan 26 15:19:01 openqa systemd[1]: Starting Journal Service...
Jan 26 15:19:01 openqa systemd[1]: systemd-logind.service: Main process exited, code=killed, status=6/ABRT
Jan 26 15:19:01 openqa systemd[1]: systemd-logind.service: Failed with result 'watchdog'.
Jan 26 15:19:01 openqa systemd[1]: systemd-logind.service: Scheduled restart job, restart counter is at 1.
Jan 26 15:19:01 openqa systemd[1]: Stopped User Login Management.
Jan 26 15:19:01 openqa systemd[1]: Starting Load Kernel Module drm...
Jan 26 15:19:01 openqa systemd[1]: modprobe@drm.service: Succeeded.
Jan 26 15:19:01 openqa systemd[1]: Finished Load Kernel Module drm.
Jan 26 15:19:01 openqa systemd[1]: Starting User Login Management...
Jan 26 15:19:01 openqa systemd[1]: Started Rule-based Manager for Device Events and Files.
Jan 26 15:19:01 openqa systemd[1]: Started User Login Management.
Jan 26 15:19:02 openqa systemd[1]: Started Journal Service.
Jan 26 15:18:53 openqa systemd[1]: systemd-journald.service: Watchdog timeout (limit 3min)!
Jan 26 15:19:02 openqa telegraf[1248]: 2022-01-26T14:18:57Z E! [outputs.influxdb] when writing to [http://openqa-monitor.qa.suse.de:8086]: Post "http://openqa-monitor.qa.suse.de:8086/write?db=telegraf": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Jan 26 15:19:02 openqa telegraf[1248]: 2022-01-26T14:18:57Z W! [agent] [inputs.http] did not complete within its interval
Jan 26 15:19:02 openqa telegraf[1248]: 2022-01-26T14:18:57Z W! [agent] ["outputs.influxdb"] did not complete within its flush interval
Jan 26 15:19:02 openqa telegraf[1248]: 2022-01-26T14:18:57Z E! [agent] Error writing to outputs.influxdb: could not write any address
Jan 26 15:19:02 openqa telegraf[1248]: 2022-01-26T14:18:57Z E! [inputs.http] Error in plugin: [url=https://openqa.suse.de/admin/influxdb/jobs]: Get "https://openqa.suse.de/admin/influxdb/jobs": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Jan 26 15:18:53 openqa chronyd[1243]: Forward time jump detected!
Jan 26 15:18:53 openqa systemd[1]: systemd-journald.service: Killing process 29178 (systemd-journal) with signal SIGABRT.
Jan 26 15:18:53 openqa chronyd[1243]: Can't synchronise: no selectable sources
Jan 26 15:18:57 openqa journalctl[29172]: Failed to connect to /run/systemd/journal/io.systemd.journal: Connection refused
Jan 26 15:18:53 openqa sshd[29059]: rexec line 119: Deprecated option UsePrivilegeSeparation
Jan 26 15:18:59 openqa systemd-udevd[29173]: Network interface NamePolicy= disabled by default.
Jan 26 15:18:53 openqa sshd[29058]: rexec line 119: Deprecated option UsePrivilegeSeparation
Jan 26 15:19:00 openqa systemd-logind[29211]: Watching system buttons on /dev/input/event3 (Power Button)
Jan 26 15:18:53 openqa sshd[29058]: Connection from 10.100.12.155 port 52736 on 10.160.0.207 port 22 rdomain ""
Jan 26 15:19:00 openqa systemd-logind[29211]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard)
Jan 26 15:18:53 openqa sshd[29059]: Connection from 10.160.0.44 port 33300 on 10.160.0.207 port 22 rdomain ""
Jan 26 15:19:00 openqa systemd-logind[29211]: New seat seat0.
Jan 26 15:18:53 openqa sshd[29059]: error: kex_exchange_identification: Connection closed by remote host
Jan 26 15:19:00 openqa systemd-logind[29211]: New session 10644 of user jpupava.
Jan 26 15:18:53 openqa sshd[29059]: Connection closed by 10.160.0.44 port 33300
Jan 26 15:19:00 openqa systemd-logind[29211]: New session 10630 of user cfconrad.
Jan 26 15:18:54 openqa sshd[29058]: Accepted key RSA SHA256:RExKE8zGUlg0EaeAdtz0wy219J2IZ+xohbnF+u5Bwgc found at /home/jpupava/.ssh/authorized_keys:1
Jan 26 15:19:00 openqa systemd-logind[29211]: New session 10465 of user jlausuch.
Jan 26 15:18:54 openqa sshd[29058]: Postponed publickey for jpupava from 10.100.12.155 port 52736 ssh2 [preauth]
Jan 26 15:19:00 openqa systemd-logind[29211]: New session 10361 of user jlausuch.
Jan 26 15:18:54 openqa sshd[29058]: Accepted key RSA SHA256:RExKE8zGUlg0EaeAdtz0wy219J2IZ+xohbnF+u5Bwgc found at /home/jpupava/.ssh/authorized_keys:1
Jan 26 15:19:02 openqa systemd[1]: Starting Flush Journal to Persistent Storage...
Jan 26 15:18:54 openqa sshd[29058]: Accepted publickey for jpupava from 10.100.12.155 port 52736 ssh2: RSA SHA256:RExKE8zGUlg0EaeAdtz0wy219J2IZ+xohbnF+u5Bwgc
Jan 26 15:19:02 openqa systemd[1]: Finished Flush Journal to Persistent Storage.
Jan 26 15:18:55 openqa nrpe[29139]: Error: (!log_opts) Could not complete SSL handshake with 10.160.0.45: 5
Jan 26 15:18:56 openqa nrpe[29145]: Error: (!log_opts) Could not complete SSL handshake with 10.160.0.44: 5
Jan 26 15:18:56 openqa nrpe[29143]: Error: (!log_opts) Could not complete SSL handshake with 10.160.0.44: 5
Jan 26 15:18:58 openqa sshd[29058]: pam_systemd(sshd:session): Failed to create session: Message recipient disconnected from message bus without replying
Jan 26 15:18:58 openqa cron[29060]: pam_systemd(crond:session): Failed to create session: Message recipient disconnected from message bus without replying
Jan 26 15:18:58 openqa sshd[29058]: pam_unix(sshd:session): session opened for user jpupava by (uid=0)
Jan 26 15:18:58 openqa cron[29060]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:18:58 openqa sshd[29058]: User child is on pid 29201
Jan 26 15:18:58 openqa sshd[29201]: Starting session: shell on pts/5 for jpupava from 10.100.12.155 port 52736 id 0
Jan 26 15:19:03 openqa systemd[1]: Started Session 10663 of user geekotest.
Jan 26 15:19:03 openqa cron[29441]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:19:08 openqa telegraf[1248]: 2022-01-26T14:19:08Z W! [agent] [inputs.procstat] did not complete within its interval
Jan 26 15:19:09 openqa CRON[29060]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:19:09 openqa systemd[1]: Started Session 10666 of user geekotest.
Jan 26 15:19:09 openqa systemd[1]: Started Session 10664 of user geekotest.
Jan 26 15:19:09 openqa systemd[1]: Started Session 10665 of user geekotest.
Jan 26 15:19:10 openqa cron[29710]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:19:10 openqa cron[29712]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:19:10 openqa cron[29711]: pam_unix(crond:session): session opened for user geekotest by (uid=0)
Jan 26 15:19:10 openqa CRON[29710]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:19:10 openqa systemd[1]: session-10666.scope: Succeeded.
Jan 26 15:19:12 openqa CRON[29441]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:19:12 openqa systemd[1]: session-10663.scope: Succeeded.
Jan 26 15:19:17 openqa CRON[29712]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:19:17 openqa systemd[1]: session-10664.scope: Succeeded.
Jan 26 15:19:17 openqa postfix/pickup[29079]: 4B0ED343F2: uid=1001 from=<geekotest>
Jan 26 15:19:17 openqa postfix/cleanup[30192]: 4B0ED343F2: message-id=<20220126141917.4B0ED343F2@localhost>
Jan 26 15:19:17 openqa postfix/qmgr[1688]: 4B0ED343F2: from=<geekotest@localhost>, size=2073, nrcpt=1 (queue active)
Jan 26 15:19:17 openqa postfix/local[30203]: 4B0ED343F2: to=<geekotest@localhost>, orig_to=<geekotest>, relay=local, delay=6.5, delays=6.4/0.03/0/0.02, dsn=2.0.0, status=sent (delivered to mailbox)
Jan 26 15:19:17 openqa postfix/qmgr[1688]: 4B0ED343F2: removed
Jan 26 15:19:17 openqa CRON[29711]: pam_unix(crond:session): session closed for user geekotest
Jan 26 15:19:17 openqa systemd[1]: session-10665.scope: Succeeded.
Jan 26 15:19:17 openqa postfix/pickup[29079]: D982D343F2: uid=1001 from=<geekotest>
Jan 26 15:19:17 openqa postfix/cleanup[30192]: D982D343F2: message-id=<20220126141917.D982D343F2@localhost>
Jan 26 15:19:17 openqa postfix/qmgr[1688]: D982D343F2: from=<geekotest@localhost>, size=1297, nrcpt=1 (queue active)
Jan 26 15:19:17 openqa postfix/local[30203]: D982D343F2: to=<geekotest@localhost>, orig_to=<geekotest>, relay=local, delay=4.7, delays=4.7/0/0/0, dsn=2.0.0, status=sent (delivered to mailbox)
Jan 26 15:19:17 openqa postfix/qmgr[1688]: D982D343F2: removed
Jan 26 15:19:24 openqa sshd[29201]: Received disconnect from 10.100.12.155 port 52736:11: disconnected by user
Jan 26 15:19:24 openqa sshd[29201]: Disconnected from user jpupava 10.100.12.155 port 52736
Jan 26 15:19:24 openqa sshd[29058]: pam_unix(sshd:session): session closed for user jpupava
For simplicity I removed all output from openqa itself but I attach the full log file in case it gives us a hint why postgresql went crazy in memory usage.
Updated by livdywan over 2 years ago
- Status changed from New to In Progress
- Assignee set to livdywan
- This could be due to postgres vaccuming
- Maybe there was a high amount of jobs being scheduled at once
Updated by openqa_review over 2 years ago
- Due date set to 2022-02-16
Setting due date based on mean cycle time of SUSE QE Tools
Updated by livdywan over 2 years ago
- Due date deleted (
2022-02-16) - Status changed from In Progress to Workable
- Assignee deleted (
livdywan)
Resetting ticket, since I wasn't actually able to just do this quickly on the side. MAye someone who's looking at related things, or just has a minute to spare can jump in.
Updated by livdywan over 2 years ago
- Subject changed from [Alerting] CPU Load alert to [Alerting] CPU Load alert size:S
- Description updated (diff)
Updated by jbaier_cz over 2 years ago
There is a correlation between the CPU load and "Disk I/O time for /dev/vda", so the high load could be actually caused by waiting for disk writes.
Updated by okurz over 2 years ago
Yes, that's very likely. A load above something like 20 I think I have only ever seen if there are processes I/O blocked.
Updated by livdywan about 2 years ago
It is my impression that multiple people already took a look, but we still haven't the foggiest what this issue is. And I'm not sure we even see this anymore... I suggest we take a look together in the mob session on Thursday 24 and get debug it together (or if applicable decide we can't anymore).
Updated by livdywan about 2 years ago
- Status changed from Workable to Rejected
- Assignee set to livdywan
Possible improvements to our alert handling:
- Extend log retention time
- Job-group specific alerts
- We could cap the jobs scheduled e.g. per isos post, or jobs per group
- Fail if max=1000 jobs are scheduled
- Clarify out to get from "too many jobs" alert to a meaningful conclusion
- Increase the threshold
Updated by okurz about 2 years ago
- Related to action #107701: [osd] Job detail page fails to load added
Actions