action #95254


[teregen] Out of memory exception during template generation

Added by jbaier_cz over 2 years ago. Updated over 2 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


Problem description

A problem during generating S:M:20293:244297 occurred, the script end up with Out of memory exception; initial observation shows, there is a lot of source packages inside that incident and all the data from OBS and SMELT cannot probably fit inside the tight memory limit on qam2; I was able to generate the template locally (it took around 1.2GB for processing).

Possible solution

Simple memory optimization would be nice. The current code stores all data (even the useless one) in memory, it would be probably nice to only store relevant data. Another solution might be to store some of the info in a temporal file when needed.

Actions #1

Updated by okurz over 2 years ago

I don't know the current memory assignment for but if it's rather low I suggest to simply ask for more memory. Jan, can you do that?

Actions #2

Updated by okurz over 2 years ago

  • Priority changed from High to Low

We could not estimate it in a feasible way without relying on jbaier_cz and due to his absence this will take a bit longer. As no other user reported the problem AFAIK we should not regard this as "High" prio.

Actions #3

Updated by jbaier_cz over 2 years ago

Yes, apparently this is a non-repeating issue (yet?). If I recall correctly, qam2 has 8GB of RAM, we can probably ask for more via an infra ticket.

Actions #4

Updated by jbaier_cz over 2 years ago

  • Status changed from New to Resolved

As discussed, more memory was requested in RT-ADM#194611. That should do the trick, at least for now.

qam2:~ #  free -h
              total        used        free      shared  buff/cache   available
Mem:           15Gi       5,2Gi       8,0Gi       938Mi       2,5Gi       9,3Gi
Swap:            0B          0B          0B
Actions #5

Updated by jbaier_cz over 2 years ago

  • Assignee set to jbaier_cz

Also available in: Atom PDF