tickets #123133


metrics.o.o: /var/log/messages flooded with GB of influxd messages

Added by cboltz over 1 year ago. Updated about 1 year ago.

Servers hosted in NBG
Target version:
Start date:
Due date:
% Done:


Estimated time:


/var/log/messages on metrics.o.o gets flooded with gigabytes of influxd messages per day, with the result that the disk fills up within some hours.

I truncated /var/log/messages some minutes ago, but the disk will be full again in some hours - and I don't want to fight this windmill 3 times per day ;-)

Please fix the influxd config so that it does less logging.

Related issues 1 (1 open0 closed)

Related to openSUSE admin - tickets #123828: metrics.o.o: no OSRT:Review graphsNewwitekbedyk2023-01-31

Actions #1

Updated by cboltz over 1 year ago

  • Private changed from Yes to No
Actions #2

Updated by witekbedyk about 1 year ago

  • Assignee changed from witekbedyk to cboltz

The format of access logs has changed on 2022-12-23.
I have updated the regex to reflect that:

Unfortunately access logs from the period 2022-12-23 to 2023-01-13 have been rotated already.

Is it possible to recover these logs from the backup to rerun the aggregation script on them?

Actions #3

Updated by cboltz about 1 year ago

  • Assignee changed from cboltz to crameleon

I guess you mean the access_log from pontifex.i.o.o?

The only possibly helpful information I can give you is that there's an OLD_LOGS file on pontifex saying that old logfiles are stored on However, this file is from 2012, and I don't know if it's still true.

Georg, do you have a better answer? (Maybe the snapshots on the filer still have the access logs since Dec 23?)

Actions #4

Updated by witekbedyk about 1 year ago

I found the logs at backup.infra.o.o in /backup/pontifex/var/log/apache2

I need the ones from and
I've seen the first missing entries are from 22.12.2022 already.

Could you please copy these logs somewhere that I can read them from metrics.infra.o.o?

Actions #5

Updated by crameleon about 1 year ago

  • Assignee changed from crameleon to bmwiedemann

I found some family and hiking photo albums from > 10 years ago on this machine, I assume those are not what you are looking for. But there are indeed some and logs as well which seem to have ended around 2019 and 2017 respectively. The ominous "OLD_LOGFILES" text file can be found as well.

The snapshots cover the whole LUN, I'm not sure I can mount and browse them easily to check for these log files. Bernhard, can you?

Actions #6

Updated by pjessen about 1 year ago

Actions #7

Updated by witekbedyk about 1 year ago

  • Status changed from New to In Progress
  • Assignee changed from bmwiedemann to witekbedyk
Actions #8

Updated by witekbedyk about 1 year ago

  • Status changed from In Progress to Resolved

All missing logs have been restored from backup.infa.o.o. Kudos an Bernhard.
I've reaggregated the metrics and all the data is visible now again.


Also available in: Atom PDF