Project

General

Profile

Actions

tickets #112298

closed

anna/elsa - named query log - size limit does not work?

Added by pjessen almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Core services and virtual infrastructure
Target version:
-
Start date:
2022-06-12
Due date:
% Done:

100%

Estimated time:

Description

Although the config says it should be limited to 100Mb, that doesn't seem to work. Today it was 3909126173 before I deleted it.

Actions #1

Updated by pjessen almost 2 years ago

  • Private changed from Yes to No

After only a couple of hours, the log has already grown past 100Mb. I have stopped the logging rndc querylog off.

Actions #2

Updated by pjessen almost 2 years ago

pjessen wrote:

After only a couple of hours, the log has already grown past 100Mb. I have stopped the logging rndc querylog off.

I have also stopped the logging on elsa, the querylog was 4.7Gb.

Actions #3

Updated by pjessen almost 2 years ago

  • Subject changed from anna - named query log - size limit does not work? to anna/elsa - named query log - size limit does not work?
Actions #4

Updated by pjessen almost 2 years ago

anna and elsa both have the following config:

    # Log queries to a file limited to a size of 100 MB.
    channel query_logging {
        file "/var/log/named_query.log" versions 3 size 100M;
        severity info; 
        print-time yes;
        print-category yes;
        print-severity yes;
    };
    category queries        { query_logging; };

This looks fine to me, 3 logfiles of 100MB each.

However, looking at the log, and I see the following (from a random date)

/var/log/messages-20220606.xz:2022-06-05T00:10:03.384320+00:00 elsa named[5890]: unable to rename log file '/var/log/named_query.log' to '/var/log/named_query.log.0': permission denied
/var/log/messages-20220606.xz:2022-06-05T00:10:06.757791+00:00 elsa named[5890]: message repeated 197 times: [ unable to rename log file '/var/log/named_query.log' to '/var/log/named_query.log.0': permission denied]
Actions #5

Updated by pjessen almost 2 years ago

  • Category set to Core services and virtual infrastructure

The first time I see this permission issue is on 2022-06-02T16:27:50 - I'm going to make a wild guess and say anna and elsa were upgraded to run Leap 15.4.

Actions #6

Updated by pjessen almost 2 years ago

Hmm, according to /etc/sysconfig/named::NAMED_RUN_CHROOTED we are supposed to be running chrooted, so why are we logging in /var/log/ ? Looking at the previous logfiles, everything worked fine up until the upgrade :-)

# l /var/lib/named/log
total 338732
drwxr-xr-x  2 named named      4096 Jun  2 12:53 ./
drwxrwxr-t 10 root  named      4096 Jun 12 14:01 ../
-rw-r--r--  1 named named     28803 Jun  2 13:41 named.memstats
-rw-r--r--  1 named named  32006980 Jun  2 13:41 named_query.log
-rw-r--r--  1 named named 104857623 Jun  2 12:53 named_query.log.0
-rw-r--r--  1 named named 104857603 Jun  2 10:12 named_query.log.1
-rw-r--r--  1 named named 104857673 Jun  2 07:50 named_query.log.2
-rw-r--r--  1 named named    203457 Jun  2 00:41 named_zone_transfers.log

I guess maybe named isn't running chrooted ? Judging by the running process, there is no '-t /var/lib/named' argument.

Interesting comment from the bind package changelog:

* Thu Mar 17 2022 josef.moellers@suse.com
- Obsolete bind-chrootenv as it is no longer needed.
  [bsc#1196990, bind.spec]
Actions #7

Updated by pjessen almost 2 years ago

I am going to disable query logging in the config, as I expect my rndc querylog off will not survive the next reboot. It seems to me that our bind package was changed quite a bit, ignoring e.g. the sysconfig setting. Instead of running chrooted, we are now using systemd sandboxing, a setup I am not at all familiar with.

Actions #8

Updated by pjessen almost 2 years ago

  • Status changed from New to Resolved
  • Assignee set to pjessen
  • % Done changed from 0 to 100

pjessen wrote:

I am going to disable query logging in the config, as I expect my rndc querylog off will not survive the next reboot. It seems to me that our bind package was changed quite a bit, ignoring e.g. the sysconfig setting. Instead of running chrooted, we are now using systemd sandboxing, a setup I am not at all familiar with.

I am going to assume that no one actually need those querylogs, that the config was just the standard one. I have amended named.conf on anna+elsa with a reference to this ticket, otherwise I am not going to pursue the core issue.

Actions #9

Updated by cboltz almost 2 years ago

I think I can explain what's going on:

pjessen wrote:

/var/log/messages-20220606.xz:2022-06-05T00:10:03.384320+00:00 elsa named[5890]: unable to rename log file '/var/log/named_query.log' to '/var/log/named_query.log.0': permission denied

While running in a chroot, named owned the log directory /var/lib/named/log/ and was able to rename the log.

However, /var/log/ (without chroot) is root-owned, which means named isn't allowed to rename its log - see the "permission denied" I quoted. I guess that's worth a bugreport - it should have a subdirectory like /var/log/named/ where it has write permissions again. https://bugzilla.opensuse.org/show_bug.cgi?id=1200685 created.

You can keep the logging switched off nevertheless ;-)

Actions #10

Updated by pjessen almost 2 years ago

cboltz wrote:

I think I can explain what's going on:

pjessen wrote:

/var/log/messages-20220606.xz:2022-06-05T00:10:03.384320+00:00 elsa named[5890]: unable to rename log file '/var/log/named_query.log' to '/var/log/named_query.log.0': permission denied

While running in a chroot, named owned the log directory /var/lib/named/log/ and was able to rename the log.

However, /var/log/ (without chroot) is root-owned, which means named isn't allowed to rename its log - see the "permission denied" I quoted. I guess that's worth a bugreport - it should have a subdirectory like /var/log/named/ where it has write permissions again. https://bugzilla.opensuse.org/show_bug.cgi?id=1200685 created.

I didn't want to start looking into the systemd sandboxing, but the service unit does have ReadWritePaths=/var/lib/named /run/named /var/log
which at first glance suggests it can write to /var/log.

You can keep the logging switched off nevertheless ;-)

Thanks :-)

Actions #11

Updated by cboltz almost 2 years ago

pjessen wrote:

I didn't want to start looking into the systemd sandboxing, but the service unit does have ReadWritePaths=/var/lib/named /run/named /var/log
which at first glance suggests it can write to /var/log.

Right, but the "normal" (DAC) file and directory permissions still apply. So it can write to files owned by named there, but (since the directory is root:root 755) it can't rename files or create new files.

That's also why I proposed a subdirectory owned by named in the bugreport - in such a directory, creating and renaming files isn't a problem.

Actions #12

Updated by pjessen almost 2 years ago

cboltz wrote:

Right, but the "normal" (DAC) file and directory permissions still apply. So it can write to files owned by named there, but (since the directory is root:root 755) it can't rename files or create new files.

Ah yes. Nice find!

Actions

Also available in: Atom PDF