On one of our remote proxies I have had the message that /var is exceeeding the limit (94% against 90%).
Out of 13.6 Gb, there is only ~920Mb remaining.
I've checked the contents of /var and it's the message logs taking up space:
Mar 15 15:45 messages 4Gb
Feb 17 03:25 messages-20130217 616Mb
Feb 24 03:47 messages-20130224 2.3Gb
Mar 3 03:13 messages-20130303 2.4Gb
Mar 10 03:28 messages-20130310 3.6Gb
I think I've managed to answer my own question in the course of research but wouldn't mind confirmation - I've been away from linux for a long time so still v much a newbie!
If i tail the message logs then it looks like it's all access.log info and each old log ends in a notification of a restart which I think is the syslog log rotation (all the log file dates are Sundays) .
Feb 17 03:25:01 MWG rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" x-pid="3506" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'restart'.
Feb 17 03:25:01 MWG kernel: Kernel logging (proc) stopped.
The rsyslog.conf is the default:
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
[root@MWG ~]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
# keep 4 weeks worth of backlogs
# create new (empty) log files after rotating old ones
# use date as a suffix of the rotated file
# uncomment this if you want your log files compressed
# RPM packages drop log rotation information into this directory
# no packages own wtmp and btmp -- we'll rotate them here
# system-specific logs may be also be configured here.
[root@MWG ~]# cat /etc/logrotate.d/syslog
I've also logged onto our 'master' proxy where i do all the config from and I have the same message log files, much smaller and (only) containing the same lines as the large logs - although it should be noted that this ships syslog out to SIEM which is possibly why it's smaller.
So, what I think is happening is that the messages log is logging everything of info and higher but not mail/ cron/ authpriv [mail.none;authpriv.none;cron.none]
This includes a weeks worth of access.logs ....
logrotate.conf specifies to rotate weekly & keep 4 weeks worth.
logrotate.d/syslog specifies the logs to rotate
Is that correct?
For now I've moved the old logs to a partition with more space until I configure the pushing ot SIEM - is that sensible?
I recommend the following:
Use the gui file editor to edit the rsyslog.conf. (Configuration > File Editor)
You will see this default line:
Please change this line to:
Notice the - in front of /var/log/messages. That's important. Together with the exclusion of daemon.info
These changes will prevent two things:
*unnecessary logging to /var/log/messages
*enable caching when it writes to /var/log/messages. Previously, it was writing every byte it received immediately causing high overhead.
This will still allow pushing to your SIEM but prevent it from actually writing the access.log to the messages file.
Thanks for the reply.
That line is the same as the one we have on the proxy that it pushing to SIEM so that would make sense.