On one of our remote proxies I have had the message that /var is exceeeding the limit (94% against 90%).
Out of 13.6 Gb, there is only ~920Mb remaining.
I've checked the contents of /var and it's the message logs taking up space:
Mar 15 15:45 messages 4Gb
Feb 17 03:25 messages-20130217 616Mb
Feb 24 03:47 messages-20130224 2.3Gb
Mar 3 03:13 messages-20130303 2.4Gb
Mar 10 03:28 messages-20130310 3.6Gb
I think I've managed to answer my own question in the course of research but wouldn't mind confirmation - I've been away from linux for a long time so still v much a newbie!
If i tail the message logs then it looks like it's all access.log info and each old log ends in a notification of a restart which I think is the syslog log rotation (all the log file dates are Sundays) .
Feb 17 03:25:01 MWG rsyslogd: [origin software="rsyslogd" swVersion="4.6.2" x-pid="3506" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'restart'.
Feb 17 03:25:01 MWG kernel: Kernel logging (proc) stopped.
The rsyslog.conf is the default:
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
[root@MWG ~]# cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
# keep 4 weeks worth of backlogs
# create new (empty) log files after rotating old ones
# use date as a suffix of the rotated file
# uncomment this if you want your log files compressed
# RPM packages drop log rotation information into this directory
# no packages own wtmp and btmp -- we'll rotate them here
# system-specific logs may be also be configured here.
[root@MWG ~]# cat /etc/logrotate.d/syslog
I've also logged onto our 'master' proxy where i do all the config from and I have the same message log files, much smaller and (only) containing the same lines as the large logs - although it should be noted that this ships syslog out to SIEM which is possibly why it's smaller.
So, what I think is happening is that the messages log is logging everything of info and higher but not mail/ cron/ authpriv [mail.none;authpriv.none;cron.none]
This includes a weeks worth of access.logs ....
logrotate.conf specifies to rotate weekly & keep 4 weeks worth.
logrotate.d/syslog specifies the logs to rotate
Is that correct?
For now I've moved the old logs to a partition with more space until I configure the pushing ot SIEM - is that sensible?
I recommend the following:
Use the gui file editor to edit the rsyslog.conf. (Configuration > File Editor)
You will see this default line:
Please change this line to:
Notice the - in front of /var/log/messages. That's important. Together with the exclusion of daemon.info
These changes will prevent two things:
*unnecessary logging to /var/log/messages
*enable caching when it writes to /var/log/messages. Previously, it was writing every byte it received immediately causing high overhead.
This will still allow pushing to your SIEM but prevent it from actually writing the access.log to the messages file.