We use Webgate and generate around 20Gb of logs per day which are then exported out to a sys log server for archive. Often we will get requests for user activity for a six month block which can generate a parsed csv of several hundred thousand lines. Can anyone recommend some options for applications that can import either the raw log or a csv parsed log and conduct meaningful interpretation over the data - Extracting out search queries, specific websites visited; YouTube channels/videos and or highlight supplied keywords and/or the URL categorization tags.
We use McAfee Web Reporter tool to parse,search mwg access logs. It is free for mwg licensed users and can generate reports for user,IP,group,category etc ...