5 Replies Latest reply on May 29, 2013 11:50 PM by rohanparath

    Parsing logs from a file

    rohanparath

      Hi All,

       

      I am new to NiroSecurity. I wanted to know how to go about configuring NitroSecurity to parse logs from a file. Say for example IIS or apache logs that are logging to a file. Is there an agent that can read and parse logs from a file. I went through the user guide and did not find any information regarding this. In the case of IIS and apache the guide mentions that these can be configured to send logs as syslogs. However the requirement is to read logs from a file and not send them as syslog. Any information regarding this would be really helpfull.

       

      Thanks in advance

      Rohan

        • 1. Re: Parsing logs from a file
          haroot

          Hi Rohan,

           

          File based logs can be collected using Mcafee's Windows/linux Agent. Below is the KB article which includes the details of using McAfee windows Agent for IIS as well as DNS log collection.

           

          http://kc.mcafee.com/corporate/index?page=content&id=KB74849&actp=search&viewloc ale=en_US&searchid=1369771659726

           

          Cheers

          1 of 1 people found this helpful
          • 2. Re: Parsing logs from a file
            rohanparath

            Thanks Haroot that was helpful.

             

            Do you have a link to a KB article or documentation for reading logs from a linux system remotely? I would need to read and parse application from a linux system remotely and don't seem to see any documentation on how to proceed with this integration.

             

            Thanks in advance.

            Rohan

            • 3. Re: Parsing logs from a file
              haroot

              Hi Rohan,

               

              You can access the KB site http://kc.mcafee.com/corporate/index?page=home  and search for LInux data source. One good thing about McAfee Siem is that you are not limited to one method of collection. As an alternative to Agent you can use collection mechanisn such as scp,sftp to collect the logs from the data sources.

               

              Hope this is helpful.

               

              Haroot

              • 4. Re: Parsing logs from a file
                chris_hankins

                Hi Rohan,

                 

                We have been researching a simliar way to retrieve application log files and the Linux agent seems to be the way to go. I came across this in my research. Hope it helps:

                 

                McAfee Linux Event Collector 9.1.3 provides you with the capability to add a local agent to your system to push several types of data to the McAfee Event Receiver.

                The installer is available by calling McAfee Support at 800-937-2237.

                 

                -------------------------

                 

                Supported Versions

                 

                Ubuntu 10.04 Uses mcafee-linux-event-collector_9.1.1.0-358_1004_amd64.deb

                Ubuntu 12.04 Uses mcafee-linux-event-collector_9.1.1.0-358_1204_amd64.deb

                Redhat 5.8  Uses mcafee-linux-event-collector_9.1.1.0-358.el5.x86_64.rpm

                Redhat 6.2  Uses mcafee-linux-event-collector_9.1.1.0-358.x86_64.rpm 

                Fedora 16   Uses mcafee-linux-event-collector_9.1.1.0-358.x86_64.rpm

                Suse 11   Uses mcafee-linux-event-collector_9.1.1.0-358.x86_64.rpm

                 

                ------------------------- 

                 

                Installing the Agent

                 

                Run the installer by double clicking the .deb or .rpm from the gui or

                using rpm -i package.rpm from the command line for rpm and dpkg -i package.deb for deb

                 

                End-User License is here:

                 

                /usr/share/doc/mcafee/EULA McAfee - Corporate-August 2010.rtf

                 

                -------------------------

                 

                Configuring the Agent

                 

                To date filetail is the only plugin "type" that is supported, but you can have as many filetail sections as you want.

                The file to be tailed must be on the local system not a mounted file.

                The path to your conf file is below you can change the default path of the conf file by changing the path in the init script.

                 

                /etc/mcafee/mcafee_event_collector.conf

                 

                bookmark_dir= Is directory where bookmark file is saved and is configurable.

                debug_level= Is the level of debug output by the collector options are error,info,warning,and debug. 

                log_path= Is the direcotry where the log is written. 

                sleep= If a file has not been modified since the agent was last shutdown, on startup will put the file in a watch list and check on it from time to time. If there are files in the watch list, the agent will check it every x number of seconds.

                inactive_sleep= If there are no files in the watch list, the agent will sleep y number of seconds, before waking and checking for files in the watch list.

                 

                rec_ip= Is the IP of the receiver to send events to.

                rec_port= Is the port of the receiver is listining on. 

                rec_encrypt= Changin this value enables or disables encryption 0=off 1=on

                 

                type= Is the plugin type.  (To date filetail is the only plugin "type" that is supported, but you can have as many filetail sections as you want.)

                subtype = Is a subtype of the plugin. ( To date big_fix is the only subtype that is supported.) Big_fix logs with a date at the top of a File with this subtype option it takes that date and appends it to the beggining of each event.

                hostid = Put a value here if you would like to use a Host ID on the receiver. 

                ft_dir = Directory where plugin will look for files to tail. 

                ft_filter = Filter for what file to tail ie. mesages or log.*

                ft_delim =  Delemiter for collector to know when a new event has happend ie. <newline>, <space>, <tab>, Regular expressions are also supported. 

                ft_delim_end_of_event = Delemiter to start at the begginging or the end of the event 0=begginging 1=end Default is 1

                ft_start_top = This tells us to start at the top of the file 0=no 1=yes

                 

                 

                 

                See example Configuration file at bottom of this docuemnt. 

                 

                -------------------------

                 

                Running the Agent

                 

                 

                Once you have completed editing the file, restart your Event Collector service with this cmd:

                 

                /etc/init.d/mcafee_event_collecotr restart or

                service mcafee_event_collector restart

                 

                start and stop are also options.

                 

                you can also run the Agent manualy run /usr/bin/event_collector -h to see your options

                To enable auto learning for the agent run event_collector manually from command line with the -a option

                 

                 

                -------------------------

                 

                Example Configuration File with two filetail sections with one using a hostid. 

                 

                ##############

                # Collector

                ##############

                bookmark_dir=/var/lib/mcafee/bookmark

                debug_level=error

                log_path=/var/log/mcafee/event_collector.log

                sleep=5

                inactive_sleep=300

                 

                ##############

                #       Receiver

                ##############

                rec_ip=10.0.0.0

                rec_port=8081

                rec_encrypt=0

                 

                ##############

                #       Plugin

                ##############

                type=filetail

                hostid=

                ft_dir=/var/log

                ft_filter=log.1

                ft_delim=<newline>

                ft_delim_end_of_event=1

                ft_start_top=1

                 

                type=filetail

                hostid=mesages

                ft_dir=/var/log

                ft_filter=messages

                ft_delim=<newline>

                ft_start_top=1

                 

                # Sample Big fix logging

                type=filetail

                subtype=big_fix

                hostid=

                ft_dir=/var/log

                ft_filter=*.log

                ft_delim=At \d*:\d*:\d* -\d*

                ft_delim_end_of_event=0

                ft_start_top=1

                 

                -------------------------

                • 5. Re: Parsing logs from a file
                  rohanparath

                  Thanks a lot Chris for this. I will check this option and let you know how it goes.

                   

                  Regards,

                  Rohan