2 Replies Latest reply on Mar 28, 2014 3:37 PM by steelejaxon

    REGEX Help - Parse firewall data to include source country

    steelejaxon

      Hello,

       

      I would like to modify the existing Threat parser rule for my PA firewalls to include the source country. The existing rule is below but it doens't capture the country (India) in the data.

       

      THREAT\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<timestamp>[^\x2c]+)\x2c([^\x2c]+)\x2c([^\x 2c]+)\x2c(?:(?!(?:\1|0.0.0.0))([^\x2c]+)|\1|0.0.0.0)\x2c(?:(?!(?:\2|0.0.0.0))([^ \x2c]+)|\2|0.0.0.0)\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\ x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2 c]*\x2c[^\x2c]*\x2c(?P<src_port>[^\x2c]*)\x2c(?P<dst_port>[^\x2c]*)\x2c(?:(?!(?: \5|0))([^\x2c]+)|\5|0)\x2c(?:(?!(?:\6|0))([^\x2c]+)|\6|0)\x2c

       

      <12>Mar 28 03:01:30 1,2014/03/28 03:01:30,0009C100960,THREAT,vulnerability,0,2014/03/28 03:01:25,202.47.118.234,10.10.1.1,0.0.0.0,0.0.0.0,Generic_Rule_Name,,,web-brows ing,vsys1,source_zone,dest_zone,ethernet1/23.115,ethernet1/23.118,Panorama,2014/ 03/28 03:01:29,309155,2,49899,80,0,0,0x0,tcp,drop-all-packets,"php",PHP CGI Query String Parameter Handling Information Disclosure and DoS Vulnerability(34804),any,medium,client-to-server,71480107,0x0,India,10.0.0.0-10.255.255.255,0,

       

      Any assistance would be appreciated.

        • 1. Re: REGEX Help - Parse firewall data to include source country
          steelejaxon

          In case anyone else looks this up, this is what I ended up with.

           

          THREAT\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<timestamp>[^\x2c]+)\x2c(?P<src_ip>[^\x2c]* )\x2c(?P<dst_ip>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<rule_name>[^\x2c]*)\x2c (?:(?P<domain>[^\x2c\x5c]+)\x5c{1,2})?(?P<src_user>[^\x2c]*)\x2c(?:(?P<dstdomain >[^\x2c\x5c]+)\x5c{1,2})?(?P<dst_user>[^\x2c]*)\x2c(?P<application>[^\x2c]*)\x2c (?P<vsys>[^\x2c]+)?\x2c(?P<src_zone>[^\x2c]*)\x2c(?P<dst_zone>[^\x2c]*)\x2c(?P<i nbound_int>[^\x2c]*)\x2c(?P<outbound_int>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c(? P<session_id>[^\x2c]*)\x2c(?P<count>[^\x2c]*)\x2c(?P<src_port>[^\x2c]*)\x2c(?P<d st_port>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<protocol>[^\x2c]*)\ x2c(?P<action>[^\x2c\x2d]*)[^\x2c]*\x2c\x22?(?P<filename>[^\x2c\x22]*)\x22?\x2c( ?P<msg>.*?)\x28(?P<sid>(?:3\d{4})|(?:4[0-4]\d{3}))\x29\x2c(?P<category>[^\x2c]*) \x2c(?P<severity>[^\x2c]*)\x2c(0|client-to-server)\x2c[^\x2c]*\x2c[^\x2c]*\x2c(? P<PCAP_Name>[^\x2c]*)(?:\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?:0|(?P<PCAP_Name>[^\x2c]+) ))?

           

          This puts the country info into the PCAP name field. I tried putting it into another field but ran into issues. Also, if you want to do a filter query on that field you have to use "contains(country)" to do it. So for example, entering "contains(Peru)" in the filter for PCAP_Name now shows all the Threat events from that country.

           

          Next I will need to modify the other Palo Alto parser rules to include the country but this should give you an idea.

           

          Hit me up if anyone reads this and has a question.

          • 2. Re: REGEX Help - Parse firewall data to include source country
            steelejaxon

            Oh and the parser rule in the original post is NOT the one I modified. I copied the second regex in the parser rule instead of the first. The regex below is the unedited version (prior to modification).

             

            THREAT\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<timestamp>[^\x2c]+)\x2c(?P<src_ip>[^\x2c]* )\x2c(?P<dst_ip>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<rule_name>[^\x2c]*)\x2c (?:(?P<domain>[^\x2c\x5c]+)\x5c{1,2})?(?P<src_user>[^\x2c]*)\x2c(?:(?P<dstdomain >[^\x2c\x5c]+)\x5c{1,2})?(?P<dst_user>[^\x2c]*)\x2c(?P<application>[^\x2c]*)\x2c (?P<vsys>[^\x2c]+)?\x2c(?P<src_zone>[^\x2c]*)\x2c(?P<dst_zone>[^\x2c]*)\x2c(?P<i nbound_int>[^\x2c]*)\x2c(?P<outbound_int>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c(? P<session_id>[^\x2c]*)\x2c(?P<count>[^\x2c]*)\x2c(?P<src_port>[^\x2c]*)\x2c(?P<d st_port>[^\x2c]*)\x2c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?P<protocol>[^\x2c]*)\ x2c(?P<action>[^\x2c\x2d]*)[^\x2c]*\x2c\x22?(?P<filename>[^\x2c\x22]*)\x22?\x2c( ?P<msg>.*?)\x28(?P<sid>(?:3\d{4})|(?:4[0-4]\d{3}))\x29\x2c(?P<category>[^\x2c]*) \x2c(?P<severity>[^\x2c]*)\x2c(0|client-to-server)(?:\x2c[^\x2c]*\x2c[^\x2c]*\x2 c[^\x2c]*\x2c[^\x2c]*\x2c[^\x2c]*\x2c(?:0|(?P<PCAP_Name>[^\x2c]+)))?