8 Replies Latest reply: Feb 1, 2013 1:54 AM by alexott RSS

    Comments in External Lists

    ctsean

      Is it possible to add comments to an external list text file? 

       

      I am setting up an IP list and want to add comments for my team members to indicate the syntax the list is looking for.  Any insight would be much appreciated.

       

      Thank you!

        • 1. Re: Comments in External Lists
          alexott

          Hello

           

          For plain text files you can specify regex that will be used to select lines that will be imported. For example, if you want to treat as comments lines that starts with the '#' character, then you need to specify regex like ^[^#].*, so only strings that don't start with '#' will be included into list

          • 2. Re: Comments in External Lists
            ctsean

            But without manually specifying the regex route, nothing build into the functionality of the list?

            • 3. Re: Comments in External Lists
              alexott

              Yes, nothing specific was built into MWG to specify comments. But regexes will work without any problem...

              • 4. Re: Comments in External Lists
                eelsasser

                however, Subscribed lists can have comments.

                 

                If i have a list like this on a web server;

                 

                type=string

                "208.99.94.78" "Comment 1"

                164.109.94.147

                "212movie.com" "Another comment"

                "actforlove.typepad.com" "What is this?"

                "active.com" "comment"

                activehealthsftp.net

                 

                Then they will be viewable on the list:

                Capture.png

                • 5. Re: Comments in External Lists
                  ctsean

                  Erik - this is helpful also.  I wanted to lead in with some comments that detail what the syntax should be, and then I can also add comments here so we can track the entries also.

                   

                  -Sean

                  • 6. Re: Comments in External Lists
                    clausonna

                    Sorry to jump in the middle here, but this raises an question for me:  Are the comments available as a property to the rest of the gateway?  My use case would be something like a custom black-list of malicious IP addresses with the comment being the date that specific entry was added to the list (or maybe it is the source of the blacklist, e.g. "Snort alert", or "ETPRO Reputation Feed".  Being able to include the comment in the Block page and/or the access.log would be pretty cool.  I looked around and couldn't find anything for this. 

                    • 7. Re: Comments in External Lists
                      eelsasser

                      The list comments cannot be captured with a property and used anywhere like a block page.

                      They are strictly for documenting and viewing within the policy.

                      I've wanted something like that myself, but hasn't happened.

                      • 8. Re: Comments in External Lists
                        alexott

                        Hi clausonna

                         

                        I think that you can do this with new Map Type that is available in 7.3.1 release. But comments should be on the same line as the data - so we could use regex to capture data & comments... You need to do following steps:

                        • Create ExtLists settings with your data source (web service), specify URL, and also specify regular expression, like this: ^(.*?)(?:\s*#\s*(.*?))?$ - this exprassion shouldl have 2 capture groups so it could be used as a map
                        • Create rule that should block destination IP. There are 2 possibilities here:
                          • you can block by checking is DestinationIP-string is in Map, with something like: Map.HasKey(ExtLists.StringMap<your settings>(params...), IP.ToString(URL.DestinationIP))
                          • or you can check is Destination IP in IP List that is also fetched by ExtLists filter with ExtLists.IPList<your settings>(params...) property - this maybe slightly faster from performance point of view, but will require additional fetch of data from external service.
                        • If rule matches, then block request with custom block page that contains comment about given IP (see below)

                         

                        File with data should have following form:

                         

                        10.149.114.44 # bad site

                        194.87.0.50 # another bad site

                        173.194.64.106

                         

                        Block page template can contain expression: Map.GetStringValue(ExtLists.StringMap<your settings>(params...), IP.ToString(URL.Destination.IP)) - this will fetch comment for given IP address

                         

                        I attached file with rules & block page, so you can play with this approach