5 Replies Latest reply on Feb 9, 2017 9:20 AM by Scott Taschler

    ATD tools

    Val

      Hi folks,

       

      I would to share two utilities which might be helpful for ATD users.

       

      ATDClient

      https://github.com/passimens/atdclient

      A command-line tool which submits specified file to ATD and saves a report based on analysis results.

      Example:

      ATDClient.exe -a 10.10.10.10 -u atduser -p atdpass -t pdf -f calc.exe

      This will submit calc.exe for analysis and save the respective pdf report to calc.exe.atd.pdf.

       

      ATDScan

      https://github.com/passimens/atdscan

      A command-line scanner tool using ATD as a back-end. Works similar to VSE On-Demand Scanner.

       

      Example:

      ATDScan10.exe c:\temp

      This will submit all the files in the temp folder to ATD and save the scan log with the results.

      ATDScan10.exe -x c:\ProgramData\Microsoft c:\Users\Val\Dropbox -q c:\atdquar -c 4 c:\ProgramData c:\Users\Val\

      This will scan all the files in ProgramData and User profile excluding Microsoft and Dropbox folders. All the files with severity 4 and above will be moved to quarantine c:\atdquar.

      ATDScan10.exe -t 5 c:\ProgramData\*\*.exe c:\Users\Val\AppData\Roaming\*

      This will scan exe files in the ProgramData one-level nested folders and all files in Roaming folder and subfolders, using 5 scanning threads.

        • 1. Re: ATD tools
          Scott Taschler

          Hi Val,

           

          This is some pretty interesting stuff.  I also recently noticed the python module you posted last spring.  Would love to hear more about how you're putting these to use.

           

          Scott

          • 2. Re: ATD tools
            Val

            Hi Scott,

             

            Great to see interest from the community!

             

            I use ATDClient during PoCs and demos to verify ATD connectivity, quickly submit file samples under different users (i.e. analysis profiles).

            It might also be used for daily operations, for express suspicious sample verification, without the need to login into web UI.

             

            ATDScan is a bit more complicated and flexible tool.

            Main use cases include:

            -Scanning a baseline image to pre-populate ATD with known good samples, identify possible false positives, reduce the time for consecutive file submissions.

            -Scanning a potentially compromised system for the signs of unknown malware pieces. This is useful when VSE and/or TIE are not available or not functioning on the system.

             

            Alternative to this is TIEScan. However, it will miss malicious content put inside a script or msoffice file.

            1 of 1 people found this helpful
            • 3. Re: ATD tools
              Scott Taschler

              Thanks for the additional context, Val.  I'm curious about what your experience been when using ATDScan on a full system?  Seems like you have the possibility to over-subscribe ATD with a very large number of submissions in a very short time period, if you chose to run it against, for example, C:\.  Does it have any sort of in-built down-selection mechanism (i.e. don't send files with known good GTI reputation)?  Any other cautions you'd offer prior to using it in the field?

               

              Scott

              • 4. Re: ATD tools
                Val

                Hi Scott,

                 

                The tool is working synchronously and multi-threaded. If we set thread number to 10 the scanner will submit up to 10 files to the ATD in parallel. Then it will wait for any on these 10 jobs to complete before submitting next samples.

                These are the main ways to avoid ATD oversubscription:

                1) do not use --reanalyze/-r option (--no-reanalyze/-R is the default) without a specific need. By default the scanner checks if ATD has any previous results for the MD5, if yes it does not submit the file and uses the latest of previous analysis results. --reanalyze forces ATD to analyze the sample again, even if Analyzer Profile set to 'skip previously analyzed'.

                2) exclude archive file types from scanning (via atdscan.ini file). Each archive file creates one job, but might create enormous number of tasks, based on its content.

                3) use a dedicated atd user (e.g. ods), and a dedicated VM profile (e.g. ODS_Profile) with a specific number of licenses (say, 5). Run the tool with 5 scanning threads (via atdscan.ini, or CLI: '-t 5').

                4) use an Analyzer profile with all downselectors (engines) enabled, and 'Continue to run all engines even after file is found malicious' option disabled.

                 

                Option (1) is extremely helpful when scanning similar sets of files multiple times.

                Options (2) & (3) can guarantee ATD is working within predicted loads.

                Option (4) is always useful when submitting a huge number of files.

                 

                I haven't done a thorough performance testing, taking note on all the relevant factors (there is a significant number of those actually).

                However, my own field experience shows the following:

                 

                1) Scanning user profile folder on Win7 (40G of files)

                - ATD-3000 box 3.4.x-3.8.x (no background load)

                - dedicated VM profile with 15 licenses

                - fast analyzer profile (all downselectors enabled, stop as soon as convicted, timeout 60s)

                - atdscan is run with 15 scan threads, excluding archive files, without reanalyze option, file size limits - 1K..50M bytes (ini)

                Results:

                - first scan completed in 16 hours

                - ATD was fully operational and responsive during all the process

                 

                2) Repeated run with the same options immediately after scenario (1)

                Results: scan completed in 1 minute (nothing actually submitted to ATD)

                 

                3) Repeated scenario (1) including archive files

                Results:

                - the scan completed in half an hour (3858 files scanned)

                - ATD was fully operational and responsive during all the process

                 

                4) Scanning the whole C:\ on a system with about 200Gb of files

                - ATD-3000 box 3.6.x (slight background load from NSP, TIE - queues do not build up)

                - shared VM profile with 20 licenses (all downselectors enabled, stop as soon as convicted, timeout 180s)

                - atdscan was run with 10 scan threads, including archive files, without reanalyze option, file size limits - 1K..50M bytes (ini)

                Results:

                - the scan started on Friday afternoon, completed in about two days

                - ATD was fully operational and responsive during the beginning of the scan and on Monday morning after the scan, not monitored during weekend

                • 5. Re: ATD tools
                  Scott Taschler

                  Very interesting.  Thank you for posting!

                   

                  Scott