cancel
Showing results for 
Search instead for 
Did you mean: 
btlyric
Level 12

Upload vs. Download

We were instructed that if the cycle = Request/Embedded Object, then that equals an upload.

I've been doing some testing and my testing doesn't back up that assertion.

One example:

POST request that results in a file downloaded from remote site

Has anyone else dug into this area?

0 Kudos
8 Replies
pbrickey
Level 11

Re: Upload vs. Download

There's two parts to the traffic in your scenario - the actual POST (request) from the client and the response from the server that results in the file downloaded.

When MWG process your request, the POST, through the rule engine that is the REQUEST cycle.

When MWG process the response frorm the server with the file that is downloaded that is the RESPONSE cycle.

If it is a zip file or other archive that MWG can extract/decompress other files it will send those files through the rule engine in the Embedded Object cycle.

Hope that helps,

Patrick

0 Kudos
btlyric
Level 12

Re: Upload vs. Download

I was probably unclear in my original port. I will try to clarify.

Based on input from Professional Services, I have a rule set with top level criteria of Request and Embedded Objects. The MWG GUI itself says HTTP(S)/FTP uploads if you hover over Requests (and IM) and says HTTP(S)/FTP downloads if you hover over Requests. But I digress.

My rule set then has additional rules...

Bypass Monitoring

- various criteria for bypassing the monitoring -- known destinations, etc.

Skip Empty Requests

Reset Properties

- in this rule set, various properties are set to false or null values

Set Specific Properties (criteria Cycle.Name equals EmbeddedObject)

- in this rule set, I set various values such as Body.Filename, Body.Size, MediaType, Body.NumberofChildren, etc.  to User-Defined Properties

I then have a series of rules that look for things like Body.IsCorrupted equals true, Body.IsEncrypted equals true, Body.Size > X bytes, etc.

This rule set triggers not only on uploads, but also on downloads.

So okay, it's triggering on the Embedded Object cycle, but if that's the case, how can I distinguish between an upload vs. a download?

I can't use BytesFromClient vs. BytesFromServer because that's only valid during the Logging cycle.

So my question is how can I differentiate between something that's being uploaded vs. something that's being downloaded given that Request cycle + Command.Name != GET doesn't do the trick?

0 Kudos
eelsasser
Level 15

Re: Upload vs. Download

Try seeing if Cycle.TopName works for you.

0 Kudos
btlyric
Level 12

Re: Upload vs. Download

Shouldn't

Applies to: Requests (and IM) be the same as Cycle.TopName = Request when it's invoked for a rule set?

If I log the Cycle.TopName and the Cycle.Name during that phase, it shows that Cycle.TopName is "Request"

Thanks!

0 Kudos
eelsasser
Level 15

Re: Upload vs. Download

Is that what you want?

The way i understand it it is the Top.CycleName is like the root cycle that calls the embedded cycles.

So on an upload, i get this as a file is getting unzipped in the embedded cycles:

"multipart/form-data" "Request/Request" "-"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|NameLessFile"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|NameLessFile"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|mingwm10.dll"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|Win32DiskImager.exe"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|QtGui4.dll"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|QtCore4.dll"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|GPL-2"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|LGPL-2.1"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|README.txt"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|libstdc++-6.dll"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|win32diskimager-RELEASE-0.3-r27-binary.zip|libgcc_s_dw2-1.dll"
"multipart/form-data" "Request/EmbeddedObject" "NameLessFile|NameLessFile"


And on a download I get this:

"application/x-zip-compressed" "Response/Response" "WinFormHtmlEditor.zip"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/de-DE.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/en-AU.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/en-CA.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/en-GB.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/en-US.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/es-ES.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/es-MX.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/fr-FR.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/it-IT.dic"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Learn about the structure of a dictionary.html"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Learn how to make Custom Dictionary.html"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Licenses/BSD.txt"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Licenses/GNU-GPL.txt"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Licenses/LGPL.txt"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Licenses/Microsoft Public License.htm"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/Licenses/_Simply Licensing Explained.html"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Dictionary files for spell checker/_Read Me.txt"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Main Control DLL/Microsoft.mshtml.dll"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Main Control DLL/WinFormHtmlEditor.dll"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Main Control DLL/_instruction.txt"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Sample Projects/Custom Context Menu/C # Sample/Form1.cs"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Sample Projects/Custom Context Menu/C # Sample/Form1.Designer.cs"
"application/x-zip-compressed" "Response/EmbeddedObject" "WinFormHtmlEditor.zip|Sample Projects/Custom Context Menu/C # Sample/Form1.resx"

<snip>

0 Kudos
btlyric
Level 12

Re: Upload vs. Download

Cycle.TopName in the overall criteria seems to do the trick. Does that mean that Applies to: is for the Cycle.Name? Or?

Also, what properties are you using to generate the portion of the logged line that shows the main file name and then the embedded file names?

0 Kudos
eelsasser
Level 15

Re: Upload vs. Download

That log was generated from this rule I put in the root of my default MediaType Filtering  rules. I just pasted the last few fields into the last message post.

Media Type Filtering
[Rules to block media types during upload and download for user group "internet_strict".]
Enabled
Applies to Requests: True / Responses: True / Embedded Objects: True
Always
EnabledRuleActionEventsComments
Enabled

Log Cycles

Always

ContinueSet User-Defined.logLine =
     DateTime.ToWebReporterString +
     " "" +
     String.ReplaceIfEquals(Authentication.UserName,"","-") +
     "" " +
     String.ReplaceIfEquals(IP.ToString(Client.IP),"","-") +
     " "" +
     String.ReplaceIfEquals(List.OfString.ToString(DNS.Lookup.Reverse(Client.IP)),"","-") +
     "" " +
     String.ReplaceIfEquals(IP.ToString(URL.Destination.IP),"","-") +
     " " +
     String.ReplaceIfEquals(Number.ToString(Response.StatusCode),"","-") +
     " "" +
     String.ReplaceIfEquals(List.OfMediaType.ToString(MediaType.EnsuredTypes),"","-") +
     "" "" +
     String.ReplaceIfEquals(Cycle.TopName,"","-") +
     "/" +
     String.ReplaceIfEquals(Cycle.Name,"","-") +
     "" "" +
     String.ReplaceIfEquals(Body.FullFileName,"","-") +
     "" "
FileSystemLogging.WriteLogEntry(User-Defined.logLine)<POST.log>

0 Kudos
btlyric
Level 12

Re: Upload vs. Download

Body.FullFileName. Bingo. Thanks!

0 Kudos