cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Former Member
Not applicable
Report Inappropriate Content
Message 1 of 7

ESM Parsing Regular expression

Hi,

Can anybody help me to understand the parsing regular expression, I'm just working on creating a custom parsing for a device. and i got stuck in the regular expressions and what to understand that.

Thank you

Regards

Soji Thomas

6 Replies
Former Member
Not applicable
Report Inappropriate Content
Message 2 of 7

Re: ESM Parsing Regular expression

This page migth be halp to unterstand regex --> RegExr: Learn, Build, & Test RegEx

Former Member
Not applicable
Report Inappropriate Content
Message 3 of 7

Re: ESM Parsing Regular expression

The goal of the regular expression is to match the fields in the log that need to be captured and put into the relevant database fields so they are available for search, correlation and reporting on the ESM/ACE. Parentheses within the regex are used to create each match group. For instance, if I have the following log line:

<22>Jul  9 12:55:49 tecate dovecot: imap(bob): Disconnected: Logged out bytes=671/3325

I can parse it with the following regex:

(\w+)\s(dovecot)\x3a\simap\x28(\w+)\x29\x3a\sDisconnected\x3a\sLogged\sout\sbytes\x3d(\d+)\x2f(\d+)

Every time you see a pair of parentheses I'm capturing a field. So the fields here would be:

hostname=tecate

application=dovecot

source user=bob

bytes_received=671

bytes_sent=3325

This information alone won't directly help you to make the parsing rules though since you do need to learn a bit of regex and there are some nuances with the editor but it should give you an idea of the process.

That being said, it looks like your log sample in your screenshot is of McAfee NSP logs. It's better to grab those via a SQL pull of the NSM. If you need to use syslog, there are existing rules for the data source.

alexander_h
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 4 of 7

Re: ESM Parsing Regular expression

That is the CheatSheet from the RegEXR that was mentioned by XDED

Now they also have a desktop app

Character classes

.any character except newline
\w \d \sword, digit, whitespace
\W \D \Snot word, digit, whitespace
[abc]any of a, b, or c
[^abc]not a, b, or c
[a-g]character between a & g
Anchors
^abc$start / end of the string
\bword boundary
Escaped characters
\. \* \\escaped special characters
\t \n \rtab, linefeed, carriage return
\u00A9unicode escaped ©
Groups & Lookaround
(abc)capture group
\1backreference to group #1
(?:abc)non-capturing group
(?=abc)positive lookahead
(?!abc)negative lookahead
Quantifiers & Alternation
a* a+ a?0 or more, 1 or more, 0 or 1
a{5} a{2,}exactly five, two or more
a{1,3}between one & three
a+? a{2,}?match as few as possible
ab|cdmatch ab or cd
alexander_h
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 5 of 7

Re: ESM Parsing Regular expression

Also you might find really useful that you could create multiple small parsers so you could match fragments instead of parsing the entire message.

Give it a try and let us know so we could give you some hints.

Clarck
Level 7
Report Inappropriate Content
Message 6 of 7

Re: ESM Parsing Regular expression

Goog afternoon

Data Source azure send logs  log to always SIEM.

But SIEM does not parsing all the fields.

Log

{"Level": 4, "callerIpAddress": "200.3.98.250", "category": "SignInLogs", "correlationId": "d50c2db2-0e37-48e8-b391-00ccf9273d96", "durationMs": 0, "identity": "Juan Jose Ferreyra", "location": "AR", "operationName": "Sign-in activity", "operationVersion": "1.0", "properties": {"appDisplayName": "Microsoft Teams Web Client", "appId": "5e3ce6c0-2b1f-4285-8d4b-75ee78787346", "appliedConditionalAccessPolicies": [{"conditionsNotSatisfied": 0, "conditionsSatisfied": 287, "displayName": "A010 MFA para grupo MFA_Users para todas las apps", "enforcedGrantControls": ["Mfa"], "enforcedSessionControls": [], "id": "419161e4-44ec-4957-b569-ed1b8ad1fb8d", "result": "success"}, {"conditionsNotSatisfied": 0, "conditionsSatisfied": 11, "displayName": "A050 MFA Todos los usuarios", "enforcedGrantControls": ["Mfa"], "enforcedSessionControls": [], "id": "c772f154-e9f5-49a6-a786-079a1c27e3dd", "result": "success"}, {"conditionsNotSatisfied": 2, "conditionsSatisfied": 1, "displayName": "A020 MFA para todos los ROLES Admin de AzureAD Siempre", "enforcedGrantControls": ["Mfa"], "enforcedSessionControls": [], "id": "45264b8a-fcde-4722-bb1c-faa589b30c91", "result": "notApplied"}, {"conditionsNotSatisfied": 2, "conditionsSatisfied": 1, "displayName": "A030 MFA Exceptuado para IPs de Pampa", "enforcedGrantControls": [], "enforcedSessionControls": ["PersistentBrowserSessionMode"], "id": "b8d5bb4d-088f-46d0-a434-e73cd93bc468", "result": "notApplied"}, {"conditionsNotSatisfied": 8, "conditionsSatisfied": 3, "displayName": "A040 Origenes no permitidos", "enforcedGrantControls": ["Block"], "enforcedSessionControls": [], "id": "ab4d0ae6-19b5-4efa-b85d-826eaddf3936", "result": "notApplied"}, {"conditionsNotSatisfied": 2, "conditionsSatisfied": 1, "displayName": "A900 Intune - Compliant Device", "enforcedGrantControls": ["RequireCompliantDevice"], "enforcedSessionControls": [], "id": "a22e3119-1379-4103-ac69-b168cb084b57", "result": "notApplied"}, {"conditionsNotSatisfied": 2, "conditionsSatisfied": 1, "displayName": "A901 Approved Client App - Android", "enforcedGrantControls": ["RequireApprovedApp"], "enforcedSessionControls": [], "id": "6452f246-a32e-43c3-abe7-333a9df09e49", "result": "notApplied"}], "authenticationDetails": [{"RequestSequence": 0, "StatusSequence": 0, "authenticationMethod": "Previously satisfied", "authenticationStepDateTime": "2021-07-01T15:12:53.9223384+00:00", "authenticationStepRequirement": "Primary authentication", "authenticationStepResultDetail": "First factor requirement satisfied by claim in the token", "succeeded": true}, {"authenticationMethod": "Previously satisfied", "authenticationStepDateTime": "2021-07-01T15:12:53.9223384+00:00", "authenticationStepRequirement": "Multi-factor authentication", "authenticationStepResultDetail": "MFA requirement satisfied by claim in the token", "succeeded": true}], "authenticationProcessingDetails": [{"key": "Login Hint Present", "value": "True"}, {"key": "IsCAEToken", "value": "False"}], "authenticationRequirement": "multiFactorAuthentication", "authenticationRequirementPolicies": [{"detail": "Conditional Access", "requirementProvider": "multiConditionalAccess"}], "clientAppUsed": "Browser", "conditionalAccessStatus": "success", "correlationId": "d50c2db2-0e37-48e8-b391-00ccf9273d96", "createdDateTime": "2021-07-01T15:12:53.9223384+00:00", "deviceDetail": {"browser": "Chrome 91.0.4472", "deviceId": "", "operatingSystem": "Windows 10"}, "flaggedForReview": false, "homeTenantId": "eef41338-8caf-4011-87bf-ff1c9c425f9e", "id": "9481698a-58dd-4048-9873-81145cc17701", "ipAddress": "200.3.98.250", "isInteractive": true, "isTenantRestricted": false, "location": {"city": "Buenos Aires", "countryOrRegion": "AR", "geoCoordinates": {"latitude": -34.611778259277344, "longitude": -58.41730880737305}, "state": "Ciudad De Buenos Aires"}, "mfaDetail": {}, "networkLocationDetails": [], "originalRequestId": "9481698a-58dd-4048-9873-81145cc17701", "processingTimeInMilliseconds": 140, "resourceDisplayName": "Microsoft Teams Chat Aggregator", "resourceId": "b1379a75-ce5e-4fa3-80c6-89bb39bf646c", "resourceTenantId": "eef41338-8caf-4011-87bf-ff1c9c425f9e", "riskDetail": "none", "riskEventTypes": [], "riskEventTypes_v2": [], "riskLevelAggregated": "none", "riskLevelDuringSignIn": "none", "riskState": "none", "servicePrincipalId": "", "status": {"additionalDetails": "MFA requirement satisfied by claim in the token", "errorCode": 0}, "tokenIssuerName": "", "tokenIssuerType": "AzureAD", "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36", "userDisplayName": "Juan Jose Ferreyra", "userId": "49307224-bc45-439e-a441-7db7e37722a6", "userPrincipalName": "**personal information omitted**", "userType": "Member"}, "resourceId": "/tenants/eef41338-8caf-4011-87bf-ff1c9c425f9e/providers/Microsoft.aadiam", "resultSignature": "None", "resultType": "0", "tenantId": "eef41338-8caf-4011-87bf-ff1c9c425f9e", "time": "2021-07-01T15:12:53.9223384Z"}

lratcliffe
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 7 of 7

Re: ESM Parsing Regular expression

This is a very old thread and your request for help may not get the appropriate attention.  Please start your own thread with your question.

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?
You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community