cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
erez
Level 8
Report Inappropriate Content
Message 11 of 23

Re: Forward Events

Hey Guys,

So the answer I got from the SR is that it seems that there is a problem with the intermittent SSL with my Syslog server.

I tried looking for an example in order to fix this and I found KB87927.

I went over it and it seems more or less what I did (I followed this guide - https://docs.logz.io/shipping/log-sources/mcafee-epolicy-orchestrator.html)

So I have no idea how to debug this.. since the syslog testing works.

Would appreciate for some direction.. Thanks!

 

cdinet
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 12 of 23

Re: Forward Events

When issue is occurring, it might be good to get a wireshark capture to see if there are any network issues.  Do the syslog server logs show any errors?

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

erez
Level 8
Report Inappropriate Content
Message 13 of 23

Re: Forward Events

Hey,

So after troubleshooting this for quite a while (since my last post, almost every day) and I finally was able to get the following errors from both sides (which was sort of finding a needle in a haystack).

On the Epo I got these errors:

E #06376 MFEFIPS mfefips_SSLSubSys.cpp(389): Error writing 3522 SSL bytes to xxxxx:6514 (-1 returned)
E #06376 EVNTPRSR source\SyslogForwarder.cpp(138): Failed to send data to syslog receiver: xxxx:6514
W #06376 MFEFIPS Ignoring host xxxxx:6514 for 2 minutes

On the Syslog side, I got those:

DEBUG [tcp] common/listener.go:162 client error {"address": "0.0.0.0:6514", "error": "read tcp xxxxx:6514->xxxxx:56585: i/o timeout"}
DEBUG [tcp] common/listener.go:170 client disconnected {"address": "0.0.0.0:6514", "remote_address": "xxxxx:56585", "total": 3}
DEBUG [tcp] common/listener.go:162 client error {"address": "0.0.0.0:6514", "error": "read tcp xxxxx:6514->xxxxxx:56361: i/o timeout"}
DEBUG [tcp] common/listener.go:170 client disconnected {"address": "0.0.0.0:6514", "remote_address": "xxxxxx:56361", "total": 2}.

Running Wireshark showed at the same time of the error an "Encrypted Alert" message and following that a "FIN" flag,

similar to what is happening in this thread I found:

https://osqa-ask.wireshark.org/questions/45195/what-is-the-reason-behind-a-client-sending-encrypted-....

To be honest, I have no idea how to continue, and the guy from the SR told me in our conversation that there is no need for a TLS certificate? which is absolutely not true.. so I have no idea what to do.

 

cdinet
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 14 of 23

Re: Forward Events

A certificate is not required, however, certain ciphers/tls is required.  What you might want to look at in the wireshark is when there is a client hello, it sends certain ciphers.  In the server hello response, it responds with a cipher (syslog) - that must match one of the ones sent by the client (epo).  You can run nmap against the syslog server to ensure tls 1.2 is enabled.  kb91115 tells how to run that - it can be run from any system as long as you point it to the syslog server on its port.  KB91194 lists tls requirements for syslog.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

erez
Level 8
Report Inappropriate Content
Message 15 of 23

Re: Forward Events

Hey,

I'm not sure how you can set up a TLS session without using some sort of certificate and a key (tls session?)

In Wireshark I see that it's using TLS 1.2 with Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384.

Basically, it's almost the same setup that there is over here

https://kc.mcafee.com/corporate/index?page=content&id=KB87927

Something that I don't understand is if there is a problem with the TLS, shouldn't it be not working at all?

From what I see, I can't really see any problems since I'm getting events into the syslog, and because I can't examine what I'm not getting so I'm not sure what is this issue causing..

cdinet
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 16 of 23

Re: Forward Events

A tls connection does not necessarily require a certificate exchange, but it does require negotiating with common ciphers.  If a certificate is presented, it is only to verify the identity of the cert sender and would just need to be trusted in the receiving server.  The wireshark would show if there is any certificate exchange or not.  Since it works intermittently, the wireshark would be needed when it is failing, or you might want to get a capture for when it is working and when it is failing to compare the two.  One thing you also want to look at is in the capture, look at the mac addresses of the epo and syslog servers.  Are the macs shown the actual macs of the system or is there a proxy or firewall that it is passing through?  The mac would show possibly belonging to a cisco or palo alto firewall device rather than the actual mac of the source or destination.  If you see that, you might want to check logs on that device for when traffic fails to see if there is anything relevant.  Since things work sometimes and sometimes not, I would suspect more some network involvement.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

erez
Level 8
Report Inappropriate Content
Message 17 of 23

Re: Forward Events

Hey,

I'm not sure if I understood you correctly, because that how you make it sound is that I could make it work with only TCP without TLS at all, which is not happening.

Or using TLS without anything else that makes the connection a TLS connection which is from what I know for years that is based on the certificate. Also when negotiating common ciphers, the server sends a digital certificate to verify its identity to the client. The server may also request a client’s digital certification if needed, I'm willing to listen to the explanation for how to make a TLS exchange without a certificate 🙂

And the Wireshark does show the certificate exchange.

Regarding the capture, I have the capture of when it's working and then dropping, but it's basically What I said in the earlier post, that is shows error an "Encrypted Alert" message and following that a "FIN" flag.

Regarding the network involvement, happily for me those are 2 EC2 instances on AWS, that are talking directly so I could for sure take down the possibility that it's a network issue. 

cdinet
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 18 of 23

Re: Forward Events

ePO will only forward events via tls, so no, you can't use just tcp without tls.  As for how to use tls without a cert, see link below.  Is it recommended?  No, it is not, but yes, it is possible.  Our kb for setting up a test syslog server even instructs to create a cert on the syslog server.  It says nothing about importing it to epo.  The cert is for when the connection is made, the certificate validates that the server is who it says it is.  

Does your syslog server have any logging?  If not, I would suggest enabling logging if it isn't enabled by default. Is there any way you can get a capture from both sides at the same time (epo and syslog server) to see exactly what is going on?  It may show something unexpected where exactly the fin is coming from.

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

erez
Level 8
Report Inappropriate Content
Message 19 of 23

Re: Forward Events

Can you resend the link? I can't see it in the post, 

Regarding the logging, do you mean to take a tcpdump? Since that the logging of the service (filebeat) is what I wrote in my earlier post:

DEBUG [tcp] common/listener.go:162 client error {"address": "0.0.0.0:6514", "error": "read tcp xxxxx:6514->xxxxx:56585: i/o timeout"}

DEBUG [tcp] common/listener.go:170 client disconnected {"address": "0.0.0.0:6514", "remote_address": "xxxxx:56585", "total": 3}

DEBUG [tcp] common/listener.go:162 client error {"address": "0.0.0.0:6514", "error": "read tcp xxxxx:6514->xxxxxx:56361: i/o timeout"}

DEBUG [tcp] common/listener.go:170 client disconnected {"address": "0.0.0.0:6514", "remote_address": "xxxxxx:56361", "total":  2}.

If I upload the pcap file would it help?

It seems strange that I'm the only one that have this kind of issue. 

cdinet
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 20 of 23

Re: Forward Events

Tcpdump is what I mean by capture from syslog.  The logging entries you posted seem to point to timeouts.  Does that match any eventparser log entries from epo server?  

No, I do not recommend posting any captures here on a public forum, as they contain sensitive company information.  

Match your debug errors that you see, such as DEBUG [tcp] common/listener.go:162 client error {"address": "0.0.0.0:6514", "error": "read tcp xxxxx:6514->xxxxxx:56361: i/o timeout"}, with the eventparser logs.  You should see similar errors in that log.  If so, that still points to possible network issues.  

Sorry, I must have forgotten to paste the link.

https://security.stackexchange.com/questions/73244/can-we-have-https-without-certificates

 

Was my reply helpful?
If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community