cancel
Showing results for 
Search instead for 
Did you mean: 

SIEM upgrade to v11.0.1 issue

getting desperate, upgrade from 10.3 to 11 and getting this:

McAfee IPSDBServer[2121]: Error -195 trying to produce. 127.0.0.1:9093/bootstrap: Connect to ipv4#127.0.0.1:9093 failed: Connection refused.
Jun 8 14:16:38 McAfee IPSDBServer[2121]: Error -187 trying to produce. 1/1 brokers are down.

 

Cpservice fails to start 

service cpservice start
Starting firewall...
Use of uninitialized value $port in numeric gt (>) at /usr/lib/perl5/5.16.1/x86_64-linux-thread-multi/Nitro/Firewall.pm line 588. [ OK ]
Starting Networking ... [ OK ]
Stopping SSH Control Channel daemon [ OK ]
Starting SSH Control Channel daemon [ OK ]
Starting ESS cpservice [ OK ]

In a minute it stops:

Jun 8 14:21:43 McAfee bootlog: (cpservice 713) cpservice init called with usage: start
Jun 8 14:21:55 McAfee bootlog: (cpservice start) calling cpservicectl
Jun 8 14:21:55 McAfee bootlog: (cpservice start) cpservicectl daemonized with exit (0) and signal (0)
Jun 8 14:21:55 McAfee bootlog: Starting ESS cpservice [ OK ]
Jun 8 14:22:01 McAfee cpservicectl[1084]: Warn: httpd failed post startup test (delayed start?).
Jun 8 14:22:04 McAfee cpserviced[1427]: Job debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Job debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Device communication debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Job debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Device communication debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Job debugging enabled
Jun 8 14:22:04 McAfee cpserviced[1427]: Starting (Parent pid = 1084)
Jun 8 14:22:05 McAfee cpserviced[1427]: Calling libESSDB Initialize
Jun 8 14:22:07 McAfee cpservicectl[1084]: Notice: signal CHLD, 1427 reclaimed. Exit status: 217. Signal number: 0
Jun 8 14:22:07 McAfee cpservicectl[1084]: Info: Started new instance of cpserviced (PID 1427).
Jun 8 14:22:07 McAfee cpservicectl[1084]: Error: cpserviced not started successfully
Jun 8 14:22:09 McAfee cpservicectl[1084]: Info: NGCP timezone setting: 1
Jun 8 14:22:11 McAfee cpservicectl[1084]: warn: cpserviced not running: cpserviced failed checkProc
Jun 8 14:22:16 McAfee cpservicectl[1084]: Warn: httpd failed post startup test (delayed start?).
Jun 8 14:22:18 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:18 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:18 McAfee cpserviced[2035]: Device communication debugging enabled
Jun 8 14:22:18 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:19 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:19 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:19 McAfee cpserviced[2035]: Device communication debugging enabled
Jun 8 14:22:19 McAfee cpserviced[2035]: Job debugging enabled
Jun 8 14:22:19 McAfee cpserviced[2035]: Starting (Parent pid = 1084)
Jun 8 14:22:19 McAfee cpserviced[2035]: Calling libESSDB Initialize
Jun 8 14:22:21 McAfee cpservicectl[1084]: Notice: signal CHLD, 2035 reclaimed. Exit status: 217. Signal number: 0
Jun 8 14:22:21 McAfee cpservicectl[1084]: Warning: Restarted cpserviced, (PID 2035) count: 1
Jun 8 14:22:21 McAfee cpservicectl[1084]: Error: cpserviced not started successfully
Jun 8 14:22:23 McAfee cpservicectl[1084]: Info: NGCP timezone setting: 1
Jun 8 14:22:26 McAfee cpservicectl[1084]: warn: cpserviced not running: cpserviced failed checkProc
Jun 8 14:22:30 McAfee cpservicectl[1084]: Warn: httpd failed post startup test (delayed start?).
Jun 8 14:22:33 McAfee cpserviced[2689]: Job debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Job debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Device communication debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Job debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Device communication debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Job debugging enabled
Jun 8 14:22:33 McAfee cpserviced[2689]: Starting (Parent pid = 1084)
Jun 8 14:22:34 McAfee cpserviced[2689]: Calling libESSDB Initialize
Jun 8 14:22:36 McAfee cpservicectl[1084]: Notice: signal CHLD, 2689 reclaimed. Exit status: 217. Signal number: 0
Jun 8 14:22:36 McAfee cpservicectl[1084]: Warning: Restarted cpserviced, (PID 2689) count: 2
Jun 8 14:22:36 McAfee cpservicectl[1084]: Error: cpserviced not started successfully
Jun 8 14:22:38 McAfee cpservicectl[1084]: Info: NGCP timezone setting: 1
Jun 8 14:22:40 McAfee cpservicectl[1084]: warn: cpserviced not running: cpserviced failed checkProc

patch 1 2 and 3 are applied, same. 

NitroError.log doesn't show any errors...

Any ideas would be much appreciated, this is killing us

5 Replies
akerr
Level 9
Report Inappropriate Content
Message 2 of 6

Re: SIEM upgrade to v11.0.1 issue

We're not on 11 yet (too many issues with 11), but from previous versions, I'd make sure cpservice is truly stopped.  I've seen issues where event though the log shows it has stopped, there is still a process running that you need to kill before cpservice can be started properly.

 

You don't need to worry about the bootsrap error.  I believe that has to do with clustering, but I haven't had a chance to play with it in the lab yet.

Re: SIEM upgrade to v11.0.1 issue

not the case i'm afraid, cpservice is fully restarted, box was restarted dozens of times......

when cpservice is restarted, it states this:

Use of uninitialized value $port in numeric gt (>) at /usr/lib/perl5/5.16.1/x86_64-linux-thread-multi/Nitro/Firewall.pm line 588. [ OK ]
Starting Networking ... [ OK ]
Stopping SSH Control Channel daemon [ OK ]
Starting ESS cpservice [ OK ]

Now, in that line there's perl code that is relating to snowflex config: 

$port = $snowclient_json->{'responsePort'};
if($port > 0)
{
writeFile("-A INPUT -m state --state NEW -p tcp -s $dc->{'host'} --dport $port -j SNOWFLEX", $fh);
}

now also getting this in the messages:

Jun 12 12:36:52 McAfee IPSDBServer[2100]: Error -195 trying to produce. 127.0.0.1:9093/bootstrap: Connect to ipv4#127.0.0.1:9093 failed: Connection refused.
Jun 12 12:36:52 McAfee IPSDBServer[2100]: Error -187 trying to produce. 1/1 brokers are down.Jun 12 12:36:52 McAfee snowflexctl[29413]: starting snowman
Jun 12 12:36:52 McAfee snowflexctl[29413]: starting nserver
Jun 12 12:36:52 McAfee snowflexctl[29413]: starting snowflex

for some reason /etc/snowflex/snowflex.conf file is empty, eg. no ports are there, etc:

"name": "local",
"peerIP": "10.0.0.0",
"peerPort": 0,
"snowflexPort": 0,
"snowmanPort": 0,
"snowfactoryPort": 0,
"cmlPort": 0,
"datacenter": "dc1",
"allowFirst": true,
"rack": "rk1",

"dflPath": "/db2/usr/local/ess/sf_data/ngcp.dfl",
"edbIP": "10.0.0.0",
"edbPort": 0,
"edbPortSecure": 0,
"dflPassword": "",
"edbBatchNumRecs": 500000,
"edbBatchNumSecs": 10,
"maxFDBConnectAttempts": 5,
"clearFDBUnhealthySec": 180,
"fdbConnTimeout": 20000,
"fdbReadTimeout": 10000,

Might be wrong, but the way I see it.....port is not difined and for that reason cpservice is not fully starting. (the only sort of clear error was about httpd failed). Now, firewall is managed by cpservice in return, so ports are getting listening to when the service is running to open them and listen to them. 

At the same time, how come that the upgrade wiped it, no idea. 

P.S. Suport call is running for 3 weeks now Smiley Sad

Re: SIEM upgrade to v11.0.1 issue

ended up reimaging the box

Highlighted

Re: SIEM upgrade to v11.0.1 issue

further update..........update from 11.0.1 to 11.0.3......similar issues! this time also kafka brokers:

 

McAfee libJobServer.so[2198]: KafkaConsumer failed to consume from kafka broker 127.0.0.1:9092 

 

This is becoming pathetic, seriously thinking of moving to splunk 

Re: SIEM upgrade to v11.0.1 issue

Had a similar experience when we upgraded to 11.0.1 - also ended up having to re-image the box after weeks of troubleshooting by McAfee. I share your sentiments, on top of the upgrades not working the SIEM also has very poor supporting for parsing basic Windows Events. I am also seriously considering moving to Splunk. 

Tags (1)