ePO upgrade 5.9.1 to 5.10 fails - wipes out directory - application server will not stay running

[cstewart28]

I'm trying to upgrade from 5.9.1 to 5.10, but the installed failed and in the EPO5100-Install-MSI.LOG file I see: 

Property(C): IDS_1024BitKeyError = Setup is unable to connect to the SQL Server "%s" over a secure connection. To rectify this situation, upgrade your SSL/TLS certificate on the "%s" system and retry. See KB 87731 for further information.
Property(C): IDS_PrivateKeyLengthError = This server cannot be upgraded due to insufficient private key length for the root CA certificate. Please contact McAfee ePO Support.
Property(C): IDS_TLSError = An attempt to establish a test connection to the SQL Server "%s" failed. The SQL Server used by McAfee ePO must support a secure communication with TLS 1.1 or 1.2. Please see KB 90222 for more information.

I created a new cert request, but when I went to start the services back the 5.9.1 Application Server failed to start and not of the other services started.  After looking around it seems the upgrade put all the files in ~epo directory. 

I did a directory restore from a valid backup file because I did not know what it was trying to do, so once the items were restored, I rebooted the server and now the services will start, but the Application Server will run for a while and then stop. 

I need to get the system back up so I can access the console.   

[cdinet]

You probably need to do a disaster recovery.  If the rollback failed, there is no way to know what else didn't roll back.  KB66616 or product guide for snapshot recovery.  

When you look at install logs, disregard any property entries - they are only cataloging potential errors.  Once you do a disaster recovery, you might want to open a ticket and share the install logs so we can possibly see what went wrong.  

[Hem]

I had one case where all ePO binaries will be dump under ~epo folder. Do you see them there?

Was my reply helpful?

If this information was helpful in any way or answered your question, will you please select Accept as Solution in my reply and together we can help other members?

[cstewart28]

Yes I do have a ~epo directory

[Hem]

Do you see binaries there?

[cstewart28]

The Server Service and Event Parser Service are running, How can I figure out why the Application Service is not staying running?

[cdinet · Accepted solution]

Check the orion log.  Most common cause when directories have been restored are mismatched extensions.  The file system must match what is in the database.  Look at KB81737 as an example.

[cstewart28]

The orion log files hows:

2020-09-02 09:40:32,830 ERROR [localhost-startStop-1] ext.RevokedExtensionUtil - Cannot load the specified revoked extension list file.
2020-09-02 09:40:34,627 ERROR [main] core.OrionCore - Detected a possible mismatch between the server and database. Try pointing the server to the correct database (ie, db versions of extensions must match what is on the filesystem)
java.lang.IllegalStateException: installed version of 'ENDP_FW_META' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_FW_META\10.7.0.541 does not match OrionExtensions DB table entry: ENDP_FW_META:10.7.0.601
installed version of 'ENDP_WP_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_WP_1000\10.7.0.550 does not match OrionExtensions DB table entry: ENDP_WP_1000:10.7.0.614
installed version of 'ENDP_ADR' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_ADR\1.0.0.371 does not match OrionExtensions DB table entry: ENDP_ADR:1.0.0.432
installed version of 'ENDP_GS_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_GS_1000\10.7.0.566 does not match OrionExtensions DB table entry: ENDP_GS_1000:10.7.0.621
installed version of 'ENDP_AM_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_AM_1000\10.7.0.585 does not match OrionExtensions DB table entry: ENDP_AM_1000:10.7.0.657

at com.mcafee.orion.core.OrionCore.verifyServerAndDatabaseExtensionCompatibility(OrionCore.java:998)
at com.mcafee.orion.core.OrionCore.afterStart(OrionCore.java:824)
at com.mcafee.orion.core.server.OrionLifecycleListener.lifecycleEvent(OrionLifecycleListener.java:108)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:388)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:155)
at org.apache.catalina.startup.Catalina.start(Catalina.java:694)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:294)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:428)
2020-09-02 09:40:34,643 FATAL [main] core.OrionCore - killing server. reason = Detected a possible mismatch between the server and database. Try pointing the server to the correct database (ie, db versions of extensions must match what is on the filesystem)

[vivs]

Hello @cstewart28 

Thanks for your response.

2020-09-02 09:40:34,627 ERROR [main] core.OrionCore - Detected a possible mismatch between the server and database. Try pointing the server to the correct database (ie, db versions of extensions must match what is on the filesystem)
java.lang.IllegalStateException: installed version of 'ENDP_FW_META' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_FW_META\10.7.0.541 does not match OrionExtensions DB table entry: ENDP_FW_META:10.7.0.601
installed version of 'ENDP_WP_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_WP_1000\10.7.0.550 does not match OrionExtensions DB table entry: ENDP_WP_1000:10.7.0.614
installed version of 'ENDP_ADR' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_ADR\1.0.0.371 does not match OrionExtensions DB table entry: ENDP_ADR:1.0.0.432
installed version of 'ENDP_GS_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_GS_1000\10.7.0.566 does not match OrionExtensions DB table entry: ENDP_GS_1000:10.7.0.621
installed version of 'ENDP_AM_1000' extension at D:\Program Files (x86)\McAfee\ePolicy Orchestrator\Server\extensions\installed\ENDP_AM_1000\10.7.0.585 does not match OrionExtensions DB table entry: ENDP_AM_1000:10.7.0.657

There is a extensuon mismatch issue as suggested please check the below KB article:

https://kc.mcafee.com/corporate/index?page=content&id=KB81737 

Was my reply helpful?
If you find this post useful, Please give it a Kudos! Also, Please don't forget to select "Accept as a solution" if this reply resolves your query!

[cstewart28]

In the Apache2\Logs\errorlog...\ I see 

[Wed Sep 02 09:25:03.643110 2020] [mpm_winnt:notice] [pid 6732:tid 428] AH00455: Apache/2.4.28 (Win32) OpenSSL/1.0.2l configured -- resuming normal operations
[Wed Sep 02 09:25:03.643110 2020] [mpm_winnt:notice] [pid 6732:tid 428] AH00456: Server built: Oct 6 2017 12:14:17
[Wed Sep 02 09:25:03.643110 2020] [core:notice] [pid 6732:tid 428] AH00094: Command line: 'D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2\\bin\\Apache.exe -d D:/Program Files (x86)/McAfee/ePolicy Orchestrator/Apache2 -d D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2 -f D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2\\conf\\httpd.conf'
[Wed Sep 02 09:25:03.643110 2020] [mpm_winnt:notice] [pid 6732:tid 428] AH00418: Parent: Created child process 4260
[Wed Sep 02 09:25:05.408742 2020] [ssl:warn] [pid 4260:tid 320] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]
[Wed Sep 02 09:28:24.856070 2020] [core:warn] [pid 2960:tid 436] AH00098: pid file D:/Program Files (x86)/McAfee/ePolicy Orchestrator/Apache2/Logs/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Wed Sep 02 09:28:25.137317 2020] [ssl:warn] [pid 2960:tid 436] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]
[Wed Sep 02 09:28:25.137317 2020] [mpm_winnt:notice] [pid 2960:tid 436] AH00455: Apache/2.4.28 (Win32) OpenSSL/1.0.2l configured -- resuming normal operations
[Wed Sep 02 09:28:25.137317 2020] [mpm_winnt:notice] [pid 2960:tid 436] AH00456: Server built: Oct 6 2017 12:14:17
[Wed Sep 02 09:28:25.137317 2020] [core:notice] [pid 2960:tid 436] AH00094: Command line: 'D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2\\bin\\Apache.exe -d D:/Program Files (x86)/McAfee/ePolicy Orchestrator/Apache2 -d D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2 -f D:\\Program Files (x86)\\McAfee\\ePolicy Orchestrator\\Apache2\\conf\\httpd.conf'
[Wed Sep 02 09:28:25.137317 2020] [mpm_winnt:notice] [pid 2960:tid 436] AH00418: Parent: Created child process 3196
[Wed Sep 02 09:28:25.996697 2020] [ssl:warn] [pid 3196:tid 324] AH01873: Init: Session Cache is not configured [hint: SSLSessionCache]
[Wed Sep 02 09:34:29.667294 2020] [mpm_winnt:notice] [pid 3196:tid 324] AH00354: Child: Starting 250 worker threads.
[Wed Sep 02 10:05:30.479445 2020] [authz_core:error] [pid 3196:tid 3704] [client X.X.179.11:61041] AH01630: client denied by server configuration: D:/Program Files (x86)/McAfee/ePolicy Orchestrator/DB/www/tools
[Wed Sep 02 10:05:30.876647 2020] [authz_core:error] [pid 3196:tid 3704] [client 147.241.179.11:61041] AH01630: client denied by server configuration: D:/Program Files (x86)/McAfee/ePolicy Orchestrator/DB/www/tools

Seems like Apache is getting restarted several times.