9 Replies Latest reply on Apr 3, 2014 4:09 AM by EPO-Janni

    Unable to start EPO WEB console after Server 2003 OS updating

    EPO-Janni

      Hi,

       

      I have the following problem on our EPO testing environment. I'm using EPO 4.6.6. build 176. I checked in the VSE 8.8 V4 patch/installer including the extensions and help. After that I updated the MS server 2003 OS regarding MS patches (including Frameworg updates). Then I performed a resart. After restarting the server the EPO console terminates with an error if I try to start. I checked the 3 McAfee services on the EPO server and they are up and running. I also cheked the DB-connection to SQL DB on that server using https://Server1:8443/core/config and all is OK. I also checked the WIndows application log on that server and found a "Apache Service error":

       

      Bild1.JPG

       

      I also checked the orion.log and found the following error:

       

      Bild2.JPG

      I also checked the server.log and found the errors below:

       

      20140327075009 I #06140 NAIMSRV  Initializing server...

      20140327075009 I #06140 NAIMSRV  Database initialization: Starting.

      20140327075009 I #06140 EPODAL   Microsoft SQL Server 2005 - 9.00.4053.00 (Intel X86)

      20140327075009 I #06140 EPODAL    May 26 2009 14:24:20

      20140327075009 I #06140 EPODAL    Copyright (c) 1988-2005 Microsoft Corporation

      20140327075009 I #06140 EPODAL    Express Edition on Windows NT 5.2 (Build 3790: Service Pack 2)

      20140327075009 I #06140 NAIMSRV  Database initialization: Succeeded.

      20140327075009 I #06140 NAIMSRV  Policy Manager initialization: Starting.

      20140327075009 I #06140 NAIMSRV  Policy Manager initialization: Succeeded.

      20140327075009 I #06140 NAIMSRV  Server state at startup: Enabled

      20140327075009 I #06140 NAIMSRV  Checking to see if the ePO server is available.  We will try 12 times.

      20140327075009 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075019 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075029 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075039 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075049 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075059 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075109 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075119 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075129 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075139 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075149 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075159 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075209 W #06140 NAIMSRV  The Agent Handler failed to connect to the ePO server.

      20140327075209 I #06140 NAIMSRV  Failed to connect to the ePO server.  The Agent Handler does not require the ePO server to be running, but it will exchange information if it is running.

      20140327075209 I #06140 MCUPLOAD Successfully disabled CA trust options.

      20140327075210 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075210 E #06140 NAIMSRV  Failed to connect to the ePO server.  Skipping certificate validation.

      20140327075210 I #06140 MCUPLOAD Successfully disabled CA trust options.

      20140327075210 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

      20140327075210 E #06140 NAIMSRV  Failed to connect to the ePO server.  Skipping agent handler certificate validation.

      20140327075210 I #06140 NAIMSRV  Unloading server private keys

      20140327075210 I #06140 NAIMSRV  ePolicy Orchestrator server started.

      20140327075210 I #06140 mod_eporepo Database initialization: Starting.

      20140327075210 I #06140 mod_eporepo Database initialization: Succeeded.

      20140327075210 I #00920 NAIMSRV  LdapSync: Status changed: Offline

       

      THe current Apache error log in \Apache2\Logs\ shows the following:

       

      [Thu Mar 27 07:31:49 2014] [notice] Parent: Received shutdown signal -- Shutting down the server.

      [Thu Mar 27 07:31:49 2014] [notice] Child 3484: Exit event signaled. Child process is ending.

      [Thu Mar 27 07:31:50 2014] [notice] Child 3484: Released the start mutex

      [Thu Mar 27 07:31:50 2014] [notice] Child 3484: All worker threads have exited.

      [Thu Mar 27 07:31:50 2014] [notice] Child 3484: Child process is exiting

      [Thu Mar 27 07:31:50 2014] [notice] Parent: Child process exited successfully.

      [Thu Mar 27 07:31:51 2014] [notice] Apache/2.2.22 (Win32) mod_ssl/2.2.22 OpenSSL/1.0.1e configured -- resuming normal operations

      [Thu Mar 27 07:31:51 2014] [notice] Server built: Mar 15 2013 14:10:51

      [Thu Mar 27 07:31:51 2014] [notice] Parent: Created child process 6036

      [Thu Mar 27 07:31:51 2014] [notice] Disabled use of AcceptEx() WinSock2 API

      [Thu Mar 27 07:31:52 2014] [notice] Child 6036: Child process is running

      [Thu Mar 27 07:33:53 2014] [notice] Child 6036: Acquired the start mutex.

      [Thu Mar 27 07:33:53 2014] [notice] Child 6036: Starting 250 worker threads.

      [Thu Mar 27 07:33:53 2014] [notice] Child 6036: Listening on port 443.

      [Thu Mar 27 07:33:53 2014] [notice] Child 6036: Listening on port 81.

      [Thu Mar 27 07:50:06 2014] [notice] Parent: Received shutdown signal -- Shutting down the server.

      [Thu Mar 27 07:50:06 2014] [notice] Child 6036: Exit event signaled. Child process is ending.

      [Thu Mar 27 07:50:07 2014] [notice] Child 6036: Released the start mutex

      [Thu Mar 27 07:50:07 2014] [notice] Child 6036: All worker threads have exited.

      [Thu Mar 27 07:50:07 2014] [notice] Child 6036: Child process is exiting

      [Thu Mar 27 07:50:07 2014] [notice] Parent: Child process exited successfully.

      [Thu Mar 27 07:50:08 2014] [notice] Apache/2.2.22 (Win32) mod_ssl/2.2.22 OpenSSL/1.0.1e configured -- resuming normal operations

      [Thu Mar 27 07:50:08 2014] [notice] Server built: Mar 15 2013 14:10:51

      [Thu Mar 27 07:50:08 2014] [notice] Parent: Created child process 3716

      [Thu Mar 27 07:50:08 2014] [notice] Disabled use of AcceptEx() WinSock2 API

      [Thu Mar 27 07:50:09 2014] [notice] Child 3716: Child process is running

      [Thu Mar 27 07:52:10 2014] [notice] Child 3716: Acquired the start mutex.

      [Thu Mar 27 07:52:10 2014] [notice] Child 3716: Starting 250 worker threads.

      [Thu Mar 27 07:52:10 2014] [notice] Child 3716: Listening on port 443.

      [Thu Mar 27 07:52:10 2014] [notice] Child 3716: Listening on port 81.

       

       

      What can I do next to check or repair something? Is it usefiul to reinstall/repair the EPO performing a reinstallation withe the  SETUP.EXE of the installer package EPO466L? To reinstall the EPO again on that server will be the last option because that is our testing environment. But if possible I'll try to fix that problem.

       

      Thanks for feedback.

       

      Best regards from Germany

       

      Janni

        • 1. Re: Unable to start EPO WEB console after Server 2003 OS updating
          hem

          Do you have certs present in \epo\apache2\conf\ssl.crt? If not then please recreate it as per KB66616 step 10

           

          20140327075009 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075019 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075029 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075039 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075049 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075059 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075109 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075119 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075129 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075139 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075149 E #06140 MCUPLOAD Failed to process the secure communication request (error=401)

          20140327075159 E #06140 MCUPLOAD Failed to process the secure communication request (error=401

           

          If certs are fine and still facing the issue then screenshot message or Event viewer talks about port conflict.

          1 of 1 people found this helpful
          • 2. Re: Unable to start EPO WEB console after Server 2003 OS updating
            EPO-Janni

            Hi Hem,

             

            thanks for fast answering. The following files are stored in that folder:

             

            Bild3.JPG

            The IE shows the following error running the EPO web console on that server:

             

            Bild4.JPG

             

             

            There are no other error messages in Event viewer application log except the Apache error posted above. But I checked the server.log again and found an additional error ID 12029 after restarting the server:

             

            ...

            20140326141656 I #03816 NAIMSRV  Checking to see if the ePO server is available.  We will try 12 times.

            20140326141658 E #03816 MCUPLOAD Failed to send http request

            20140326141658 E #03816 MCUPLOAD Failed to process the secure communication request (error=12029)

            20140326141709 E #03816 MCUPLOAD Failed to send http request

            20140326141709 E #03816 MCUPLOAD Failed to process the secure communication request (error=12029)

            20140326141721 E #03816 MCUPLOAD Failed to process the secure communication request (error=401)

            20140326141731 E #03816 MCUPLOAD Failed to process the secure communication request (error=401)

            ...

             

            What can I ceck or do next to analyze or fix that problem?

             

            Best regards

             

            Janni

            • 3. Re: Unable to start EPO WEB console after Server 2003 OS updating
              EPO-Janni

              Hi,

               

              I additionaly performed the steeps as described in KB66616 step 10. I got an error and the ahsetup.log in new ssl.crt folder shows the following:

               

              20140327105429 I #04308 AHSETUP  Creating Agent Handler Certs.

              20140327105429 I #04308 AHSETUP  Checking to see if the ePO server is available.  We will try 5 times.

              20140327105429 E #04308 MCUPLOAD Failed to process the secure communication request (error=401)

              20140327105439 E #04308 MCUPLOAD Failed to process the secure communication request (error=401)

              20140327105449 E #04308 MCUPLOAD Failed to process the secure communication request (error=401)

              20140327105459 E #04308 MCUPLOAD Failed to process the secure communication request (error=401)

              20140327105510 E #04308 MCUPLOAD Failed to process the secure communication request (error=401)

              20140327105520 W #04308 AHSETUP  The Agent Handler failed to connect to the ePO server.

              20140327105520 E #04308 AHSETUP  Failed to connect to the ePO server 'server1:8443'

               

              There were no other files in ssl.crt folder created except ahsetup.log. Then I renamed the ssl.crt folder back.

               

              I also checked the EPO agent connection. The EPO agent can communicate withe the EPO server without problems and DAT signature update is also working fine.

               

              But I can't open the EPO console anymore :-(.

               

              Best regards

               

              Janni

              • 4. Re: Unable to start EPO WEB console after Server 2003 OS updating
                JoeBidgood

                These errors imply that the server service can't talk to Tomcat. Is the ePO Application Server service running?

                If it is, restart the service, wait for a couple of minutes, then attach the orion.log here and we can have  a look.

                 

                HTH -

                 

                Joe

                1 of 1 people found this helpful
                • 5. Re: Unable to start EPO WEB console after Server 2003 OS updating
                  EPO-Janni

                  Hi Joe,

                   

                  yes the ePO Application Server service is running but it takes some minutes after restarting until the progress bar disappear.

                   

                  Bild5.JPG

                   

                  I restarted the ePO Application Server service, copied the orion.log after a couple of minutes and attached it here.

                   

                  I found out anoter point. As I wrote the ePO agent communication with the ePO server is working. BUT it seems that the newest DAT files of the last two days were not downloaded from ePO. The latest DAT version in VSE on the ePO is 7388.0000. The latest action on the ePO two days ago before restart  was that I checked in the VSE 8.8 patch 4 incl. extensions and also MS update patching. But of course at first the ePO console should open to check what's going on.

                   

                  Best regards

                   

                  Janni.

                  • 6. Re: Unable to start EPO WEB console after Server 2003 OS updating
                    hem

                    Orion log syas that VSE 8.8 help extension is in partially installed state.

                     

                    org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'core.ext.taskGlobals' defined in URL [jndi:/localhost/core/WEB-INF/beans.xml]: Invocation of init method failed; nested exception is java.lang.IllegalStateException: The following extensions are in a partially installed state: [vse_880_help:]

                    Caused by: java.lang.IllegalStateException: The following extensions are in a partially installed state: [vse_880_help:]

                     

                    Make sure that VSE 8.8 help extension is removed from all the three places \server\extensions\installed, \server\conf\catalina\localhost and from DB (from OrionExtensions and OrionExtensionBackup) table.

                     

                    Once partially installed extension is cleaned then please restart the ePO services and try to launch the console

                    1 of 1 people found this helpful
                    • 7. Re: Unable to start EPO WEB console after Server 2003 OS updating
                      EPO-Janni

                      Hi Hem,

                       

                      that was the right hint! You are really great guys regarding supporting McAfee here!!! I did the following. I only removed vse_880_help in \server\extensions\installed\ . In  \server\conf\catalina\localhost\ I doesn't find any XML file regarting vse_880_help. And regarding DB I don't know how to search or delete somewhat. But after performing the first step and restarting the ePO services the WEB console is working now *happy*.

                       

                      BTW there was an error checking in the VSE patch 4 help. Now I have the following VSE help in EPO help extensions:

                       

                      Bild6.JPG

                       

                      If I try to check in the new VSE patch 4 help extension that come with VSE patch 4 installer package I get the following message:

                       

                      Bild7.JPG

                       

                      Two days before I clicked on OK (now I aborted that task). Then I found two VSE help extensions two days before in ePO help extension section. I could understood that bacause normally the old extension will be overwritten from the new one. But in that case of VSE help was showing with "vse_880_help" and another one with "vse_help" shown in the screenshot above. Then I tried to remove the older one. But after that the VSE help wasn't available anymore. Then I tried to remove also the new VSE help extension "vse_880_help" to check it in again. After removing and trying to re-checkin I got an error that someone is particularry wrong with checkin process. Then I performed a restart after Windows updating.

                       

                      Now I'm afraid to check in that new VSE patch 4 help extension again. The VSE patch 4 reporting and policy extensions with new versions could I check in without any problems and they overwrote the older ones. What do you recommand regarding VSE patch 4 help extension?

                       

                      Best regards

                       

                      Janni

                      • 8. Re: Unable to start EPO WEB console after Server 2003 OS updating
                        EPO-Janni

                        Hi guys,

                         

                        no additional hints regarding VSE patch 4 help check in?

                         

                        Regards Janni

                        • 9. Re: Unable to start EPO WEB console after Server 2003 OS updating
                          EPO-Janni

                          Hi,

                           

                          is there no additional hint available for VSE_880_help patch4 check-in?

                           

                          Regards Janni