5 Replies Latest reply on Nov 25, 2009 9:40 AM by MarcoKling

    Crash EPO Application Server (4.0.0 Patch 5)

      Hello,

       

      we are using here ePO 4.0.0 (Patch 5). Updatet from Patch 4 to 5 last week. Today the ePO Application Server crash with following error message in EpoApSvr.log:

       

      20091123072947    I    #6096    EPOJNI    Getting license data...
      20091123073014    E    #6096    EPOJNI    Caught win32 exception in processing encrypt-encode string: Exception caught...Dumping info
      20091123073014    E    #6096    EPOJNI              code: 0xc0000005
      20091123073014    E    #6096    EPOJNI              addr: 0x6dc2a3bc
      20091123073014    E    #6096    EPOJNI              EAX = 00000000 EBX = 00000000
      20091123073014    E    #6096    EPOJNI              ECX = 00000000 EDX = 2d27ed58
      20091123073014    E    #6096    EPOJNI              ESI = 2c5fc8f4 EDI = 2c5fc800
      20091123073014    E    #6096    EPOJNI              EIP = 6dc2a3bc ESP = 2d27ecac
      20091123073014    E    #6096    EPOJNI              EBP = 00000000 EFL = 00010246
      20091123073014    E    #6096    EPOJNI   
      20091123073014    E    #6096    EPOJNI    The instruction at 0x6dc2a3bc referenced memory at 0x00000000.  The memory could not be Read
      20091123073014    E    #6096    EPOJNI   
      20091123073014    E    #6096    EPOJNI    Dumping stack with module mappings
      20091123073014    E    #6096    EPOJNI   
      20091123073014    E    #6096    EPOJNI    List of modules in application address space:
      20091123073014    E    #6096    EPOJNI       tomcat5.exe      base: 0x00400000, size: 0x0000e000
      20091123073014    E    #6096    EPOJNI       ntdll.dll        base: 0x7c920000, size: 0x000c8000
      20091123073014    E    #6096    EPOJNI       kernel32.dll     base: 0x7c800000, size: 0x00115000
      20091123073014    E    #6096    EPOJNI       USER32.dll       base: 0x77e20000, size: 0x00092000
      20091123073014    E    #6096    EPOJNI       GDI32.dll        base: 0x77bd0000, size: 0x00049000
      20091123073014    E    #6096    EPOJNI       ADVAPI32.dll     base: 0x77f30000, size: 0x000ac000
      20091123073014    E    #6096    EPOJNI       RPCRT4.dll       base: 0x77c20000, size: 0x0009f000
      20091123073014    E    #6096    EPOJNI       Secur32.dll      base: 0x76e40000, size: 0x00013000
      20091123073014    E    #6096    EPOJNI       MSVCRT.dll       base: 0x77b70000, size: 0x0005a000
      20091123073014    E    #6096    EPOJNI       SHELL32.dll      base: 0x7c9f0000, size: 0x00809000
      20091123073014    E    #6096    EPOJNI       SHLWAPI.dll      base: 0x77ec0000, size: 0x00052000
      20091123073014    E    #6096    EPOJNI       comctl32.dll     base: 0x77340000, size: 0x00103000
      20091123073014    E    #6096    EPOJNI       jvm.dll          base: 0x6dac0000, size: 0x003ba000
      20091123073014    E    #6096    EPOJNI       WINMM.dll        base: 0x76990000, size: 0x0002e000
      20091123073014    E    #6096    EPOJNI       MSVCR71.dll      base: 0x7c340000, size: 0x00056000
      20091123073014    E    #6096    EPOJNI       hpi.dll          base: 0x6d320000, size: 0x00008000
      20091123073014    E    #6096    EPOJNI       PSAPI.DLL        base: 0x76a60000, size: 0x0000b000
      20091123073014    E    #6096    EPOJNI       verify.dll       base: 0x6d820000, size: 0x0000c000
      20091123073014    E    #6096    EPOJNI       java.dll         base: 0x6d3c0000, size: 0x0001f000
      20091123073014    E    #6096    EPOJNI       zip.dll          base: 0x6d860000, size: 0x0000f000
      20091123073014    E    #6096    EPOJNI       rsaenh.dll       base: 0x68000000, size: 0x00035000
      20091123073014    E    #6096    EPOJNI       net.dll          base: 0x6d620000, size: 0x00013000
      20091123073014    E    #6096    EPOJNI       WS2_32.dll       base: 0x71a10000, size: 0x00017000
      20091123073014    E    #6096    EPOJNI       WS2HELP.dll      base: 0x71a00000, size: 0x00008000
      20091123073014    E    #6096    EPOJNI       mswsock.dll      base: 0x71930000, size: 0x00042000
      20091123073014    E    #6096    EPOJNI       DNSAPI.dll       base: 0x76dc0000, size: 0x0002b000
      20091123073014    E    #6096    EPOJNI       winrnr.dll       base: 0x76e60000, size: 0x00007000
      20091123073014    E    #6096    EPOJNI       WLDAP32.dll      base: 0x76e00000, size: 0x0002f000
      20091123073014    E    #6096    EPOJNI       rasadhlp.dll     base: 0x76e70000, size: 0x00005000
      20091123073014    E    #6096    EPOJNI       sunmscapi.dll    base: 0x6d800000, size: 0x00008000
      20091123073014    E    #6096    EPOJNI       CRYPT32.dll      base: 0x760a0000, size: 0x00095000
      20091123073014    E    #6096    EPOJNI       MSASN1.dll       base: 0x76080000, size: 0x00012000
      20091123073014    E    #6096    EPOJNI       hnetcfg.dll      base: 0x2cb60000, size: 0x0005b000
      20091123073014    E    #6096    EPOJNI       wshtcpip.dll     base: 0x718f0000, size: 0x00008000
      20091123073014    E    #6096    EPOJNI       WinAuth.dll      base: 0x2f040000, size: 0x0000f000
      20091123073014    E    #6096    EPOJNI       security.dll     base: 0x71d80000, size: 0x00004000
      20091123073014    E    #6096    EPOJNI       awt.dll          base: 0x6d0b0000, size: 0x0012e000
      20091123073014    E    #6096    EPOJNI       WINSPOOL.DRV     base: 0x72e90000, size: 0x00027000
      20091123073014    E    #6096    EPOJNI       IMM32.dll        base: 0x76180000, size: 0x0001d000
      20091123073014    E    #6096    EPOJNI       ole32.dll        base: 0x774f0000, size: 0x00139000
      20091123073014    E    #6096    EPOJNI       JniLoader.dll    base: 0x2f230000, size: 0x00010000
      20091123073014    E    #6096    EPOJNI       epojni.dll       base: 0x54200000, size: 0x00045000
      20091123073014    E    #6096    EPOJNI       NETAPI32.dll     base: 0x71a50000, size: 0x00057000
      20091123073014    E    #6096    EPOJNI       OLEAUT32.dll     base: 0x77cd0000, size: 0x0008b000
      20091123073014    E    #6096    EPOJNI       MSVCP71.dll      base: 0x7c3a0000, size: 0x0007b000
      20091123073014    E    #6096    EPOJNI       nailog.dll       base: 0x50400000, size: 0x00007000
      20091123073014    E    #6096    EPOJNI       nmcomn32.dll     base: 0x50e00000, size: 0x0001c000
      20091123073014    E    #6096    EPOJNI       MPR.dll          base: 0x719e0000, size: 0x00011000
      20091123073014    E    #6096    EPOJNI       VERSION.dll      base: 0x77b60000, size: 0x00008000
      20091123073014    E    #6096    EPOJNI       RManJNI.dll      base: 0x55400000, size: 0x0004c000
      20091123073014    E    #6096    EPOJNI       NAIZLB32.DLL     base: 0x50800000, size: 0x00017000
      20091123073014    E    #6096    EPOJNI       cryptocme2.dll   base: 0x2f290000, size: 0x0032e000
      20091123073014    E    #6096    EPOJNI       DAL.dll          base: 0x50000000, size: 0x00077000
      20091123073014    E    #6096    EPOJNI       PONTUtil.dll     base: 0x50a00000, size: 0x0000b000
      20091123073014    E    #6096    EPOJNI       SAMLIB.dll       base: 0x7e020000, size: 0x0000f000
      20091123073014    E    #6096    EPOJNI       CLBCatQ.DLL      base: 0x77630000, size: 0x00083000
      20091123073014    E    #6096    EPOJNI       COMRes.dll       base: 0x76f00000, size: 0x00193000
      20091123073014    E    #6096    EPOJNI       xpsp2res.dll     base: 0x2f5e0000, size: 0x002d9000
      20091123073014    E    #6096    EPOJNI       msado15.dll      base: 0x4bc70000, size: 0x00092000
      20091123073014    E    #6096    EPOJNI       MSDART.DLL       base: 0x74930000, size: 0x0001a000
      20091123073014    E    #6096    EPOJNI       oledb32.dll      base: 0x4c8e0000, size: 0x00079000
      20091123073014    E    #6096    EPOJNI       OLEDB32R.DLL     base: 0x4dbe0000, size: 0x00013000
      20091123073014    E    #6096    EPOJNI       comsvcs.dll      base: 0x4a1b0000, size: 0x00145000
      20091123073014    E    #6096    EPOJNI       sqloledb.dll     base: 0x4e260000, size: 0x00081000
      20091123073014    E    #6096    EPOJNI       MSDATL3.dll      base: 0x61310000, size: 0x00015000
      20091123073014    E    #6096    EPOJNI       MSWSTR10.DLL     base: 0x2fc10000, size: 0x00095000
      20091123073014    E    #6096    EPOJNI       DBNETLIB.DLL     base: 0x6d770000, size: 0x0001c000
      20091123073014    E    #6096    EPOJNI       WSOCK32.dll      base: 0x719c0000, size: 0x0000a000
      20091123073014    E    #6096    EPOJNI       msv1_0.dll       base: 0x76b80000, size: 0x00027000
      20091123073014    E    #6096    EPOJNI       iphlpapi.dll     base: 0x76be0000, size: 0x0001a000
      20091123073014    E    #6096    EPOJNI       ntdsapi.dll      base: 0x765e0000, size: 0x00014000
      20091123073014    E    #6096    EPOJNI       schannel.dll     base: 0x76640000, size: 0x00027000
      20091123073014    E    #6096    EPOJNI       USERENV.dll      base: 0x76810000, size: 0x000c4000
      20091123073014    E    #6096    EPOJNI       dssenh.dll       base: 0x68100000, size: 0x00027000
      20091123073014    E    #6096    EPOJNI       msadrh15.dll     base: 0x64210000, size: 0x0000e000
      20091123073014    E    #6096    EPOJNI       msxml3.dll       base: 0x72c70000, size: 0x00117000
      20091123073014    E    #6096    EPOJNI       msi.dll          base: 0x42c50000, size: 0x00441000
      20091123073014    E    #6096    EPOJNI       msxml4.dll       base: 0x69b10000, size: 0x0012f000
      20091123073014    E    #6096    EPOJNI       ApConfigJNI.dll  base: 0x32de0000, size: 0x00009000
      20091123073014    E    #6096    EPOJNI       LZ32.dll         base: 0x73bc0000, size: 0x00003000
      20091123073014    E    #6096    EPOJNI       msvcp60.dll      base: 0x32df0000, size: 0x00065000
      20091123073014    E    #6096    EPOJNI       nailite.dll      base: 0x32e60000, size: 0x000c1000
      20091123073014    E    #6096    EPOJNI       ddraw.dll        base: 0x73690000, size: 0x0004b000
      20091123073014    E    #6096    EPOJNI       DCIMAN32.dll     base: 0x739c0000, size: 0x00006000
      20091123073014    E    #6096    EPOJNI       fontmanager.dll  base: 0x6d2c0000, size: 0x00053000
      20091123073014    E    #6096    EPOJNI       dcpr.dll         base: 0x6d230000, size: 0x00023000
      20091123073014    E    #6096    EPOJNI       cmm.dll          base: 0x6d200000, size: 0x0002f000
      20091123073014    E    #6096    EPOJNI       jpeg.dll         base: 0x6d460000, size: 0x00024000
      20091123073014    E    #6096    EPOJNI       nio.dll          base: 0x6d640000, size: 0x00009000
      20091123073014    E    #6096    EPOJNI       DBGHELP.DLL      base: 0x33100000, size: 0x000a8000
      20091123073014    E    #6096    EPOJNI   
      20091123073014    E    #6096    EPOJNI  

       

       

      I can reproduce the crash if i made a change in  Configuration > Serverconfiguration > MyAvert Security Threats. This error appears the first time after the Update with Patch 5. Maybe a bug in Patch 5 ???

       

      Can anyone help me, please.

       

      Thanx

      Marco

        • 1. Re: Crash EPO Application Server (4.0.0 Patch 5)
          GWIRT

          Do you get this error when you simply go to that page or when you click edit? Do you have a proxy specified, and if so what kind of authentication is it using?

           

          I would also check the ..\Server\Logs\Orion.log at the same time to see if there is any corresponding messages to this crash.

          1 of 1 people found this helpful
          • 2. Re: Crash EPO Application Server (4.0.0 Patch 5)

            Hello,

             

            thanx for the fast reply...ePO Application Server crashes, when i click on edit at the "MyAvert Security Threats" configuration.

             

            We are using here Squid Proxy V2.6.5-6, running on Debian, using basic auth ("auth_param basic program /etc/webmin/squid/squid-auth.pl /etc/webmin/squid/users
            ")

             

            At the same time from the error above Orion.log says:

             

            2009-11-23 07:28:04,909 WARN  [Thread-1] email.EmailServiceConfig  - username was null. assuming no username or password
            2009-11-23 07:28:09,784 INFO  [pool-5-thread-1] auth.AuthChallengeProcessor  - basic authentication scheme selected
            2009-11-23 07:29:08,347 INFO  [Thread-1] core.StandardService  - Starting service Catalina
            2009-11-23 07:29:08,347 INFO  [Thread-1] core.StandardEngine  - Starting Servlet Engine: Apache Tomcat/5.5.25
            2009-11-23 07:29:08,378 INFO  [Thread-1] core.StandardHost  - XML validation disabled
            2009-11-23 07:29:12,394 INFO  [Thread-1] http11.Http11BaseProtocol  - Starting Coyote HTTP/1.1 on http-8443
            2009-11-23 07:29:12,690 INFO  [Thread-1] http11.Http11BaseProtocol  - Starting Coyote HTTP/1.1 on http-8444
            2009-11-23 07:29:12,769 INFO  [Thread-1] http11.Http11BaseProtocol  - Starting Coyote HTTP/1.1 on http-8445
            2009-11-23 07:29:12,815 INFO  [Thread-1] storeconfig.StoreLoader  - Find registry server-registry.xml at classpath resource
            2009-11-23 07:29:13,737 WARN  [Thread-1] email.EmailServiceConfig  - username was null. assuming no username or password
            2009-11-23 07:29:18,597 INFO  [pool-5-thread-1] auth.AuthChallengeProcessor  - basic authentication scheme selected
            2009-11-23 07:29:18,612 ERROR [pool-5-thread-1] cmd.CommandInvoker  - Error invoking command:AvertAlerts.Web.Service

            java.lang.StringIndexOutOfBoundsException: String index out of range: -1981052140
                at java.lang.String.getChars(String.java:853)
                at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:391)
                at java.lang.StringBuffer.append(StringBuffer.java:224)
                at org.apache.commons.httpclient.auth.BasicScheme.authenticate(BasicScheme.java:24 2)
                at org.apache.commons.httpclient.auth.BasicScheme.authenticate(BasicScheme.java:19 9)
                at org.apache.commons.httpclient.HttpMethodDirector.authenticateProxy(HttpMethodDi rector.java:319)
                at org.apache.commons.httpclient.HttpMethodDirector.authenticate(HttpMethodDirecto r.java:231)
                at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirect or.java:169)
                at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
                at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.sendSoapRequest(AvertAlert sClient.java:161)
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.getReport(AvertAlertsClien t.java:195)
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.getReportByName(AvertAlert sClient.java:229)
                at com.mcafee.epo.avertalerts.command.AvertWebService.invoke(AvertWebService.java: 84)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:835)
                at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:626)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:615)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:474)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:449)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:643)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:651)
                at com.mcafee.epo.avertalerts.daemon.AvertService$AvertDaemon.run(AvertService.jav a:94)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
                at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
                at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (ScheduledThreadPoolExecutor.java:98)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(ScheduledThreadPoolExecutor.java:181)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Schedu ledThreadPoolExecutor.java:205)
                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
                at java.lang.Thread.run(Thread.java:619)
            2009-11-23 07:29:18,612 ERROR [pool-5-thread-1] daemon.AvertService  - Exception while executing the AvertAlerts command: String index out of range: -1981052140
            2009-11-23 07:29:19,331 WARN  [Thread-1] table.StringLookupColumn  - Invalid column query flag detected and ignored: order
            2009-11-23 07:29:19,331 WARN  [Thread-1] table.EnumColumn  - Invalid column query flag detected and ignored: order
            2009-11-23 07:29:19,331 WARN  [Thread-1] table.EnumColumn  - Invalid column query flag detected and ignored: order
            2009-11-23 07:29:23,472 INFO  [Thread-1] startup.Catalina  - Server startup in 15422 ms

             

             

             

            While looking in Orinon.log i noticed another error:

             

             

            3 07:18:57,472 INFO  [Thread-1] storeconfig.StoreLoader  - Find registry server-registry.xml at classpath resource
            2009-11-23 07:18:58,269 WARN  [Thread-1] email.EmailServiceConfig  - username was null. assuming no username or password
            2009-11-23 07:19:02,925 INFO  [pool-5-thread-1] auth.AuthChallengeProcessor  - basic authentication scheme selected
            2009-11-23 07:19:02,940 INFO  [pool-5-thread-1] httpclient.HttpMethodDirector  - Failure authenticating with BASIC 'Squid proxy-caching web server'@proxy.xxx.de:3128
            2009-11-23 07:19:02,940 ERROR [pool-5-thread-1] cmd.CommandInvoker  - Error invoking command:AvertAlerts.Web.Service
            java.io.IOException: POST Command got unexpected HTTP Status:407
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.sendSoapRequest(AvertAlert sClient.java:164)
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.getReport(AvertAlertsClien t.java:195)
                at com.mcafee.epo.avertalerts.myavert.AvertAlertsClient.getReportByName(AvertAlert sClient.java:229)
                at com.mcafee.epo.avertalerts.command.AvertWebService.invoke(AvertWebService.java: 84)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:835)
                at com.mcafee.orion.core.cmd.CommandInvoker.invokeCommand(CommandInvoker.java:626)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:615)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:474)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:449)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:643)
                at com.mcafee.orion.core.cmd.CommandInvoker.invoke(CommandInvoker.java:651)
                at com.mcafee.epo.avertalerts.daemon.AvertService$AvertDaemon.run(AvertService.jav a:94)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
                at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
                at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101 (ScheduledThreadPoolExecutor.java:98)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodi c(ScheduledThreadPoolExecutor.java:181)
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Schedu ledThreadPoolExecutor.java:205)
                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java: 885)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
                at java.lang.Thread.run(Thread.java:619)
            2009-11-23 07:19:02,940 ERROR [pool-5-thread-1] daemon.AvertService  - Exception while executing the AvertAlerts command: POST Command got unexpected HTTP Status:407

             

             

            On the ePO Server is Java (JRE) 6 Update 13 installed.

             

            Like i see in the description above, the error has probably to do with the proxy auth. MyAvert was still functionional until i installed Patch 5. Are there anything wrong with this Patch?

             

            Thanx for reply.

             

            Marco

             

             

            PS: Sorry for my bad english... try my best :-)

            • 3. Re: Crash EPO Application Server (4.0.0 Patch 5)
              GWIRT

              No worries Marco.

               

              This may be the root of the issue:

               

              POST Command got unexpected HTTP Status:407

               

              We are getting a "Failed to authenticate" message from the proxy which may be causing the error. Tomcat shouldn't crash because of this, however. This may be an issue support will need to address. I would suggest getting an SR open so we can investigate further.

              1 of 1 people found this helpful
              • 4. Re: Crash EPO Application Server (4.0.0 Patch 5)

                Hello,

                 

                I am facing a similar situation. I recently patched EPO 4.0 with patch 5. McAfee Application Server has been failing since then. It mostly happens when we are running a replication task. I have a ticket raised and escalated with McAfee support at this time. As for the troubleshooting that we have been doing, we have tried everything up to a fresh install of the EPO itself. We feared that the Patch 5 was not installed properly. But that does not seem to be the problem as even after the clean install we have Application Server failing. Any inputs / ideas are appreciated !

                 

                PS: I have the MER logs available if anyone wants to have a look at them. 

                • 5. Re: Crash EPO Application Server (4.0.0 Patch 5)

                  As workaround i deactivated the proxy config for getting data from master repository and MyAvert data and ePO works fine. To clear the proxy behaviour i made a support call to McAfee.

                   

                  Greets

                  Marco