6 Replies Latest reply on Jun 13, 2013 9:58 AM by alexn

    Tomcat7.exe takes 100% CPU when AD synchronization is running

    mdungen

      We've recently upgraded from ePO 4.6.6 to ePO 5.0 and expanded the number of objects to be managed by our ePO server.

      I've excluded the tomcat7.exe process from being scanned by VSE and also excluded the tomcat7.exe process from the Access Protection Policy "Prevent mass mailing worms from sending mail".

       

      The AD synchronization task is running on a daily basis and is synchronizing multiple OU's from our AD. In total around 50.000 object are synchronized.

      I've noticed that this daily task takes about 30 mins. every time it runs and during that time the Tomcat7.exe process takes up 100% of the CPU on the ePO server.

      The ePO server, at the moment, has 4 cores and 12 Gb of memory.

       

      Is there anything I can do to optimize this, because during the AD sync it's almost impossible to do other things within the ePO console.

      I'm also planning to seperate the database onto another server to take some load of the ePO server so that might improve the performance as well.

       

      Any advise is welcome

       

      Thanks

      Martijn.

        • 1. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
          Nielsb

          I have this also in the client task catalogue, while opening or editing a client task.

          Per action 25% CPU utilization (tomcat7) per core on a quad core server.

          • 2. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
            alexn

            Hi mdungen and Nielsb,

             

            Please post error  from stderr.log if there is any.

            Please also check your JAVA memory pool value in Registry?

             

            Alex

            • 3. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
              mdungen

              Hi Alexn,

               

              I've checked the Java memory pool value in the Registry and currently it's set to 4096. I think this was increased when upgrading from 4.6.6 to 5.0. Next I've also lowered it's value back to 1024 as you suggested in post "https://community.mcafee.com/message/290199#290199" but that didn't seem to have any effect.

               

              After I restart the ePO server this is written to the stderr.log: (including a lot more "did not find a matching property" warnings".

               

              2013-06-12 16:37:26 Commons Daemon procrun stderr initialized

              jun 12, 2013 4:37:40 EM org.apache.catalina.core.AprLifecycleListener init

              INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: D:\PROGRA~1\McAfee\EPOLIC~1\Server\bin

              jun 12, 2013 4:37:40 EM org.apache.catalina.startup.SetAllPropertiesRule begin

              WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'id' to 'orion.server.https' did not find a matching property.

              jun 12, 2013 4:37:40 EM org.apache.catalina.startup.SetAllPropertiesRule begin

              WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'id' to 'orion.server.clientCert' did not find a matching property.

              jun 12, 2013 4:37:40 EM org.apache.catalina.realm.JAASRealm setUseContextClassLoader

              INFO: Setting useContextClassLoader = false

              jun 12, 2013 4:37:40 EM org.apache.tomcat.util.digester.SetPropertiesRule begin

              WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlNamespaceAware' to 'false' did not find a matching property.

              jun 12, 2013 4:37:40 EM org.apache.tomcat.util.digester.SetPropertiesRule begin

              WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlValidation' to 'false' did not find a matching property.

              jun 12, 2013 4:37:41 EM org.apache.coyote.AbstractProtocol init

              INFO: Initializing ProtocolHandler ["http-bio-8443"]

              jun 12, 2013 4:37:42 EM org.apache.coyote.AbstractProtocol init

              INFO: Initializing ProtocolHandler ["http-bio-8444"]

              jun 12, 2013 4:37:42 EM org.apache.catalina.startup.Catalina load

              INFO: Initialization processed in 4196 ms

              jun 12, 2013 4:38:04 EM org.apache.catalina.core.StandardService startInternal

              INFO: Starting service Catalina

              jun 12, 2013 4:38:04 EM org.apache.catalina.core.StandardEngine startInternal

              INFO: Starting Servlet Engine:

              jun 12, 2013 4:38:04 EM org.apache.catalina.startup.HostConfig deployDescriptor

               

              Message was edited by: mdungen on 6/12/13 9:51:00 AM CDT
              • 4. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
                alexn

                I dont find anything wrong here and your issue is quit different from Neilsb.

                Could you post orion.log just after synch tasks finshes and you see high spike there.

                 

                Also install Proces monitor and drilll down to see what dll files are executed by tocate process, look for all dll files related with trhis process and find the LOC of these dll.

                Please post back any new findings.

                • 5. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
                  mdungen

                  Our AD synchronization task is scheduled at 2:00 AM.

                   

                  When I check the orion.log I see only 1 real error message just before the AD sync task starts:

                   

                  "2013-06-13 01:59:05,293 ERROR [mfs:pool-2-thread-2] server.OrionLifecycleListener  - I am error"

                   

                  After this error message there is a gap in the Orion.log for about 4 hours...

                  • 6. Re: Tomcat7.exe takes 100% CPU when AD synchronization is running
                    alexn

                    Restart your SQL server service

                    Restart your SQL browser service.

                     

                    See if that does help?