2 Replies Latest reply on Apr 20, 2010 5:32 AM by jawuk

    Agent Handler to SQL Database Backend- Bandwidth Connectivity Requirements

      Hi All

       

      i am currently designing the EPO topology for a customer, and one of the questions you may be able to help me with is this.

       

      I am looking at the possible implementation of Agent Handlers, but with 3 core data centers and hub sites off each one, it would make sense to possibly implement an Agent Handler at 2 of the datacenters, connecitng back to a central high spec SQL back end database.

       

      While i know that Agent handlers are not perfoming queries onto the SQL database in anyway near the requirements for the EPO Application Server hosting the UI Tomcat component, which would no doubt require a low latency, high bandwidth link, i cannot see any mention as to what link of connection is recommended for the Agent Handlers connecting back to the SQL Database. Understanding that the use of Agent Handlers is NOT to reduce bandwidth (although there can be some caching on them of certain things), events from clients will still need to go ''via'' the Agent Handler, and thus the 1.24KB per  event payload will stay pretty much the same, as though the agent were talking  directly to the EPO Server and then onto the SQL Database anyway.

       

      McAfee say, and i quote : - Agent Handlers co-ordinate work between themselves, and the application server communicates
      with remote agent handlers. A work queue in the database is used as the primary communication
      mechanism. Agent handlers check the work queue frequently (approximately every ten seconds)
      and perform the requested action. Typical actions include agent wake-ups, deployments, and
      data channel messages. This is one of the reasons that each agent handler needs a relatively
      high speed, low latency connection to the database.

       

      What is ''relatively high speed" and low latency ? 10 MegaBit? Less than 100ms?

       

      The only benefit i see i can derive from using multiple agent handlers in the core across the datacenters at the moment is redundancy, and load balancing. There will be zero bandwidth savings i know, but, i want to know if i will run into problems if i choose to put an Agent Handler on a line of 10MB or less , between "it" and the backend SQL Database

       

       

      Many thanks

       

      James

       

       

      Message was edited by: jawuk on 13/04/10 10:05:49 CDT
        • 1. Re: Agent Handler to SQL Database Backend- Bandwidth Connectivity Requirements
          jking

          Hi James.

           

          Although there are general performance considerations with low-speed and high latency links between the agent handlers and the database, the primary concern is due to the fact that the agent handlers are frequently performing updates in various tables in the database (computer props, directory structure, events, etc).  Updates and inserts require locks to maintain data consistency, and it's possible for a handler on the other end of a thin pipe to hold a lock on a critical table long enough that it impacts the performance of the entire ePO infrastructure.

           

          The main thing we're interested in maintaining is the "Agent Requests Completed / sec" if you're looking at the performance counters.

           

          Hope that helps.  I would expect 10mb to be more than sufficient.

          Jon

          1 of 1 people found this helpful
          • 2. Re: Agent Handler to SQL Database Backend- Bandwidth Connectivity Requirements

            Thanks Jon

             

            thats great. Useful to know the detail around why it is dependant. I used the 10Mbit connection figuratively though. The Agent handler will actually be on the end of a

             

            The client has an MPLS cloud at thus contention is on the tail circuits into the remote locations. The location we are wanting to put the Agent hander is infact, on a site with a a 12Mbit Link with perhaps 1-4 Mbit free all of the time.

             

             

             

             

            I did some extensice calculations for other reasons to work out requirements to sustain the ASCI we wanted based on the number of events the clients will generate. We worked out that to sustain an ASCI of 1 hour, with an average of 4.16 events per hour (dont ask why its so specific. . .it was based on some math ), that each client would need 0.0324kbps of bandwidth connectivity.(1.5kb for each agent comms (no changes) + 2kb per event *figures i have taken from the whitepapers)

             

            So, for example, a 64kbps link could support 1975 clients, assuming all of that bandwidth was available, and all of the clients were completely randomized over the course of an hour. (i know you cannot randomize the ASCI, it just polls in every hour based on when the clock started ticking).

            Remember, this is only ASCI, a 64 kbps would require a local software repository of course.  So, what i am getting at is, one could say that 1MBit is actually more than enough as far as passing on client messages from clients.

             

            If Agent Handlers only send the content that a client would send to the EPO server, back to the database, then we could base it on the calculations i made, but i know it is not as simple as that, as Agent handlers, on top of client events they have to send back they also have bandwidth requirements for:-

             

            - Pull Policies from the databases for clients

            - Cache required software from the master repository

            - load on the database server for its heartbeat (updated every minute),

            - checking the work queue (every 10 seconds),

            - pool of database connections held open to the database (2xCPU for EventParser + 4xCPU for Apache).

             

             

             

            i quote from the Agent handler Whitepaper: -


            Bandwidth between the Agent Handler and the database will vary based on the number of
            agents connecting to that Agent Handler. However, each Agent Handler will place a fixed
            load on the database server for its heartbeat (updated every minute), checking the work
            queue (every 10 seconds), and by its pool of database connections held open to the database
            (2xCPU for EventParser + 4xCPU for Apache).

             

             

             

             

            I guess it depends on the way the agent handler is supposed to work. I know it is NOT supposed to ''reduce'' bandwidth requirements, so, for example, you could NOT use it to ''consolodate events on a segment'' so, for example, 500 agents tell the agent handler something of a value of something, it parses it , and it tell the database the message ''once'', summising, or "compressing' the clients events. It is not meant for that. Though, if say an agent tells the Agent handler something with a value of 5kb, what is then ''spoken'' back to the database backend? 5kb again? or more? or less?  Is there a formula?

             

             

            Of course it is dependant on the number of clients the Agent Handler is going to support and the ASCI, but,  based on the calculations above

            Still, i would have thought that 1Mbit (120kb/s)  of AVAILABLE bandwidth could infact be enough  dependant on how many clients and the ASCI interval.

             

            A connection of 1024kbps available bandwidth from the clients ''to'' the Agent Handler could support 31,605 clients over the course of an hour, sending the events payload listed above, but, from the Agent handler to the Databse, the question i pose above comes into play.

             

            Regards

             

            James

             

             


             

             

            Message was edited by: jawuk on 20/04/10 05:32:39 CDT