2 Replies Latest reply on Mar 14, 2014 11:51 AM by bozhinov

    Slow API

    bozhinov

      Hello,

       

      I understand the gurus are faster found here than through the regular support channels

       

      I started porting my automation for McAfee ePO (atm 4.6.6) and I used the web api as the obvious choice

       

      Couple of things though:

       

      1.  When using core.executeQuery queryId I don't have the option to select the output (e.g. XML or JSON)

      I had to code in a parser for the raw output.

      (Yes, I m using PHP)

       

      2. Using system.find to search for the epo host name(e.g. srvepohost), the API takes about 120 seconds to execute where the regular interface returns the info for about 3 seconds

       

      3. Getting result of about 60 000 records is mission impossible. After setting the timeout at about 10 minutes, I gave up. Regular interface only takes 3 minutes.

       

      I expected the API to be faster than the regular interface

       

      4. epo.getVersion requires global admin rights, which I find odd

      if you open the epo login page you get the version right on top

       

      5. "Explain" as a parameter to a method would be a good idea. (MongoDB style). It could force the method to return the result from the sql query builder where applicable

       

      Great potential though

      Thank you

      Momchil

       

      Message was edited by: bozhinov on 2/21/14 4:18:49 PM CST

       

      Message was edited by: bozhinov on 2/21/14 4:19:52 PM CST
        • 1. Re: Slow API
          jking

          Hey Momchil, sorry for a late response.  Here's a couple thoughts:

           

          1) if you're using the python client it will automatically choose json as the output and return objects to you.  If you're using anything else, you can use the :output parameter to choose the return type you want.  Valid values are json,xml,terse,verbose.  (ex: https://<server>:8443/remote/system.find?searchText=<foo>&:output=terse).  json and xml are what you'd expect.  Terse returns a table, and verbose returns "human readable" although honestly I find terse or json easier to read myself.

           

          2) system.find does a full text search in a number of columns; there are places in the UI where we can start the search, move to the page, and dynamically load the results once they've completed.  But 3s to 120s sounds like an order of magnitude off -- it's possible that the api is searching more columns than the UI.

           

          3) the UI uses cursors and dynamically loads data pages based on how you scroll the table (it's not very noticable, but if you see the 'green dot' above the right scroll bar of any table view, it will flash when we're loading new pages of data).  When you're asking for large data sets from the API, we don't control both sides -- you need to change the script to query for things in batches.  For example, you can add filters to your queries (see my ppt in the doc section here for how to add a where clause).  If you're getting systems, for example, maybe you should get them by group, or by first letter, etc. 

           

          4) that could be loosened up; sounds like it's basically the default (restricted).

           

          5) I'm only cursorily familiar with MongoDB, sorry.  However, you can always do a core.help?command=<foo> to get more help about the <foo> command. 

           

          Jon

          • 2. Re: Slow API
            bozhinov

            Hi Jon,

             

            1) I have read the documentation. Problem is that the "output" parameter is not valid for the Core.executeQuery method

            2) I guess so

            3) According to the log file the query is using LOCKS and fails with resourse locked.

            Now, when running against a 60 000 client environment it is very hard not to hit a lock

            4) that was a minor note. It will ease up exploitation as well as legit usage

            5) The point was to get the generated SQL query instead of the result of the query

             

            Thank you for your response

            Momchil