2 Replies Latest reply on Aug 26, 2015 1:18 AM by lnurmi

    NGFW High Memory RAM Usage

    layer0

      Hello

       

       

      We have a Stonesoft NGFW L2 Security engine, since 4 days ago we are seeing high memory RAM usage (99%). On the other hand the load of the engine is normal 16%.How can i know what is generating the

       

       

      root@FW-XX:~# sg-status -l

      System startup: Tue, 21 Jul 2015 14:53:49 +0000

      Software version: McAfee NGFW version 5.8.1.12053 (x86_64), NGFW

      Engine role: Layer-2 Firewall

      ...

      Single node:

        Current status: +

        Current load  : 16%

       

       

      root@FW-XX:~# less /proc/meminfo

       

       

      MemTotal:        3951044 kB

      MemFree:           41884 kB

      Buffers:          187872 kB

      Cached:          1829612 kB

      SwapCached:            0 kB

      Active:          2369436 kB

      Inactive:         320160 kB

      Active(anon):    1181348 kB

      Inactive(anon):     1832 kB

      Active(file):    1188088 kB

      Inactive(file):   318328 kB

      Unevictable:       19732 kB

      Mlocked:               0 kB

      SwapTotal:        965624 kB

      SwapFree:         965624 kB

      Dirty:                88 kB

      Writeback:             0 kB

      AnonPages:        446216 kB

      Mapped:           949672 kB

      Shmem:            736736 kB

      Slab:             141088 kB

      SReclaimable:      96444 kB

      SUnreclaim:        44644 kB

      KernelStack:        2208 kB

      PageTables:         9128 kB

      NFS_Unstable:          0 kB

      Bounce:                0 kB

      WritebackTmp:          0 kB

      CommitLimit:     2941144 kB

      Committed_AS:    1570604 kB

      VmallocTotal:   34359738367 kB

      VmallocUsed:      298776 kB

      VmallocChunk:   34359379983 kB

      HugePages_Total:       0

      HugePages_Free:        0

      HugePages_Rsvd:        0

      HugePages_Surp:        0

      Hugepagesize:       2048 kB

      DirectMap4k:       10332 kB

      DirectMap2M:     4173824 kB

       

      With the top command i see the following.

       

      Alto uso RAM.PNG

       

      Thanks

        • 1. Re: NGFW High Memory RAM Usage
          thyvarin

          Hi,

           

          My guess would be that this is caused by memory leak. There is several bugs in 5.8.1 version so please as first step upgrade to latest 5.8 version 5.8.4. NGFW 5.8.2 and 5.8.3 has several fixes and 5.8.4 has updated OpenSSL libraries to fix the latest OpenSSL vulnerability CVE-2015-1793.

           

          BR,

          Tero

          CVE

          -
          2015
          -

          1793

          • 2. Re: NGFW High Memory RAM Usage
            lnurmi

            Hi,

             

            to be precise the memory usage is not 99% and the usage looks normal. If you look more closely, buffers and cache consume about 2GB of RAM:

             

            MemTotal:        3951044 kB
            MemFree:           41884 kB
            Buffers:          187872 kB
            Cached:          1829612 kB

             

            As you can see cache is using up most of the space. The Linux kernel borrows unused memory for disk caching, and if any process wants more memory it will just be taken from the cache immediately. Buffers can also be discarded to some extent if needed so both of these types of memory can be considered available. In fact, if the buffers and cache are close to zero then you've likely got a memory problem as it indicates there is no unused memory to borrow for caching. In such case you'd likely also see swap being used heavily, which is another clear sign of memory issues.

             

            So in practice you've got MemFree + Buffers + Cached available: 41884 + 187872 + 1829612 = 2059368 (2GB). The other 2GB is used by sg_inspection as indicated by "top", and this is not abnormal in inspection-heavy roles like L2FW and IPS (or even FW/VPN if you inspect all traffic).

             

            I recommend using the "free" command to monitor memory instead, in below example you can see on the second line (-/+ buffers/cache) that by subtracting buffers/cache you've actually got 1245116 kB free. So that's the number to look at instead of the free amount on first line.

             

            root@ngfw:~# free

                         total       used       free     shared    buffers     cached

            Mem:       1984292    1690620     293672          0      96044     855400

            -/+ buffers/cache:     739176    1245116

            Swap:       499704        632     499072

             

            If there was a severe memory leak you would see more memory being used than what is reported by processes in top. For example if you have 90% usage even though when counting together mem usage of all processes it only adds up to 50%, i.e. more memory is allocated than what is claimed by processes.

             

            Anyway despite this I would recommend upgrade to 5.8.4 like Tero said, there are lot of fixes.

             

            BR,

            Lauri