cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted

OSPF adjacency loss

Hi team,

Please, your help

Incident description:

Our customer has a McAfee firewall, model S3008, between a Cisco VPN router and a Cisco ASA firewall. The topology is:

VPN router                          McAfee S3008 FW                    ASA firewall

               ------------------- 1-1          1-0 -------------------

IP 192.168.13.65                   .66          .130                   IP 192.168.13.129

The McAfee firewall is directly connected to the VPN router through its 1-1 interface, running at 100 Mbps, and directly connected to the ASA through its 1-0 interface, also at 100 Mbps. The three boxes are running OSPF, area 102.

Last night, at 23:03:52, for unknown reasons, the McAfee firewall lost OSPF protocol adjacency with its neighbors, even when the interfaces seemed to be up. This adjacency was not reestablished until a kernel watchdog event, ocurred at 00:26:54. After this event, the adjacency with both boxes was reestablished.

The client wants an explanation about this issue, because it disrupted its normal operation, so we have the following questions about this:

- Is this problem hardware or software related?

- Under what circumstances does a kernel watchdog of this type happens?

- Why did it took so long to the watchdog to detect an anomaly?

- Could this event happen again?

Please, let us know if you require more information.

These are the relevant system logs:

  1. Messages.log

Jun 23 23:04:19 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:12:53 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:18:11 corpbanca2 sfagent[1447]: Failed to look up host 'list.smartfilter.com': error 8

Jun 23 23:18:11 corpbanca2 sfagent[1447]: The download function returned a permanent error

Jun 23 23:18:11 corpbanca2 sfagent[1447]: Failed to download a list

Jun 23 23:18:11 corpbanca2 sfagent[1447]: SmartFilter was unable to download the Internet Database on corpbanca2.rbi.cl.^M ^M Reason: Invalid hostname (list.smartfilter.com) for download server^M ^M SmartFilter will retry the download in 30 minutes.  You can also try to manually download the Internet Database:  On the SmartFilter Administration Console toolbar, click the Download Internet Database button.^M ^M If this problem continues, contact Secure Computing Technical Support online at www.securecomputing.com.  To contact Technical Support directly, call +1.800.700.8328 or +1.651.628.1500.^M

Jun 23 23:21:27 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:30:00 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:38:34 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:47:07 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 23 23:48:17 corpbanca2 sfagent[1447]: Failed to look up host 'list.smartfilter.com': error 8

Jun 23 23:48:17 corpbanca2 sfagent[1447]: The download function returned a permanent error

Jun 23 23:48:17 corpbanca2 sfagent[1447]: Failed to download a list

Jun 23 23:48:17 corpbanca2 sfagent[1447]: SmartFilter was unable to download the Internet Database on corpbanca2.rbi.cl.^M ^M Reason: Invalid hostname (list.smartfilter.com) for download server^M ^M SmartFilter will retry the download in 30 minutes.  You can also try to manually download the Internet Database:  On the SmartFilter Administration Console toolbar, click the Download Internet Database button.^M ^M If this problem continues, contact Secure Computing Technical Support online at www.securecomputing.com.  To contact Technical Support directly, call +1.800.700.8328 or +1.651.628.1500.^M

Jun 23 23:55:41 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 24 00:04:14 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 24 00:18:23 corpbanca2 sfagent[1447]: Failed to look up host 'list.smartfilter.com': error 8

Jun 24 00:18:23 corpbanca2 sfagent[1447]: The download function returned a permanent error

Jun 24 00:18:23 corpbanca2 sfagent[1447]: Failed to download a list

Jun 24 00:18:23 corpbanca2 sfagent[1447]: SmartFilter was unable to download the Internet Database on corpbanca2.rbi.cl.^M ^M Reason: Invalid hostname (list.smartfilter.com) for download server^M ^M SmartFilter will retry the download in 30 minutes.  You can also try to manually download the Internet Database:  On the SmartFilter Administration Console toolbar, click the Download Internet Database button.^M ^M If this problem continues, contact Secure Computing Technical Support online at www.securecomputing.com.  To contact Technical Support directly, call +1.800.700.8328 or +1.651.628.1500.^M

Jun 24 00:21:19 corpbanca2 ntpd[1444]: sendto(192.168.5.66): No route to host

Jun 24 00:26:54 corpbanca2 kernel: igb0: Watchdog timeout -- resetting

Jun 24 00:26:54 corpbanca2 kernel: igb0: Queue(0) tdh = 2990, hw tdt = 3672

Jun 24 00:26:54 corpbanca2 kernel: igb0: TX(0) desc avail = 8,Next TX to Clean = -256

Jun 24 00:29:32 corpbanca2 sshd[7082]: Connection closed by 163.250.240.95 [preauth]

Jun 24 00:31:48 corpbanca2 sshd[7090]: Disconnecting: Change of username or service not allowed: (interside,ssh-connection) -> (HDDG2636,ssh-connection) [preauth]

Jun 24 00:32:56 corpbanca2 sshd[7099]: Disconnecting: Change of username or service not allowed: (interside,ssh-connection) -> (HRRG8203,ssh-connection) [preauth]

Jun 24 00:33:14 corpbanca2 sshd[7101]: Disconnecting: Change of username or service not allowed: (interside,ssh-connection) -> (HRRG8203,ssh-connection) [preauth]

Jun 24 00:34:04 corpbanca2 sshd[7105]: fatal: Read from socket failed: Connection reset by peer [preauth]

Jun 24 00:36:08 corpbanca2 login: 3 LOGIN FAILURES ON tty??

Jun 24 00:38:18 corpbanca2 sshd[7136]: Received disconnect from 163.250.240.55: 13: The user canceled authentication.  [preauth]

Jun 24 00:40:24 corpbanca2 login: 3 LOGIN FAILURES ON tty??

Jun 24 00:48:23 corpbanca2 sfagent[1447]: Failed to download a list

  1. daemon.log

Tue Jun 23 02:00:00 2015 corpbanca2.rbi.cl rollaudit[5669]: logfile turned over

Jun 23 23:03:52 corpbanca2 ospfd[1468]: nsm_change_state(192.168.13.65, Full -> Init): scheduling new router-LSA origination

Jun 23 23:03:52 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 0.0.0.0

Jun 23 23:03:52 corpbanca2 ospfd[1468]: DR-Election[1st]: DR     192.168.13.66

Jun 23 23:03:54 corpbanca2 ospfd[1468]: *** sendmsg in ospf_write failed to 192.168.13.129, id 0, off 0, len 80, interface 1-0, mtu 1500: Host is down

Jun 23 23:03:54 corpbanca2 ospfd[1468]: nsm_change_state(192.168.13.162, Full -> Init): scheduling new router-LSA origination

Jun 23 23:03:54 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 0.0.0.0

Jun 23 23:03:54 corpbanca2 ospfd[1468]: DR-Election[1st]: DR     192.168.13.130

Jun 23 23:18:11 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 23 23:18:11 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 23 23:48:17 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 23 23:48:17 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 24 00:18:23 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 24 00:18:23 corpbanca2 named[1452]: error (host unreachable) resolving 'list.smartfilter.com/A/IN': 192.168.5.66#53

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 0.0.0.0

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.66

Jun 24 00:26:54 corpbanca2 ospfd[1468]: Packet[DD]: Neighbor 192.168.13.65: Initial DBD from Slave, ignoring.

Jun 24 00:26:54 corpbanca2 ospfd[1468]: *** sendmsg in ospf_write failed to 192.168.13.65, id 0, off 0, len 52, interface 1-1, mtu 1500: Host is down

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.65

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.66

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.65

Jun 24 00:26:54 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.66

Jun 24 00:26:59 corpbanca2 ospfd[1468]: Packet[DD]: Neighbor 192.168.13.65: Initial DBD from Slave, ignoring.

Jun 24 00:26:59 corpbanca2 ospfd[1468]: Packet[DD]: Neighbor 192.168.13.65 Negotiation done (Master).

Jun 24 00:26:59 corpbanca2 ospfd[1468]: nsm_change_state(192.168.13.65, Loading -> Full): scheduling new router-LSA origination

Jun 24 00:27:03 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 0.0.0.0

Jun 24 00:27:03 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.129

Jun 24 00:27:03 corpbanca2 ospfd[1468]: DR-Election[2nd]: Backup 192.168.13.130

Jun 24 00:27:03 corpbanca2 ospfd[1468]: DR-Election[2nd]: DR 192.168.13.129

Jun 24 00:27:03 corpbanca2 ospfd[1468]: Packet[DD]: Neighbor 192.168.13.162 Negotiation done (Slave).

Jun 24 00:27:03 corpbanca2 ospfd[1468]: nsm_change_state(192.168.13.162, Loading -> Full): scheduling new router-LSA origination

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.129

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.129

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[2nd]: Backup 192.168.13.129

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[2nd]: DR 192.168.13.129

Jun 24 00:27:07 corpbanca2 ospfd[1468]: interface 192.168.13.130 [1] leave AllDRouters Multicast group.

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.129

Jun 24 00:27:07 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.129

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.130

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.129

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[2nd]: Backup 192.168.13.130

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[2nd]: DR 192.168.13.129

Jun 24 00:27:17 corpbanca2 ospfd[1468]: interface 192.168.13.130 [1] join AllDRouters Multicast group.

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[1st]: Backup 192.168.13.130

Jun 24 00:27:17 corpbanca2 ospfd[1468]: DR-Election[1st]: DR 192.168.13.129

Jun 24 00:47:48 corpbanca2 ospfd[1468]: Vty connection from 127.0.0.1

Jun 24 00:48:13 corpbanca2 ospfd[1468]: Vty connection from 127.0.0.1

  • ospfd.conf

router ospf

ospf router-id 192.168.13.66

network 192.168.13.64/27 area 0.0.0.102

network 192.168.13.128/27 area 0.0.0.102

!

Best regards,

More McAfee Tools to Help You
  • Subscription Service Notification (SNS)
  • How-to: Endpoint Removal Tool
  • Support: Endpoint Security
  • eSupport: Policy Orchestrator
  • Community Help Hub

      New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

    • Find Forum FAQs
    • Learn How to Earn Badges
    • Ask for Help
    Go to Community Help

    Join the Community

      Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

    • Get helpful solutions from McAfee experts.
    • Stay connected to product conversations that matter to you.
    • Participate in product groups led by McAfee employees.
    Join the Community
    Join the Community