If you update iLO to 1.26 (interim FW update before you can update to anything later) and afterwards the web interface still shows 1.10 (or earlier) and the login box is greyed out, purge temporary internet files and you should be good to carry on!
So recently we had a total network failure, urrrgghhh!
HP Intelligent Management and VMware VCentre started to report that there was a network issue. The reports included a lost connection to every switch on the network and a critical temperate warning on our ESX server of 101 Celsius!!
Well after rebooting some switches and rebuilding a few more the issue was still present, no traffic over the switches….
So we started unplugging cables from the core switch as a last ditched attempt to fix the issue. After unplugging about 30 suddenly the network came back to life!!
Long story short we had two days of total network failure because a cleaner plugged a one network cable into two active ports! NETWORK LOOP and no we didnt have spanning tree enabled.
So the next job was to biwire everything to separate switches and enable Spanning tree on every switch:
Or on a Vlan:
HPSWITCH(config)# vlan10 HPSWITCH(config-vlan-10)# spanning-tree
Note: When you configure a VLAN, the VLAN inherits the global STP settings. However, once you begin to define a VLAN, you can no longer configure standard STP parameters globally using the CLI. From that point on, you can configure STP only within individual VLANs.
As well as our main site we have a remote site, lets call it Remote1. Remote1 is on a basic ADSL line, the site connects to the main site via a site to site VPN between two SonicWall’s. Remote1 has two Meru Access Points (AP332e) which are configured to communicate with the Meru controller at the main site which is where our issue was.
Here’s a pretty picture to help see what I’m on about:
With the help of Meru support who were brilliant I carried out the follow analysis:
So normally Meru AP’s talk to the controller via UDP broadcast packets i.e. UDP port 9292, 9393. If that doesnt work it uses layer 3 IP routing.
From the remote site I can ping (IP address, server name and broadcast address), telnet and http access the Meru Controller via the VPN. Great Layer 3 is good to go.
From the Controller I can ping the Access Points. Again great.
We have two AP’s on the remote site, to test one is set to L3 and one to L2 but neither work…hmmmm
From connecting to the AP’s via a cable we can see the packets are broadcasting and the AP’s have a valid IP address,
A packet trace on the firewalls show the UDP broadcast packets arrive and leave the remote firewall, are ingested and forwarded at the main site,
A port mirror on the controller shows no traffic from the remote site subnet.
A port mirror of the Main Sites firewall show the packets entering the network but when you connect to the next switch and port mirror I cant see any traffic (see wireshark results below):
So what the hell is going on??? Well it turned out I hadnt drawn the network diagram properly (above), here’s the proper topography:
Between the firewall and the first switch we have a Lightspeed Rocket that does a great job of email protection and website filtering. Well after looking on the main web filtering page I noticed a tick box under “Block all unidentified UDP connections, Skype, UltraSurf type traffic, and file-sharing networks such as BitTorrent.”….well bugger!!
So I un-ticked this section and Boom the AP’s came one line!!
Now this isnt great as users could start using P2P so I re-ticked the box and added a exception for AP’s and we have a winner!!!
Big thanks to Meru Support, Lightspeed Support, SonicWall Support, HP Support and Commercial LTD (who in the end helped find my missing piece in the diagram).
We’ve recently implemented a BYOD wireless SSID for end users to check facebook or what ever it is they do. As we’re short on resources we had to use existing DC’s to doll out IP’s but we were obviously worried around users “hacking” into the system. So first thing we did was create a VLAN and assigned it to the wireless then applied several access control lists (ACL’s) to the core switch limiting the access to only the DC’s, proxy, core switch, each other and firewall then blocked the rest:
Firstly check the subnet mask of the VLAN you want to apply the ACL too as the wildcard/subnet address in the ACL will change dependant of the subnet.
The below example will use a subnet mask of 255.255.255.0 (the bits in red are notes only – do not try and apply them)
Logon to a switch and go into config mode:
#Here we specify the ACL will be extended rather than standard then we create a name for the ACL ip access-list extended NEWACL #The below rules specify what servers the users on the ACL can have access too: 10 permit ip 192.168.241.0 0.0.0.255 192.168.250.2 0.0.0.0 15 permit ip 192.168.241.0 0.0.0.255 192.168.250.3 0.0.0.0 20 permit ip 192.168.241.0 0.0.0.255 10.0.0.1 0.0.0.0 25 permit ip 192.168.241.0 0.0.0.255 192.168.250.18 0.0.0.0 30 permit ip 192.168.241.0 0.0.0.255 10.0.0.76 0.0.0.0 31 permit ip 192.168.241.0 0.0.0.255 192.168.241.254 0.0.0.0 35 permit ip 192.168.241.254 0.0.0.0 0.0.0.0 255.255.255.255 #The below denies access to certain subnets 50 deny ip 192.168.241.0 0.0.0.255 192.168.0.0 0.0.255.255 55 deny ip 192.168.241.0 0.0.0.255 10.0.0.0 0.255.255.255 57 deny ip 192.168.241.0 0.0.0.255 22.214.171.124 0.255.255.255 #This final permit allows access to itself and therefor out of its network to other networks: 60 permit ip 192.168.241.0 0.0.0.255 0.0.0.0 255.255.255.255 #Apply this ACL to a vlan, in this case 3241: VLAN 3241 ip access-group NEWACL in #Show me my handy work: Show Access-List GuestACL
Ok so what if you have a network that has a subnet of 255.255.252.0, well the wildcard changes in the ACL or above we had 0.0.0.255 where as in a 255.255.252.0 subnet we’d need 0.0.3.255.
#Name of ACL and extended: ip access-list extended "StudentACL1" #List of allowed servers: 10 permit ip 192.168.216.0 0.0.3.255 192.168.250.2 0.0.0.0 15 permit ip 192.168.216.0 0.0.3.255 192.168.250.3 0.0.0.0 20 permit ip 192.168.216.0 0.0.3.255 10.0.0.1 0.0.0.0 25 permit ip 192.168.216.0 0.0.3.255 192.168.250.18 0.0.0.0 26 permit ip 192.168.216.0 0.0.3.255 192.168.250.30 0.0.0.0 27 permit ip 192.168.216.0 0.0.3.255 192.168.250.68 0.0.0.0 30 permit ip 192.168.216.0 0.0.3.255 10.0.0.76 0.0.0.0 31 permit ip 192.168.216.0 0.0.3.255 192.168.216.254 0.0.0.0 35 permit ip 192.168.216.254 0.0.0.0 0.0.0.0 255.255.255.255 #List denied subnets: 50 deny ip 192.168.216.0 0.0.3.255 192.168.0.0 0.0.255.255 55 deny ip 192.168.216.0 0.0.3.255 10.0.0.0 0.255.255.255 57 deny ip 192.168.216.0 0.0.3.255 126.96.36.199 0.255.255.255 #Apply to this VLAN: VLAN 3216 ip access-group StudentACL1 in #Show me the money: Show Access-List StudentACL1 Save
So what happens if you want to delete a ACL from a VLAN:
#remove the ACL from the VLAN: no VLAN 3216 ip access-group StudentACL in #Remove the ACL from the switch: no ip access-list extended StudentACL Save #Now to check its gone either do Show Access-List or Show Config.
Started at a new company recently and they’ve had a ongoing issues for ages where they could only access one of two switches in their server room if they were on the same subnet.
So Switch A was the problem switch and had a IP of 10.0.0.123,
Switch B they could access from anywhere and had a IP of 10.0.0.124.
My Computer had a IP of 192.168.111.1.
So initially I started thinking the issue was down to vlans as we have around 50 vlans and I kind of just presumed it would be a issue with cross vlan routing. Well actually I was wrong.
Firstly I pinged both switches:
SwicthA – Can not ping.
SwitchB – Can ping.
Next Trace route (TraceRT from a win 7 PC)
SwitchA – Could not trace
SwitchB – two hop trace, Core switch then the switch.
Ok so what next…..Lets try the above from the switches (so after connecting to server on the same subnet I ran the tests):
Ping my PC:
SwicthA – could not ping,
SwitchB – could ping.
SwitchA – No route,
SwitchB – two hop route.
So after this I decided it must be a routing issue, as both switches could ping the core switch and both switches could trace route the core switch. So lets have a butchers at the routing.
Switch A# show ip route IP Route Entries Destination Gateway VLAN Type Sub-Type Metric Dist. ------------------ --------------- ---- --------- ---------- ---------- ----- 10.0.0.0/8 DEFAULT_VLAN 1 connected 1 0 127.0.0.0/8 reject static 0 0 127.0.0.1/32 lo0 connected 1 0 192.168.101.0/24 iSCSI-VLAN 1010 connected 1 0
Switch B# show ip route IP Route Entries Destination Gateway VLAN Type Sub-Type Metric Dist. ------------------ --------------- ---- --------- ---------- ---------- ----- 0.0.0.0/0 10.0.0.2 1 static 250 1 10.0.0.0/8 DEFAULT_VLAN 1 connected 1 0 127.0.0.0/8 reject static 0 0 127.0.0.1/32 lo0 connected 1 0 192.168.101.0/24 iSCSI-VLAN 1010 connected 1 0
Hold the phone, there’s no sodding return route to the core switch (10.0.0.2)
So I added the route like so:
Switch B# conf Switch B (conf)# ip route-static 0.0.0.0 0.0.0.0 10.0.0.2 Switch B# write
And bobs your uncle it worked!!