Cisco

Cisco ISE – Live Logs Broken

Posted by robd on February 24, 2022
Cisco / 5 Comments

Hello,

After upgrading to Cisco ISE 3.0 and updating some certs we noticed the Radius Live Logs broke:

So to fix, I changed certs and rebooted nodes and basically spent hours trying everything.

I dint get anywhere so I raised a Cisco TAC and they fixed it by doing the following (which took two seconds), they un-checked:

”

 

 

Tags: , ,

Firmware Update Cisco SG350

Posted by robd on January 12, 2022
Cisco / 1 Comment

I had to update a Cisco SG350 recently, which should have been really easy but ended up being a bit of a pain.

Here’s how I’d do it again:

  • Download the firmware:

https://www.cisco.com/c/en/us/support/switches/sg350-28-28-port-gigabit-managed-switch/model.html

  • Ignore the GUI, pretend it never existed, as best I can work out it just doesn’t work well.
  • Download a tftp software, I suggest tftpd64 as it brilliant:

https://pjo2.github.io/tftpd64/

  • Run the tftp software and point it at the firmware, make a note of the IP.
  • SSH to the switch with putty and run the following to check the version:
show ver
  • Then run this command on the switch (The IP if the IP of the tftp software):
boot system tftp://192.168.1.2/image_tesla_hybrid_2.5.8.15_release_cisco_signed.bin

  • Now reboot:
reboot
  • Once it’s rebooted, check the version, and you are done!
show ver

Tags: , ,

Upgrade Cisco ISE from 2.6 to 2.7

Posted by robd on February 03, 2021
Cisco, Wireless / No Comments

Hello,

Upgrading Cisco ISE is pretty straight forward, there’s a pretty GUI that makes sure you back everything up before you start and then you specify a repository and the files get downloaded and bobs your uncle.

FYI – This is good blog on backing up ISE before you start:  https://www.letsconfig.com/how-to-backup-cisco-ise-2-7/

My issue is I have a ISE node in a DMZ which cant contact the main repository on the network due to it being ultra secure.

So I had to do it manually via the Cisco ISE CLI on the node in the dmz.

First find a server that can access the DMZ on port 21 – Note, I tried tftp but the transfer would fail every time.

Then download this portable ftp server: https://www.xlightftpd.com/download.htm

Run the ftp server, setup the NIC and create a user with a home directory (a folder on the server)

Download the ISE upgrade file and put it in the home directory: ise-upgradebundle-2.2.x-2.6.x-to-2.7.0.356.SPA.x86_64.tar.gz

Next logon to your ISE node,

Create a repository on the ISE node:

conf t
repository dmzf
url ftp://172.25.61.42
user FTPAdmin password plain FTPPassword

Now you can pull the upgrade file – Note, this will just download and unpackaged the file, NOT run the update.

application upgrade prepare ise-upgradebundle-2.2.x-2.6.x-to-2.7.0.356.SPA.x86_64.tar.gz dmzf

Wait for that to finish:

Now you’re ready to actually upgrade.

application upgrade proceed

Wait for the reboot and update:

Then  you are done!!

Double check

Show Version

 

Done. Boom

Tags: , , ,

Cisco Wireless & DHCP

Posted by robd on December 02, 2020
Cisco, Wireless / No Comments

Had a very frustraiting issue recently where our Zebra RF Scanners werent getting DHCP addresses on certain Cisco Access Points.

Only the scanners were not working, everything else seemed fine!

So I checked a heap of things:

Data Rates

Some of RF scanners are OLD, so its important to find out what data rates they require and then match your RF profile.

I suggest you profile the scanner using sometime like a WLANPi first just so you dont have to enable any older data rates.

Or use:

Show client detail <MAC Address>

Read more about old data rates here.

Port Config

We run FlexConennect so was every port in a Trunk and did every port have have the correct vlans tags?

Trunk

Where all the vlans trunked up to the core switch?

DHCP Server

Rebooted it and everything seems fine, lots of DHCP requests from other devices etc.

To be sure I did run wireshark and there were no requests from the scanners while on the “broken” APs.

Debug, Debug, Debug

I then started these debugs and waited forced the client to join again:

From AP:
config ap client-trace address add 5c:87:9c:93:da:4b
config ap client-trace filter all enable 
config ap client-trace output console-log enable 
config ap client-trace start 
term mon

#when 
config ap client-trace stop


From WLC:
Debug client 11:22:33:44:55:66
Show client detail 11:22:33:44:55:66

So the results showthis:

When it works it looks like this:

DHCP request,

DOT11 Auth

DOT11 Association

ARP

DHCP Request

DHCP ACK

Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682125] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_DISCOVER : TransId 0xefec6e9f
Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682163] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_DISCOVER : TransId 0xefec6e9f
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2458] [1606813363:245845] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DISASSOC : (.)
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2465] [1606813363:246587] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DEAUTHENTICATION : (.)
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9712] [1606813363:971275] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_AUTHENTICATION : (.)
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9721] [1606813363:972101] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_AUTHENTICATION : (.)
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9829] [1606813363:982985] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_REASSOC_REQUEST : (.)
Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9839] [1606813363:983901] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_REASSOC_RESPONSE : (.)
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0783] [1606813364: 78316] [AP16] [11:22:33:44:55:66] <wired0> [D:E] EAP_PACKET.Request : Id 0x01 type 1 Identity
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0784] [1606813364: 78397] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0x01 type 1 Identity
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1278] [1606813364:127862] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0x01 type 1 Identity
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1279] [1606813364:127968] [AP16] [11:22:33:44:55:66] <wired0> [U:E] EAP_PACKET.Response : Id 0x01 type 1 Identity
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1745] [1606813364:174565] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0xa7 type 25 Other
Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1773] [1606813364:177337] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0xa7 type 25 Other
Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8440] [1606813365:843995] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAPOL_KEY.M1 : DescType 0x02 KeyInfo 0x008a
Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8906] [1606813365:890656] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAPOL_KEY.M2 : DescType 0x02 KeyInfo 0x010a
Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28207] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1
Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28252] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1
Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29096] [AP16] [11:22:33:44:55:66] <wired0> [D:E] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29138] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29187] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1
Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1
Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0555] [1606813367: 55585] [AP16] [11:22:33:44:55:66] <wired0> [D:C] DHCP_ACK : TransId 0xa68db1f1
Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0556] [1606813367: 55636] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DHCP_ACK : TransId 0xa68db1f1

When it doesnt, everything looks good until the end, no ACK from DHCP:

Dec  1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682125] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_DISCOVER : TransId 0xefec6e9f
Dec  1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682163] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_DISCOVER : TransId 0xefec6e9f
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.2458] [1606813363:245845] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DISASSOC : (.)
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.2465] [1606813363:246587] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DEAUTHENTICATION : (.)
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.9712] [1606813363:971275] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_AUTHENTICATION : (.)
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.9721] [1606813363:972101] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_AUTHENTICATION : (.)
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.9829] [1606813363:982985] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_REASSOC_REQUEST : (.)
Dec  1 09:02:43 kernel: [*12/01/2020 09:02:43.9839] [1606813363:983901] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_REASSOC_RESPONSE : (.)
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.0783] [1606813364: 78316] [AP16] [11:22:33:44:55:66] <wired0> [D:E] EAP_PACKET.Request : Id 0x01 type 1 Identity
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.0784] [1606813364: 78397] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0x01 type 1 Identity
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.1278] [1606813364:127862] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0x01 type 1 Identity
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.1279] [1606813364:127968] [AP16] [11:22:33:44:55:66] <wired0> [U:E] EAP_PACKET.Response : Id 0x01 type 1 Identity
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.1745] [1606813364:174565] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0xa7 type 25 Other
Dec  1 09:02:44 kernel: [*12/01/2020 09:02:44.1773] [1606813364:177337] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0xa7 type 25 Other
Dec  1 09:02:45 kernel: [*12/01/2020 09:02:45.8440] [1606813365:843995] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAPOL_KEY.M1 : DescType 0x02 KeyInfo 0x008a
Dec  1 09:02:45 kernel: [*12/01/2020 09:02:45.8906] [1606813365:890656] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAPOL_KEY.M2 : DescType 0x02 KeyInfo 0x010a
Dec  1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28207] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1
Dec  1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28252] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1
Dec  1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29096] [AP16] [11:22:33:44:55:66] <wired0> [D:E] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec  1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29138] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec  1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29187] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1
Dec  1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1

So what does this tell us? The DHCP requests are not getting to the DHCP server.

The Fix

So based on the above, I doubled checked the switches.

Trunks and ports were fine BUT I had missed something!!

Show VLAN Brief

Showed me I hadnt actually added the sodding vlan on the switch…… 🙁

Why did other devices work?

Well we use one SSID and Cisco ISE moves RF scanners to a different vlan when they’ve authed. Other devices dont use our special RF scanners VLAN.

The Lesson

Its never Wireless, its always something else!

 

Tags: , , ,

Updating Cisco Prime 3.4 to 3.7

Posted by robd on March 06, 2020
Cisco / 2 Comments

Well what a fun few days its been.  I’ve been on a mad mission to update our Cisco products, first I did Cisco ISE (I’ll try and blog about that later) and then Cisco Prime.

Here’s the Cisco Prime tac I took:

SSH to Prime

First thing first, backup Prime.  In my case that was on the prime server in the default repo.

copy NAMEofBACKUP.tar.gz ftp://10.1.1.2/

Next now ftp the update the file to prime:

copy FTP://10.1.1.2/PI-Upgrade-31x_32x_33x_34x_to_3.7.0.0.159.tar.gz disk:/defaultRepo

next check its there:

show repository defaultRepo

Now update:

application upgrade PI-Upgrade-31x_32x_33x_34x_to_3.7.0.0.159.tar.gz defaultRepo

Oh no ERROR:

ERROR : Please run the application upgrade from the system console to monitor upgrade progress. Use system monitor, serial terminal or a virtual console to initiate the upgrade.

You have to open the VM console!!!!!!!

Try again!

application upgrade PI-Upgrade-31x_32x_33x_34x_to_3.7.0.0.159.tar.gz defaultRepo

Ugh, its full.  Delete the old backups

Delete disk://BACKUP.tar.gz defaultRepo

Try again

application upgrade PI-Upgrade-31x_32x_33x_34x_to_3.7.0.0.159.tar.gz defaultRepo

Arggggh now what, stop Prime….

Ncs stop

Try again:

application upgrade PI-Upgrade-31x_32x_33x_34x_to_3.7.0.0.159.tar.gz defaultRepo

Now its working!!

Tags: , ,