Had a very frustraiting issue recently where our Zebra RF Scanners werent getting DHCP addresses on certain Cisco Access Points.
Only the scanners were not working, everything else seemed fine!
So I checked a heap of things:
Data Rates
Some of RF scanners are OLD, so its important to find out what data rates they require and then match your RF profile.
I suggest you profile the scanner using sometime like a WLANPi first just so you dont have to enable any older data rates.
Or use:
Show client detail <MAC Address>
Read more about old data rates here.
Port Config
We run FlexConennect so was every port in a Trunk and did every port have have the correct vlans tags?
Trunk
Where all the vlans trunked up to the core switch?
DHCP Server
Rebooted it and everything seems fine, lots of DHCP requests from other devices etc.
To be sure I did run wireshark and there were no requests from the scanners while on the “broken” APs.
Debug, Debug, Debug
I then started these debugs and waited forced the client to join again:
From AP: config ap client-trace address add 5c:87:9c:93:da:4b config ap client-trace filter all enable config ap client-trace output console-log enable config ap client-trace start term mon #when config ap client-trace stop From WLC: Debug client 11:22:33:44:55:66 Show client detail 11:22:33:44:55:66
So the results showthis:
When it works it looks like this:
DHCP request,
DOT11 Auth
DOT11 Association
ARP
DHCP Request
DHCP ACK
Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682125] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_DISCOVER : TransId 0xefec6e9f Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682163] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_DISCOVER : TransId 0xefec6e9f Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2458] [1606813363:245845] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DISASSOC : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2465] [1606813363:246587] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DEAUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9712] [1606813363:971275] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_AUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9721] [1606813363:972101] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_AUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9829] [1606813363:982985] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_REASSOC_REQUEST : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9839] [1606813363:983901] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_REASSOC_RESPONSE : (.) Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0783] [1606813364: 78316] [AP16] [11:22:33:44:55:66] <wired0> [D:E] EAP_PACKET.Request : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0784] [1606813364: 78397] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1278] [1606813364:127862] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1279] [1606813364:127968] [AP16] [11:22:33:44:55:66] <wired0> [U:E] EAP_PACKET.Response : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1745] [1606813364:174565] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0xa7 type 25 Other Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1773] [1606813364:177337] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0xa7 type 25 Other Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8440] [1606813365:843995] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAPOL_KEY.M1 : DescType 0x02 KeyInfo 0x008a Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8906] [1606813365:890656] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAPOL_KEY.M2 : DescType 0x02 KeyInfo 0x010a Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28207] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28252] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29096] [AP16] [11:22:33:44:55:66] <wired0> [D:E] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29138] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29187] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0555] [1606813367: 55585] [AP16] [11:22:33:44:55:66] <wired0> [D:C] DHCP_ACK : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0556] [1606813367: 55636] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DHCP_ACK : TransId 0xa68db1f1
When it doesnt, everything looks good until the end, no ACK from DHCP:
Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682125] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_DISCOVER : TransId 0xefec6e9f Dec 1 09:02:39 kernel: [*12/01/2020 09:02:39.6821] [1606813359:682163] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_DISCOVER : TransId 0xefec6e9f Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2458] [1606813363:245845] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DISASSOC : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.2465] [1606813363:246587] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_DEAUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9712] [1606813363:971275] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_AUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9721] [1606813363:972101] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_AUTHENTICATION : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9829] [1606813363:982985] [AP16] [11:22:33:44:55:66] <apr1v0> [U:W] DOT11_REASSOC_REQUEST : (.) Dec 1 09:02:43 kernel: [*12/01/2020 09:02:43.9839] [1606813363:983901] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] DOT11_REASSOC_RESPONSE : (.) Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0783] [1606813364: 78316] [AP16] [11:22:33:44:55:66] <wired0> [D:E] EAP_PACKET.Request : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.0784] [1606813364: 78397] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1278] [1606813364:127862] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1279] [1606813364:127968] [AP16] [11:22:33:44:55:66] <wired0> [U:E] EAP_PACKET.Response : Id 0x01 type 1 Identity Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1745] [1606813364:174565] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAP_PACKET.Request : Id 0xa7 type 25 Other Dec 1 09:02:44 kernel: [*12/01/2020 09:02:44.1773] [1606813364:177337] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAP_PACKET.Response : Id 0xa7 type 25 Other Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8440] [1606813365:843995] [AP16] [11:22:33:44:55:66] <apr1v0> [D:W] EAPOL_KEY.M1 : DescType 0x02 KeyInfo 0x008a Dec 1 09:02:45 kernel: [*12/01/2020 09:02:45.8906] [1606813365:890656] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] EAPOL_KEY.M2 : DescType 0x02 KeyInfo 0x010a Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28207] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0282] [1606813366: 28252] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] ARP_QUERY : Sender 10.10.10.1 TargIp 10.20.20.1 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29096] [AP16] [11:22:33:44:55:66] <wired0> [D:E] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29138] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:46 kernel: [*12/01/2020 09:02:46.0291] [1606813366: 29187] [AP16] [11:22:33:44:55:66] <wired0> [D:C] ARP_REPLY : Sender 10.10.10.1 HwAddr 66:55:44:33:22:11 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52031] [AP16] [11:22:33:44:55:66] < wifi1> [U:W] DHCP_REQUEST : TransId 0xa68db1f1 Dec 1 09:02:47 kernel: [*12/01/2020 09:02:47.0520] [1606813367: 52070] [AP16] [11:22:33:44:55:66] <apr1v0> [U:C] DHCP_REQUEST : TransId 0xa68db1f1
So what does this tell us? The DHCP requests are not getting to the DHCP server.
The Fix
So based on the above, I doubled checked the switches.
Trunks and ports were fine BUT I had missed something!!
Show VLAN Brief
Showed me I hadnt actually added the sodding vlan on the switch…… 🙁
Why did other devices work?
Well we use one SSID and Cisco ISE moves RF scanners to a different vlan when they’ve authed. Other devices dont use our special RF scanners VLAN.
The Lesson
Its never Wireless, its always something else!
