Archive for the ‘Documentation’ Category

Cisco 1142 AP won’t join after WLC reboot.

Update: It looks like it the end of the line for 2106, because there is no update and hasn’t been since 2015 which I have installed.

I have a couple Cisco LAP1142N access points and a Cisco WLC2106. I noticed some pretty consistent packet loss on the management interface of the WLC. I opted to reload the WLC, since it had been up a long time, to see if it would help. However, when it came up and the access points attempted to join the WLC, I was getting certificate errors like these:

*Aug 30 18:17:08.097: %LWAPP-3-CLIENTERRORLOG: Peer certificate verification failed
*Aug 30 18:17:08.097: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*Aug 30 18:17:08.097: DTLS_CLIENT_ERROR: ../capwap/capwap_wtp_dtls.c:352 Certificate verified failed!
*Aug 30 18:17:08.097: %DTLS-4-BAD_CERT: Certificate verification failed. Peer IP: 192.168.0.141
*Aug 30 18:17:08.097: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.0.141:5246
*Aug 30 18:17:08.097: %DTLS-3-BAD_RECORD: Erroneous record received from 192.168.0.141: Malformed Certificate
*Aug 30 18:17:08.097: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.0.141:5246
*Aug 30 18:17:08.098: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 3 combination.
*Aug 30 18:17:08.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.0.141 peer_port: 5246
*Aug 30 18:17:08.095: %PKI-3-CERTIFICATE_INVALID_EXPIRED: Certificate chain validation has failed. The certificate (SN: myserialnumber) has expired. Validity period ended on 20:42:36 UTC Aug 18 2017
*Aug 30 18:17:08.096: %LWAPP-3-CLIENTERRORLOG: Peer certificate verification failed
*Aug 30 18:17:08.097: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*Aug 30 18:17:08.097: DTLS_CLIENT_ERROR: ../capwap/capwap_wtp_dtls.c:352 Certificate verified failed!
*Aug 30 18:17:08.097: %DTLS-4-BAD_CERT: Certificate verification failed. Peer IP: 192.168.0.141
*Aug 30 18:17:08.097: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.0.141:5246
*Aug 30 18:17:08.097: %DTLS-3-BAD_RECORD: Erroneous record received from 192.168.0.141: Malformed Certificate
*Aug 30 18:17:08.097: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.0.141:5246
*Aug 30 18:17:08.098: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 3 combination.
*Aug 30 18:17:08.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.0.141 peer_port: 5246

I checked the time on all of the devices. My WLC is synced to my internal NTP server, and the access points were syncing their time with the WLC when they would load. I found the following field notice from Cisco that addresses the issue:

Field Notice: FN – 63942 – Wireless Lightweight Access Points and WLAN Controllers Fail to Create CAPWAP/LWAPP Connections Due to Certificate Expiration

I applied the workaround, since I currently do not have the software upgrade:

(Cisco Controller) config>ap lifetime-check mic enable

(Cisco Controller) config>ap lifetime-check ssc enable

(Cisco Controller) config>exit
(Cisco Controller) >save
(Cisco Controller) save>config

Are you sure you want to save? (y/n) y

Configuration Saved!

I reloaded one of the access points, but by the time it came up and joined, I noticed that the other access point had already joined, so I guess I didn’t need to do that.

Enable Windows Server To Utilize Invoke-Command Remotely

When I attempted to the Powershell option Invoke-Command against an old server, I was getting the following:

Connecting to remote server servername failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests.
Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure
the WinRM service: “winrm quickconfig”. For more information, see the about_Remote_Troubleshooting Help topic.

Fortunately, it told me what to do resolve the issue. Nice:

C:\>winrm quickconfig
WinRM already is set up to receive requests on this machine.
WinRM is not set up to allow remote access to this machine for management.
The following changes must be made:

Create a WinRM listener on HTTP://* to accept WS-Man requests to any IP on this
machine.

Make these changes [y/n]? y

WinRM has been updated for remote management.

Created a WinRM listener on HTTP://* to accept WS-Man requests to any IP on this
machine.

Cisco 1941 password recovery

Note: This procedure is applicable to most Cisco routers, however the key is knowing the register to use.

Turn the power off.
Turn the power on.
About when you see the following message, hit Ctrl-Break (yes the Pause/Break key).

Readonly ROMMON initialized

You should be presented with the following prompt:

rommon 1 >

Enter confreg 0x2142:

rommon 1 > confreg 0x2142

Then, you will get the following message:

You must reset or power cycle for new config to take effect

Enter reset:

rommon 2 > reset

The router will reboot and start the initial configuration wizard. Just say “No” to skip. This will drop you to a “Router>” prompt.

Enter enable, and you will presented with a “Router#” prompt.

Copy your startup-config to running-config (make sure you do not switch the order or you will lose your configuration):

Router#copy startup-config running-config

Then reset the password (I set it to “cisco” below.):

Router#configure term
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#enable secret cisco

Then, type the following:

config-register 0x2142

If you cannot remember the register number from earlier, you can find by issuing the following:

Router(config)#do show version

Enter:

Router(config)#end

And save:

Router#write mem
Building configuration…
[OK]

Then reload to test:

Router#reload
Proceed with reload? [confirm]

Awk and cases

Good stuff here. I always like to pick up these little things along the way.

If you want to change the case of a string using awk:
Lower case:

$ echo myuppercasestring | awk ‘{print tolower($1)}’

Upper case:

$ echo mylowercasestring | awk ‘{print toupper($1)}’

I used something like this to create little of commands to rename a bunch of upper case file names to lower case file names:

$ ls -c1 | awk ‘{print “mv ” $1 ” ” tolower($1)}’

CentOS 7 – package conflict during update.

I was having trouble getting the most recently installed kernel to boot (not the latest release in the repository). It just immediately crashed like it was a grub issue. So, I decided to update the server to an even later kernel, since it is not really a production server.

However, when I did I kept getting the following conflict message:

Error: kernel conflicts with kmod-20-8.el7_2.x86_64

This what took care of the issue for me:

After running this command, I discovered that it was not an issue with an incomplete installation during my last updates.

# yum-complete-transaction –cleanup-only

Then, I ran the following, which removed a lot of duplicate packages:

package-cleanup –cleandupes

Then, I updated the server again:

# yum -y update

Rebooted the latest kernel in the repository without any issues.

CentOS – disable ciphers in openssh

I used the following procedure to disable the weak ciphers enabled in openssh on CentOS 7:

You could probably guess where you this should be configured, but one of the challenges can be getting of complete list of what is supported.

Get a list of supported ciphers:

# ssh -Q cipher
3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com

To disable one or more, you need to explicitly specify the ciphers you do want to use. For example, arcfour:

# vi /etc/ssh/sshd_config

Ciphers 3des-cbc,blowfish-cbc,cast128-cbc,aes128-cbc,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com

And then, restart sshd:

# systemctl restart sshd

And check:

$ ssh -c arcfour localhost
no matching cipher found: client arcfour server 3des-cbc,blowfish-cbc,cast128-cbc,aes128-cbc,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com

Remotely enable RDP.

Download psexec.exe to run commands remotely on the remote machine.
See Windows 10 note at the end of this post.

Once installed, run psexec to bring up a command prompt on the remote machine:
C:\Tools> psexec \\remotecomputer cmd

Turn off the firewall:
C:\Windows\system32> netsh advfirewall set currentprofile state off
Default Profiles: AllProfiles, CurrentProfile, DomainProfile, PrivateProfile, or PublicProfile.

Create a rule to allow Remote Desktop through the firewall:
C:\Windows\system32> netsh advfirewall firewall set rule group=”Remote Desktop Access” new enable=Yes

These netsh commands will return an “Ok!” when successful.

Next ensure that the “Remote Registry” service is started, so you can modify the registry to enable Remote Desktop:
C:\Windows\system32> net start “Remote Registry”

Then, from your local machine open regedit and select File/Connect Network Registry…
Enter the name or I.P. address of the remote machine.
Once connected, navigate to “REMOTEMACHINE\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server”
Then, double click fDenyTSConnections and change it from a 1 to a 0.

Then, back on your psexec session restart the “Remote Desktop Services” service:
C:\Windows\system32> net stop “Remote Desktop Services”
C:\Windows\system32> net start “Remote Desktop Services”

Now, you should be able to connect, and still connect after rebooting if you set the “Remote Desktop Services” service to Automatic so it starts at boot.

Windows 10 note:
You can also use REG.EXE to edit the registry from your PSEXEC.EXE session. This worked well for Windows 10 without needing to enable Remote Administration:
C:\Windows\system32> REG ADD “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server” /v fDenyTSConnections /t REG_DWORD /d 0 /f

Reset/remove Windows 10 policies

If you need to reset policies on a Windows 10 machine, back to the defaults you can do the following from an elevated command prompt:

To reset the Local Policies:

C:\Windows\system32>setedit /configure /cfg C:\Windows\Inf\defltbase.inf /db C:\Windows\defltbase.sdb

Reset Group Policies by removing the following directories. This file remove the directories with prompting to remove the directory tree as well:

C:\Windows\system32>rmdir /S /Q c:\windows\system32\GroupPolicyUsers
C:\Windows\system32>rmdir /S /Q c:\windows\system32\GroupPolicy

Set Windows Update Proxy Settings via Command Line

I was having trouble with updates on server. It was complaining about the proxy settings. So, I wanted to have the Windows Update use a different proxy configuration. The following commands can be used to manage the proxy settings from the command line:

Display the current settings:

netsh winhttp show proxy

Set the proxy:

netsh winhttp set proxy proxyservername:portnumber

Set proxy and bypass options:

netsh winhttp set proxy proxy-server=”proxyservername:portnumber” bypass-list=”*.mylocal.domain”

And then, when you really screw something up and just want to start over. Reset:

netsh winhttp reset proxy

Back out a yum update.

Boy, I tell ya, the more I learn about linux, the more I love it. Especially, yum. Something was wrong after I used yum to update a package. I didn’t have time to look into it in detail, so I just wanted to back out the change and downgrade the packages that were updated during the update and restore functionality. My initial thought was to restore a snapshot of the system from the night before, but I thought better and decided I would try to rollback the installation and downgrade the application. This is what I did:

Got a list of all the yum transactions on the system using the yum history command:

# yum history list all
Loaded plugins: fastestmirror
ID | Login user | Date and time | Action(s) | Altered
——————————————————————————-
18 | | 2017-03-24 17:32 | I, U | 8 EE

Then, based on the time, I was to determine the transaction ID to get more information about the transaction to verify I had the correct one using the yum history info command:

# yum history info 18
Loaded plugins: fastestmirror
Transaction ID : 18
Begin time : Fri Mar 24 17:32:42 2017
Begin rpmdb : 335:80c8ab3d529a99f5edc0570b5dbf0a9a2475ffda
End time : 17:34:11 2017 (89 seconds)
End rpmdb : 342:0f129cb344b7c87fe9b0f9b0ff74715215284aea
User :
Return-Code : Success
Command Line : update wikid-server-enterprise-4.2.0.b2007-1.noarch.rpm
Transaction performed with:
Installed rpm-4.8.0-55.el6.x86_64 @base
Installed yum-3.2.29-75.el6.centos.noarch @updates
Installed yum-metadata-parser-1.1.2-16.el6.x86_64 @anaconda-CentOS-201311272149.x86_64/6.5
Installed yum-plugin-fastestmirror-1.1.30-37.el6.noarch @base
Packages Altered:
Dep-Install audit-libs-python-2.4.5-3.el6.x86_64 @base
Dep-Install libcgroup-0.40.rc1-18.el6_8.x86_64 @updates
Dep-Install libsemanage-python-2.0.43-5.1.el6.x86_64 @base
Dep-Install policycoreutils-python-2.0.83-30.1.el6_8.x86_64 @updates
Dep-Install rsync-3.0.6-12.el6.x86_64 @base
Dep-Install setools-libs-3.3.7-4.el6.x86_64 @base
Dep-Install setools-libs-python-3.3.7-4.el6.x86_64 @base
Updated wikid-server-enterprise-4.2.0.b1977-1.noarch @/wikid-server-enterprise-4.2.0.b1977-1.noarch
Update 4.2.0.b2007-1.noarch @/wikid-server-enterprise-4.2.0.b2007-1.noarch
Scriptlet output:
1 Stopping Tomcat server … Success!
2 Stopping TimeCop service … Success!
3 Stopping wAuth protocol daemon … Success!
4 RADIUS protocol daemon already stopped.
5 LDAP protocol not enabled.
6 Stopping Logger service … Success!
7 Stopping database … Success!
history info

And then, to downgrade the packages, I used the yum history undo command:

# yum history undo 18
Loaded plugins: fastestmirror
Undoing transaction 18, from Fri Mar 24 17:32:42 2017
Dep-Install audit-libs-python-2.4.5-3.el6.x86_64 @base
Dep-Install libcgroup-0.40.rc1-18.el6_8.x86_64 @updates
Dep-Install libsemanage-python-2.0.43-5.1.el6.x86_64 @base
Dep-Install policycoreutils-python-2.0.83-30.1.el6_8.x86_64 @updates
Dep-Install rsync-3.0.6-12.el6.x86_64 @base
Dep-Install setools-libs-3.3.7-4.el6.x86_64 @base
Dep-Install setools-libs-python-3.3.7-4.el6.x86_64 @base
Updated wikid-server-enterprise-4.2.0.b1977-1.noarch @/wikid-server-enterprise-4.2.0.b1977-1.noarch
Update 4.2.0.b2007-1.noarch @/wikid-server-enterprise-4.2.0.b2007-1.noarch
Loading mirror speeds from cached hostfile
* base: mirror.keystealth.org
* extras: mirror.linuxfix.com
* updates: mirror.sigmanet.com
Failed to downgrade: wikid-server-enterprise-4.2.0.b1977-1.noarch
Resolving Dependencies
–> Running transaction check
—> Package audit-libs-python.x86_64 0:2.4.5-3.el6 will be erased
—> Package libcgroup.x86_64 0:0.40.rc1-18.el6_8 will be erased
—> Package libsemanage-python.x86_64 0:2.0.43-5.1.el6 will be erased
—> Package policycoreutils-python.x86_64 0:2.0.83-30.1.el6_8 will be erased
–> Processing Dependency: policycoreutils-python for package: wikid-server-enterprise-4.2.0.b2007-1.noarch
—> Package rsync.x86_64 0:3.0.6-12.el6 will be erased
—> Package setools-libs.x86_64 0:3.3.7-4.el6 will be erased
—> Package setools-libs-python.x86_64 0:3.3.7-4.el6 will be erased
–> Running transaction check
—> Package wikid-server-enterprise.noarch 0:4.2.0.b2007-1 will be erased
–> Finished Dependency Resolution

Dependencies Resolved

===
Package Arch Version Repository Size
===
Removing:
audit-libs-python x86_64 2.4.5-3.el6 @base 279 k
libcgroup x86_64 0.40.rc1-18.el6_8 @updates 331 k
libsemanage-python x86_64 2.0.43-5.1.el6 @base 312 k
policycoreutils-python x86_64 2.0.83-30.1.el6_8 @updates 1.3 M
rsync x86_64 3.0.6-12.el6 @base 682 k
setools-libs x86_64 3.3.7-4.el6 @base 1.1 M
setools-libs-python x86_64 3.3.7-4.el6 @base 1.6 M
Removing for dependencies:
wikid-server-enterprise noarch 4.2.0.b2007-1 @/wikid-server-enterprise-4.2.0.b2007-1.noarch 99 M

Transaction Summary
===
Remove 8 Package(s)

Installed size: 104 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Stopping Tomcat server … Success!
TimeCop process already stopped.
wAuth protocol daemon already stopped.
RADIUS protocol daemon already stopped.
LDAP protocol not enabled.
Stopping Logger service … Success!
Stopping database … Success!
Erasing : wikid-server-enterprise-4.2.0.b2007-1.noarch 1/8
Erasing : policycoreutils-python-2.0.83-30.1.el6_8.x86_64 2/8
Erasing : setools-libs-python-3.3.7-4.el6.x86_64 3/8
Erasing : setools-libs-3.3.7-4.el6.x86_64 4/8
Erasing : audit-libs-python-2.4.5-3.el6.x86_64 5/8
Erasing : libcgroup-0.40.rc1-18.el6_8.x86_64 6/8
Erasing : libsemanage-python-2.0.43-5.1.el6.x86_64 7/8
Erasing : rsync-3.0.6-12.el6.x86_64 8/8
Verifying : rsync-3.0.6-12.el6.x86_64 1/8
Verifying : wikid-server-enterprise-4.2.0.b2007-1.noarch 2/8
Verifying : policycoreutils-python-2.0.83-30.1.el6_8.x86_64 3/8
Verifying : libsemanage-python-2.0.43-5.1.el6.x86_64 4/8
Verifying : setools-libs-python-3.3.7-4.el6.x86_64 5/8
Verifying : libcgroup-0.40.rc1-18.el6_8.x86_64 6/8
Verifying : audit-libs-python-2.4.5-3.el6.x86_64 7/8
Verifying : setools-libs-3.3.7-4.el6.x86_64 8/8

Removed:
audit-libs-python.x86_64 0:2.4.5-3.el6 libcgroup.x86_64 0:0.40.rc1-18.el6_8 libsemanage-python.x86_64 0:2.0.43-5.1.el6 policycoreutils-python.x86_64 0:2.0.83-30.1.el6_8
rsync.x86_64 0:3.0.6-12.el6 setools-libs.x86_64 0:3.3.7-4.el6 setools-libs-python.x86_64 0:3.3.7-4.el6

Dependency Removed:
wikid-server-enterprise.noarch 0:4.2.0.b2007-1

Complete!

Now, in my case, I was not able downgrade the software package directly, as you can tell from the “Failed to downgrade:” message for the wikid server, the heavy lifting was done.
All I had to do, is install the original package using yum, and I was back in business:

# yum install wikid-server-enterprise-4.2.0.b1977-1.noarch.rpm

And, start the application back up:

# wikidctl start

Return top

INFORMATION