Nagios log date conversion

To convert the nagios date timestamp in the nagios.log to a standard time format, use:

perl -pe ‘s/(\d+)/localtime($1)/e’ /var/log/nagios3/nagios.log

Linux “at” command

Long ago I used to use the at command quite often in a Unix environment. I would just use the following syntax:

# at now /path/scriptname

And it worked.

Now, in Ubuntu, when I use the same syntax, I get the following:

syntax error. Last token seen: /
Garbled time

The solution is to enter “at now” and hit enter. At the at> prompt, enter the full path and name of the script you want to run. Then, hit Ctrl-D to run the job and exit out of at.

# at now
warning: commands will be executed using /bin/sh
at> /path/scriptname
job 7 at Thu Oct 30 11:30:00 2014

Inactive VMware ESXi NFS volumes.

Sometimes, when an NFS server is rebooted or off for a while, the volume will remain inactive or inaccessible and greyed out in vCenter/vSphere. To restore an inactive NFS volume in ESXi version 5.x, after obviously verifying that the NFS server is in fact up, do the following from the command line:

List the mounted volumes:

~ # esxcli storage nfs list
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
———– ————- —– ———- ——- ——— ———————
nfsvol1 /nfs1 false true false Unknown
nfsvol2 /nfs2 false true false Unknown

Then, remove the volumes:

~ # esxcli storage nfs remove -v nfsvol1
~ # esxcli storage nfs remove -v nfsvol2
~ # esxcli storage nfs remove -v nfsvol2

List to ensure that all inactive or accessible volumes are gone:

~ # esxcli storage nfs list

Add or mount the storage:

~ # esxcli storage nfs add -H -s /nfs1 -v nfsvol1
~ # esxcli storage nfs add -H -s /nfs2 -v nfsvol2

And list again to verify that the volumes are mounted:

~ # esxcli storage nfs list
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
———– ————- —– ———- ——- ——— ———————
nfsvol1 /nfs1 true true false Not Supported
nfsvol2 /nfs2 true true false Not Supported

Note: You can achieve the same thing by remounting the volumes through the vCenter UI, but why when the command is so much more fun? Besides, in the UI, you might get an error that doesn’t seem to happen from the command line.

mail command – modify the From address

Here is how you can send an email message from linux command line and use a different from address. This is useful in scripts that run as a user that would not necessarily manage the process or even look at or care about the email.

$ mail -s “Test Subject” — -f -F ‘”Do Not Reply” ‘ < someinputfile

VMware ESXi – Common Information Model

Version: VMware ESXi 5.1 and 5.5.

To determine the hardware in your VMware ESXi host, you can access the Common Information Model (CIM) data using a VMware provided shell script when you install ESXi.

Note: You have to have ssh enabled.

I found it easiest to just run the script and search the output from the less command.

Once logged via ssh as root:

~ # /bin/ > tmp.fil

~ # less tmp.fil

In my case, I was looking for specific physical memory information, so I searched for PhysicalMemory.

I found the following useful to determine the categories in the output:

~ # egrep “^Dumping” tmp.fil
Dumping instances of CIM_Namespace
Dumping instances of CIM_RegisteredProfile
Dumping instances of CIM_Sensor
Dumping instances of OMC_RawIpmiSensor
Dumping instances of OMC_RawIpmiEntity
Dumping instances of CIM_ComputerSystem
Dumping instances of CIM_Chassis
Dumping instances of CIM_SoftwareIdentity
Dumping instances of CIM_Memory
Dumping instances of CIM_PhysicalMemory
Dumping instances of CIM_Processor
Dumping instances of CIM_LogRecord
Dumping instances of CIM_RecordLog
Dumping instances of CIM_EthernetPort
Dumping instances of CIM_PowerSupply
Dumping instances of CIM_PCIDevice
Dumping instances of VMware_StorageExtent
Dumping instances of VMware_Controller
Dumping instances of VMware_StorageVolume
Dumping instances of VMware_Battery
Dumping instances of VMware_SASSATAPort

Chroot vsftpd – if you must.

Tested on Ubuntu 12.04 LTS.

This should be used only for an account for ftp only, because filesystem write access will be removed.

Install vsftp:

$ sudo apt-get install vsftpd

Edit /etc/vsftpd.conf:

$ sudo cp -p /etc/vsftpd.conf /etc/vsftpd.conf.orig
$ sudo vi /etc/vsftpd.conf

# JGZ 9/3/2014 – disable anonymous

# JGZ 9/3/2014 – use local accounts

# JGZ 9/3/2014 – allow writing

# JGZ 9/3/2014 – jail local user accounts in their home directory

# JGZ 9/3/2014 – don’t allow list

# JGZ 9/3/2014 – list permission if in the file

Edit/create chroot list file:

$ sudo vi /etc/vsftpd.chroot_list


Remove write access from the localusername directory:

$ sudo chmod a-w /home/localusername

Exchange Message Tracking using EMS

Sometimes I just love linux even more. Message tracking and just plain logging in Exchange is just unbearable. I love the way it is so simple to get right to the problem a linux system.

Determining what has happened to message in Exchange is just a nightmare. It seems even worse in Exchange 2013, but I know there is a lot of information there. It would just be nice to see a simple standards based SMTP type of log. I have yet to stumble on it in the mountain of logging options in EMS.

First, you have to set the event log level. At least, I believe you do. Regardless, it is something to note here, because it could useful for troubleshooting other kind of issues.

To check the current event log levels:

[PS] C:\>Get-EventLogLevel

I highly recommend piping this out to more, because there are a lot of them. By default, almost all of the log levels are set to Lowest.

To change a log level:

[PS] C:\>Set-EventLogLevel -Identity identityname -Level newlevel

For example:

[PS] C:\>Set-EventLogLevel -Identity MSExchangeTransport\SmtpReceive -Level High

[PS] C:\>get-messagetrackinglog -start “6/6/2014 10:00:00″ -end “6/17/2014 23:59″ -recipient “recipientemailaddress” -sender “senderemailaddress” | format-list | more

Replication – Event ID: 13568

I was getting the following error in the “File Replication Service” event log in a Windows 2003 ADS environment, and replication was not working at all.

The File Replication Service has detected that the replica set “DOMAIN SYSTEM VOLUME (SYSVOL SHARE)” is in JRNL_WRAP_ERROR.

WARNING: It is possible that you can lose some data (policies and scripts). This assumes that the PDC is the machine from where all changes are made, and contains the master copies from which everything will be replicated.

I am not exactly sure this was the correct way to resolve this issue, but I do know that the Event ID 13568 and others have stopped and my policies and scripts are now replicating fine.

This is what I did to resolve the issue:

Stop the “File Replication Service” on the server that is holding your FSMO roles (PDC) and that is your master from which all of your changes are made.

Modify this following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup:

Change the value of “BurFlags” to D2 (non-authoritative restore).

Start the “File Replication Service”

Restart Netlogon

On each of the DCs:

Stop the “File Replication Service”

Modify this following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup:

Change the value of “BurFlags” to D4 (authoritative restore).

Start the “File Replication Service”

Restart Netlogon

Here is a good link for more information from Microsoft, and for how to handle other such event IDs:

S.M.A.R.T. status for VMware.

I wanted to try to find out the health of my hard drives on a server running VMware ESXi 5. Not even thinking, I used the smartctl command in one of my linux guests.

# smartctl –all /dev/sda
smartctl 5.43 2012-06-30 r3573 [x86_64-linux-2.6.32-431.17.1.el6.x86_64] (local build)
Copyright (C) 2002-12 by Bruce Allen,

Vendor: VMware
Product: Virtual disk
Revision: 1.0
User Capacity: 107,374,182,400 bytes [107 GB]
Logical block size: 512 bytes
Device type: disk
Local Time is: Mon Aug 18 11:18:29 2014 PDT
Device does not support SMART

Error Counter logging not supported
Device does not support Self Test logging

When I saw the output above, I was hoping for something I could run from the guest to get the information. Unfortunately, it looks like this something that has to be done from the host OS. You need to use the esxcli command to get the information. First, you need to identify the disks:

~ # esxcli storage core device list
Display Name: Local ATA Disk (t10.ATA_____ST32000542AS________________________________________5XW25D4F)
Has Settable Display Name: true
Size: 1907729
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/t10.ATA_____ST32000542AS________________________________________5XW25D4F
Vendor: ATA
Model: ST32000542AS
Revision: CC34
SCSI Level: 5
Is Pseudo: false
Status: on
Is RDM Capable: false
Is Local: true
Is Removable: false
Is SSD: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unknown
Other UIDs: vml.01000000002020202020202020202020203558573235443446535433323030
Is Local SAS Device: false
Is Boot USB Device: false

As you can see, this provides a lot interesting information about the drive. However, if you have a lot of drives, it can be more information than you need. I used grep to filtered out the unneeded information:

~ # esxcli storage core device list | egrep “^t10″

That way, I can identify all the drives in my system in a neat list.

Then, to view the S.M.A.R.T. information:

~ # esxcli storage core device smart get -d t10.ATA_____ST32000542AS________________________________________5XW25D4F
Parameter Value Threshold Worst
—————————- —– ——— —–
Health Status OK N/A N/A
Media Wearout Indicator N/A N/A N/A
Write Error Count N/A N/A N/A
Read Error Count 111 6 99
Power-on Hours 66 0 66
Power Cycle Count 100 20 100
Reallocated Sector Count 100 36 100
Raw Read Error Rate 111 6 99
Drive Temperature 42 0 46
Driver Rated Max Temperature 58 45 54
Write Sectors TOT Count 200 0 200
Read Sectors TOT Count N/A N/A N/A
Initial Bad Block Count 100 99 100

Find IP address of a device connected to a Cisco switch

Cisco 3750:
IOS Version: 12.2(55)SE8

Yes, I know there are some pricey software tools that will do this with a couple clicks. But I don’t have budget for them. So here is a simple way to determine the IP address of a device connected to a particular port on a Cisco switch.

First, determine the MAC address:

show mac address-table

This will output the VLAN, MAC address and the port for all the connected devices. Here is an example:

30 000c.2928.8da7 DYNAMIC Gi1/0/18
30 000e.0c30.e42a DYNAMIC Gi1/0/17
30 0018.1900.19e0 DYNAMIC Gi1/0/17
100 000c.29b0.0b89 DYNAMIC Gi2/0/36
100 0080.a399.90f9 DYNAMIC Gi3/0/7
200 0023.246f.c77a DYNAMIC Gi2/0/42
200 2c44.fd28.d315 DYNAMIC Gi2/0/42
200 7427.ea1b.551d DYNAMIC Gi3/0/42
300 0004.006f.47ee DYNAMIC Gi3/0/47
300 0021.b7ae.3f00 DYNAMIC Gi6/0/45
300 0030.c1ad.995b DYNAMIC Gi3/0/48
300 0060.b05f.b290 DYNAMIC Gi4/0/45
300 00c0.ee19.f178 DYNAMIC Gi2/0/48
300 00c0.ee3a.a3c8 DYNAMIC Gi3/0/43

So, if you know the port you can obviously find the corresponding MAC address. Once you know the MAC address, you can determine the IP address by cross referencing that MAC address to the output from a “show arp” command.

Here is some sample output:

Protocol Address Age (min) Hardware Addr Type Interface
Internet 0 000e.0c09.fc6a ARPA Vlan400
Internet – 000a.b8c0.0bc3 ARPA Vlan300
Internet 0 0004.23b5.525c ARPA Vlan400
Internet 0 000e.0c08.b9d4 ARPA Vlan400
Internet 0 000c.2934.423f ARPA Vlan400
Internet 4 0007.e924.54cc ARPA Vlan400
Internet 0 000e.0c08.f4e4 ARPA Vlan400
Internet 2 000e.0c32.350e ARPA Vlan400
Internet 0 0007.e924.1c07 ARPA Vlan400
Internet 0 aa00.0400.0404 ARPA Vlan400

Tacking on an “include” to filter the output, makes it pretty simple:

#show mac address-table | include Gi3/0/45
300 00c0.eeb1.0b57 DYNAMIC Gi3/0/45
#show arp | include 00c0.eeb1.0b57
Internet 2 00c0.eeb1.0b57 ARPA Vlan300

Now, I know that is on switch 3 port 45.

Return top