VMWare ESXi and HP DL380 Heap issue.

I was getting the following message when I would try to start up a VM on an HP DL380.

Heap globalCartel-1 already at its maximum size of 7869288. Cannot expand. Could not start VMX: msg.vmk.status.VMK_NO_MEMORY

The machine had plenty of memory and seemed to operating fine for currently running VMs. When I brought up the direct console on the machine, there was screen full of the following:

/bin/sh: can’t fork

I could not enter any commands other than “exit.” I could log in and out, and that is about it.

In VMWare KB article 2085618, it acknowledged the issue with the HP specific image and the HP AMS driver.

Updated the HP AMS driver per VMWare KB article 2085618.

To determine version running:

esxcli software vib list | grep ams

Old version:

# esxcli software vib list | grep ams
hp-ams 550.10.0.0-18.1198610 Hewlett-Packard PartnerSupported 2014-11-07

Download updated driver from HP:

Then, I used scp to copy the downloaded zip file to my ESXi 5.5 server. I copied it to a hp-ams-esxi5.5-bundle- directory I created. Directory: /vmfs/volumes/vmhost02-datastore1/Software/hp-ams-esxi5.5-bundle-10.0.1-2.

Login to the ESXi host using ssh.

# cd /vmfs/volumes/vmhost02-datastore1/Software/hp-ams-esxi5.5-bundle-10.0.1-2

Unzip the file:

# unzip hp-ams-esxi5.5-bundle-10.0.1-2.zip
Archive: hp-ams-esxi5.5-bundle-10.0.1-2.zip
inflating: index.xml
inflating: vendor-index.xml
inflating: metadata-hp-ams-esxi5.5-bundle-10.0.1-2.zip
inflating: hp-esxi-fc-enablement-550.2.1.8-1198610.vib
inflating: hp-smx-limited-550.
inflating: char-hpilo-550.
inflating: char-hpcru-
inflating: hp-ams-550.10.0.1-07.1198610.vib

Files contained in the zip:

# ls -l
total 19456
-rw-r–r– 1 root root 13590 Apr 29 12:15 char-hpcru-
-rw-r–r– 1 root root 12676 Apr 29 12:15 char-hpilo-550.
-rw-r–r– 1 root root 1271770 Apr 29 12:15 hp-ams-550.10.0.1-07.1198610.vib
-rw-r–r– 1 root root 6829546 Apr 29 12:15 hp-ams-esxi5.5-bundle-10.0.1-2.zip
-rw-r–r– 1 root root 1213780 Apr 29 12:15 hp-esxi-fc-enablement-550.2.1.8-1198610.vib
-rw-r–r– 1 root root 4482274 Apr 29 12:15 hp-smx-limited-550.
-rw——- 1 root root 328 Apr 29 12:15 index.xml
-rw-r–r– 1 root root 8531 Apr 29 12:15 metadata-hp-ams-esxi5.5-bundle-10.0.1-2.zip
-rw——- 1 root root 238 Apr 29 12:15 vendor-index.xml

Shutdown or migrate all the guests on the ESXi server, and then put the server in maintenance mode:

# esxcli system maintenanceMode set –enable on

Update the software:

# esxcli software vib update -v /vmfs/volumes/542c3aae-03b91418-ff2a-a0d3c1f0cdf0/Software/hp-ams-esxi5.5-bundle-10.0.1-2/hp
Installation Result
Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
Reboot Required: true
VIBs Installed: Hewlett-Packard_bootbank_hp-ams_550.10.0.1-07.1198610
VIBs Removed: Hewlett-Packard_bootbank_hp-ams_550.10.0.0-18.1198610
VIBs Skipped:

At this point, you need to reboot the ESXi host.

New version:
# esxcli software vib list | grep ams
hp-ams 550.10.0.1-07.1198610 Hewlett-Packard PartnerSupported 2015-04-29

Take the server out of maintenance mode:

# esxcli system maintenanceMode set –enable off

Migrate or startup guests back to the server.

SQLite and fail2ban

OS: CentOS7

I wanted to see exactly how and where the fail2ban IP addresses were stored. Looking in the fail2ban.conf file, I found the following:

dbfile = /var/lib/fail2ban/fail2ban.sqlite3

So, I did a little research to try to find out how access the database.

To open or connect to the database:

# sqlite3 /var/lib/fail2ban/fail2ban.sqlite3

To list all the tables in the database:

sqlite> .tables
bans fail2banDb jails logs

To query a table:

sqlite> SELECT * FROM logs;

Another table:

sqlite> SELECT * FROM bans;

To disconnect from the database:

sqlite> .quit

CentOS – Convert PST to MBOX

To convert a PST file that was created by exporting email from Outlook in Windows, you can use libpst. It is in the CentOS 7 base repository.

# yum install libpst

Then, to convert a PST to MBOX:

# readpst my.pst

This will create an mbox file for each of the folders, including sub-folders if they were exported in Outlook.

GPG basic commands

Export your public key:
gpg –armor -o filename.asc –export localemailaddress

Generate key:
gpg –gen-key

Generate key with more options:
gpg –no-default-keyring –keyring $PWD/.gpg/jgz-keyring –secret-keyring $PWD/.gpg/jgz-secring –trustdb-name $PWD/.gpg/jgz-trustdb –no-random-seed-file –gen-key

Delete private keyring:
gpg –delete-secret-key user # Either email or Real Name in quotes

Delete pub key from keyring:
gpg –delete-key user # Either email or Real Name in quotes

Import public key:
gpg –import publickeyfile.asc

Decrypt a file:
gpg -o filename –decrypt filename.gpg

Encrypt a file:
gpg –output filename.gpg –encrypt –recipient recipeientemailaddress filename # recipientemailaddress is the email address used for a public previously imported.

Edit/trust key:
gpg –edit-key 12345678
gpg> trust

Check an existing key fingerprint:
gpg –fingerprint emailaddress

CentOS 7 change timezone.

To List all timezones:

# timedatectl list-timezones

To set to the new timezone:

# timedatectl set-timezone newtimezone

Sample output:

# timedatectl list-timezones | grep Angeles
# timedatectl set-timezone America/Los_Angeles
# date
Thu Mar 5 11:56:43 PST 2015
# ls -l /etc/localtime
lrwxrwxrwx. 1 root root 41 Mar 5 11:56 /etc/localtime -> ../usr/share/zoneinfo/America/Los_Angeles

To view your current timezone configuration:

# timedatectl status

More netsh firewall command options in Windows.

Here is some very good information from Microsoft for the netsh command in Windows:


I have just listed a few of the commands in both the new version of the command and the old.

Show all the current firewall rules:

netsh advfirewall firewall show rule name=all

To disable or enable all firewall profiles (Domain,Private or Public):

netsh firewall set opmode ENABLE


netsh firewall set opmode DISABLE


netsh advfirewall set allprofiles state on


netsh advfirewall set allprofiles state off

To reset the firewall back to the defaults:

netsh firewall reset


netsh advfirewall reset

To log firewall activity:

netsh firewall set logging “C:\FWLogs\FW.log” 4096 ENABLE ENABLE


netsh advfirewall set currentprofile logging filename “C:\FWLogs\FW.log”

To open or close access to a network port:

netsh firewall add portopening TCP 80 “Open Port 80”


netsh firewall delete portopening TCP 80 “Open Port 80”


netsh advfirewall firewall add rule name=”Open Port 80″ dir=in action=allow protocol=TCP localport=80


netsh advfirewall firewall delete rule name=”Open Port 80″ protocol=tcp localport=80

To enable a program:

netsh firewall add allowedprogram C:\myprograms\myprogram.exe “Allow My Program” ENABLE


netsh advfirewall firewall add rule name=”Allow My Program” dir=in action=allow program=”C:\myprograms\myprogram.exe”

Chroot SFTP only on CentOS 6.

CentOS: 6.6

When setting an SFTP server, you may want to restrict or jail the SFTP users to only one location without restricting all aspects of openssh. This is how I restricted SFTP without impacting all of openssh:

Create the group you will match to and therefore add users to to grant SFTP access:

# groupadd sftp

Create a user:

# useradd -G sftp -d /into -s /sbin/nologin testuser

Notice the home directory. This is the logical root location for the user. Also, note that the shell is nologin to prevent ssh access.

Set the password:

# passwd testuser

Make a backup copy of the sshd_config file and make the following changes to the existing file:

# cp -rp sshd_config sshd_config.orig
# vi sshd_config

# JGZ – Force to use openssh in-process sftp server
#Subsystem sftp /usr/libexec/openssh/sftp-server
Subsystem sftp internal-sftp

# JGZ – Match to group to chroot
Match Group sftp
ChrootDirectory /sftpdir/%u
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp

Restart the service:

# service sshd restart

It is very important that the directory permissions are correct. Create directories and set permissions:

# mkdir /sftpdir
# chmod 755 /sftpdir
# ls -ld /sftpdir
drwxr-xr-x. 3 root root 4096 Feb 27 05:53 /sftpdir
# mkdir /sftpdir/testuser
# chmod 755 /sftpdir/testuser
# ls -ld /sftpdir/testuser/
drwxr-xr-x. 3 root root 4096 Feb 27 14:57 /sftpdir/testuser/
# mkdir /sftpdir/testuser/into
# chown testuser.sftp /sftpdir/testuser/into
# chmod 755 /sftpdir/testuser/into
# ls -ld /sftpdir/testuser/into
drwxr-xr-x. 2 testuser sftp 4096 Feb 27 15:07 /sftpdir/testuser/into/

It should be simple enough to create a script to create new users. Basically, this what you need:
# useradd -G sftp -d /intocbb -s /sbin/nologin testuser1
# mkdir -p /home/testuser1/intocbb
# chown testuser1.sftp intocbb/
# passwd testuser1

Nagios log date conversion

To convert the nagios date timestamp in the nagios.log to a standard time format, use:

perl -pe ‘s/(\d+)/localtime($1)/e’ /var/log/nagios3/nagios.log

Linux “at” command

Long ago I used to use the at command quite often in a Unix environment. I would just use the following syntax:

# at now /path/scriptname

And it worked.

Now, in Ubuntu, when I use the same syntax, I get the following:

syntax error. Last token seen: /
Garbled time

The solution is to enter “at now” and hit enter. At the at> prompt, enter the full path and name of the script you want to run. Then, hit Ctrl-D to run the job and exit out of at.

# at now
warning: commands will be executed using /bin/sh
at> /path/scriptname
job 7 at Thu Oct 30 11:30:00 2014

Inactive VMware ESXi NFS volumes.

Sometimes, when an NFS server is rebooted or off for a while, the volume will remain inactive or inaccessible and greyed out in vCenter/vSphere. To restore an inactive NFS volume in ESXi version 5.x, after obviously verifying that the NFS server is in fact up, do the following from the command line:

List the mounted volumes:

~ # esxcli storage nfs list
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
———– ————- —– ———- ——- ——— ———————
nfsvol1 /nfs1 false true false Unknown
nfsvol2 /nfs2 false true false Unknown

Then, remove the volumes:

~ # esxcli storage nfs remove -v nfsvol1
~ # esxcli storage nfs remove -v nfsvol2
~ # esxcli storage nfs remove -v nfsvol2

List to ensure that all inactive or accessible volumes are gone:

~ # esxcli storage nfs list

Add or mount the storage:

~ # esxcli storage nfs add -H -s /nfs1 -v nfsvol1
~ # esxcli storage nfs add -H -s /nfs2 -v nfsvol2

And list again to verify that the volumes are mounted:

~ # esxcli storage nfs list
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
———– ————- —– ———- ——- ——— ———————
nfsvol1 /nfs1 true true false Not Supported
nfsvol2 /nfs2 true true false Not Supported

Note: You can achieve the same thing by remounting the volumes through the vCenter UI, but why when the command is so much more fun? Besides, in the UI, you might get an error that doesn’t seem to happen from the command line.

Return top