{"id":1228,"date":"2015-04-30T10:30:07","date_gmt":"2015-04-30T17:30:07","guid":{"rendered":"http:\/\/jim-zimmerman.com\/?p=1228"},"modified":"2015-04-30T10:30:07","modified_gmt":"2015-04-30T17:30:07","slug":"vmware-esxi-and-hp-dl380-heap-issue","status":"publish","type":"post","link":"https:\/\/jim-zimmerman.com\/?p=1228","title":{"rendered":"VMWare ESXi and HP DL380 Heap issue."},"content":{"rendered":"

I was getting the following message when I would try to start up a VM on an HP DL380. <\/p>\n

Heap globalCartel-1 already at its maximum size of 7869288. Cannot expand. Could not start VMX: msg.vmk.status.VMK_NO_MEMORY<\/p><\/blockquote>\n

The machine had plenty of memory and seemed to operating fine for currently running VMs. When I brought up the direct console on the machine, there was screen full of the following:<\/p>\n

\/bin\/sh: can’t fork<\/p><\/blockquote>\n

I could not enter any commands other than “exit.” I could log in and out, and that is about it.<\/p>\n

In VMWare KB article 2085618, it acknowledged the issue with the HP specific image and the HP AMS driver.<\/p>\n

Updated the HP AMS driver per VMWare KB article 2085618.<\/p>\n

To determine version running:<\/p>\n

esxcli software vib list | grep ams<\/p><\/blockquote>\n

Old version:<\/p>\n

# esxcli software vib list | grep ams
\nhp-ams 550.10.0.0-18.1198610 Hewlett-Packard PartnerSupported 2014-11-07 <\/p><\/blockquote>\n

Download updated driver from HP:
\nhp-ams-esxi5.5-bundle-10.0.1-2.zip<\/a><\/p>\n

Then, I used scp to copy the downloaded zip file to my ESXi 5.5 server. I copied it to a hp-ams-esxi5.5-bundle-10.0.1.2 directory I created. Directory: \/vmfs\/volumes\/vmhost02-datastore1\/Software\/hp-ams-esxi5.5-bundle-10.0.1-2. <\/p>\n

Login to the ESXi host using ssh.<\/p>\n

# cd \/vmfs\/volumes\/vmhost02-datastore1\/Software\/hp-ams-esxi5.5-bundle-10.0.1-2<\/p><\/blockquote>\n

Unzip the file:<\/p>\n

# unzip hp-ams-esxi5.5-bundle-10.0.1-2.zip
\nArchive: hp-ams-esxi5.5-bundle-10.0.1-2.zip
\n inflating: index.xml
\n inflating: vendor-index.xml
\n inflating: metadata-hp-ams-esxi5.5-bundle-10.0.1-2.zip
\n inflating: hp-esxi-fc-enablement-550.2.1.8-1198610.vib
\n inflating: hp-smx-limited-550.03.06.00.22-1198610.vib
\n inflating: char-hpilo-550.9.0.2.3-1OEM.550.0.0.1198610.x86_64.vib
\n inflating: char-hpcru-5.5.6.6-1OEM.550.0.0.1198610.x86_64.vib
\n inflating: hp-ams-550.10.0.1-07.1198610.vib\n<\/p><\/blockquote>\n

Files contained in the zip:<\/p>\n

# ls -l
\ntotal 19456
\n-rw-r–r– 1 root root 13590 Apr 29 12:15 char-hpcru-5.5.6.6-1OEM.550.0.0.1198610.x86_64.vib
\n-rw-r–r– 1 root root 12676 Apr 29 12:15 char-hpilo-550.9.0.2.3-1OEM.550.0.0.1198610.x86_64.vib
\n-rw-r–r– 1 root root 1271770 Apr 29 12:15 hp-ams-550.10.0.1-07.1198610.vib
\n-rw-r–r– 1 root root 6829546 Apr 29 12:15 hp-ams-esxi5.5-bundle-10.0.1-2.zip
\n-rw-r–r– 1 root root 1213780 Apr 29 12:15 hp-esxi-fc-enablement-550.2.1.8-1198610.vib
\n-rw-r–r– 1 root root 4482274 Apr 29 12:15 hp-smx-limited-550.03.06.00.22-1198610.vib
\n-rw——- 1 root root 328 Apr 29 12:15 index.xml
\n-rw-r–r– 1 root root 8531 Apr 29 12:15 metadata-hp-ams-esxi5.5-bundle-10.0.1-2.zip
\n-rw——- 1 root root 238 Apr 29 12:15 vendor-index.xml<\/p><\/blockquote>\n

Shutdown or migrate all the guests on the ESXi server, and then put the server in maintenance mode:<\/p>\n

# esxcli system maintenanceMode set –enable on<\/p><\/blockquote>\n

Update the software:<\/p>\n

# esxcli software vib update -v \/vmfs\/volumes\/542c3aae-03b91418-ff2a-a0d3c1f0cdf0\/Software\/hp-ams-esxi5.5-bundle-10.0.1-2\/hp
\n-ams-550.10.0.1-07.1198610.vib
\nInstallation Result
\n Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
\n Reboot Required: true
\n VIBs Installed: Hewlett-Packard_bootbank_hp-ams_550.10.0.1-07.1198610
\n VIBs Removed: Hewlett-Packard_bootbank_hp-ams_550.10.0.0-18.1198610
\n VIBs Skipped: <\/p><\/blockquote>\n

At this point, you need to reboot the ESXi host.<\/p>\n

New version:
\n# esxcli software vib list | grep ams
\nhp-ams 550.10.0.1-07.1198610 Hewlett-Packard PartnerSupported 2015-04-29 <\/p>\n

Take the server out of maintenance mode:<\/p>\n

# esxcli system maintenanceMode set –enable off<\/p><\/blockquote>\n

Migrate or startup guests back to the server.<\/p>\n","protected":false},"excerpt":{"rendered":"

I was getting the following message when I would try to start up a VM on an HP DL380. Heap globalCartel-1 already at its maximum size of 7869288. Cannot expand. Could not start VMX: msg.vmk.status.VMK_NO_MEMORY The machine had plenty of memory and seemed to operating fine for currently running VMs. When I brought up the [&hellip<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[397,153,395,396,394,33],"class_list":["post-1228","post","type-post","status-publish","format-standard","hentry","category-documentation","tag-dl380","tag-esxi","tag-heap","tag-hp","tag-vmk_no_memory","tag-vmware"],"share_on_mastodon":{"url":"","error":""},"_links":{"self":[{"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/posts\/1228","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1228"}],"version-history":[{"count":7,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/posts\/1228\/revisions"}],"predecessor-version":[{"id":1237,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=\/wp\/v2\/posts\/1228\/revisions\/1237"}],"wp:attachment":[{"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1228"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1228"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jim-zimmerman.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1228"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}