I recently came across a customer who was trying to download .vmem from the vSAN datastore, so that the memory dump can be shared with the Guest OS vendor like Microsoft for core dump analysis . Since vSAN datastore is an object store file system , the OSFS namespace will only contain the pointer file of about 118 bytes which points to the associated object within vSAN datastore similar to all the VMDK files and v-swap file .
The following work around helps you to download the vmem file from the vsan datastore .
Option 1 :
If we have a local datastore which should have equivalent to the size of the .vmem file (memory size of the virtual machine) or greater free space available then
- Browse the datastore , select the .vmem file .
- Next drag and drop the .vmem file from the vSAN datastore to the local datastore .This will initiate a full copy of the pointer file and the associated vSAN object to the local datastore
- Next download the file from the local datastore by either from webclient GUI or WINSCP , this file can be later shared with respective OS vendor for debugging.
Option 2 :
If the vSAN cluster does not have a local datastore configured . We will need to follow below steps to download the file .
- To start first Suspend the VM following the steps mentioned in the KB :- https://kb.vmware.com/kb/2005831
- This will create a vmss and a .vmem file for Debugging
- Log on to the Esxi host using a Putty, where the VM in question is registered and running
- Navigate to Virtual machine directory location for example “cd /vmfs/volumes/vsan:524fcc9a876dcefd-be39dfd52bd37e0e/<VM directory>”
- Run a command “cat” against the .vmem file :- cat <file name>.vmem
- Retrieve the Object ID information as show in the output below :
[root@host4:/vmfs/volumes/vsan:524fcc9a876dcefd-be39dfd52bd37e0e/94231059-ce43-435b-58f5-0050560150c2] cat vm-win-crash-5675fa8a.vmem
# Object DescriptorFile
version = “1”
objectID = “vsan://e6a82759-adc5-24b5-1853-0050560150c2” –> Use the Object ID to copy
object.class = “vmem”
- Then run the command :- /usr/lib/vmware/osfs/bin/objtool open -u <object ID> , see example below :
In this case the command is :- /usr/lib/vmware/osfs/bin/objtool open -u e6a82759-adc5-24b5-1853-0050560150c2
====================================================================================================================================
[root@host4: /usr/lib/vmware/osfs/bin/objtool open -u e6a82759adc5-24b5-1853-0050560150c2
Object opened at /vmfs/devices/vsan/ e6a82759-adc5-24b5-1853-0050560150c2 –> Shows opened
[root@host4:/vmfs/volumes/vsan:524fcc9a876dcefd-be39dfd52bd37e0e/94231059-ce43-435b-58f5-0050560150c2]
====================================================================================================================================
- Then use “winscp”, logon to the Esxi host, browse to the file path which is /vmfs/devices/vsan/e6a82759-adc5-24b5-1853-0050560150c2 to move the file to the local location or remote location of the machine from which you have logged in .
- Once the file has been copied over to your local system , you will need to rename the file extension to .vmem so that Microsoft or any vendor can use this file for Debugging purpose .