In order to implement DataCore storage with vSphere ESXi, you need to follow some best practices.
It is true for both iSCSI and FC.

You will find all details regarding those best practices in the DataCore support website, here is the link : VMware ESXi Configuration Guide

This article’s goal is to summarize these information and give some tips regarding the implementation.

All the settings which are described in this article, and more specifically the multipathing rules, have to be applied before serving Virtual Disk to ESXi hosts. If you need to change the settings after, you will have to check the DataCore Configuration Guide.

ESXi settings

Three settings need to be checked here:

• The Disk.DiskMaxIoSize advanced setting
• The VAAI configuration
• DalyedACK, if you a dealing with an iSCSI architecture

Advanced Settings: Disk.DiskMaxIoSize

It describes the IO size which is transmitted to the storage, by default it is set to 32767 KB, DataCore recommendation is to set it to 512 KB.

It’s no longer needed to change parameters such as UseLunReset or CRTimeoutDuringBoot anymore (for those who remember SANmelody!))

Changing through PowerCLI

Checking the parameter for all the hosts in a cluster (replace “Cluster” by your Cluster name):

1

PS> Get-Cluster Cluster | Get-VMHost | Select-Object @{Name="ESXi";Expression={$_.Name}}, @{Name="DiskMaxIoSize";Expression={$_ | Get-AdvancedSetting -Name "Disk.DiskMaxIoSize" | Select-Object -ExpandProperty Value}}

Set the parameter to 512:

1

PS> Get-Cluster Cluster | Get-VMHost | Foreach-Object {$_ | Get-AdvancedSetting -Name "Disk.DiskMaxIoSize" | Set-AdvancedSetting -Value 512 -Confirm:$false}

Changing it through Web Client

It is also possible to change the setting though the Web Client on the vCenter or on the ESXi itself.

Advanced settings:

Setting the parameter:

VAAI Configuration: ATS for VMFS Heartbeat

I will intentionally not focus on VAAI here, I will probably publish a dedicated article later.

There was a change in vSphere ESXi 5.5U2 and 6.0 on the way ESXi is dealing with VMFS Heartbeat. Nowon, ATS (Atomic Test and Set) is used instead if SCSI Read/Write commands.

ATS for VMFS Heartbeat is not supported with SANsymphony, therefore, it should be disactivated:

It is specified VMFS5 but it also valid for VMFS6: https://kb.vmware.com/s/article/2113956

The setting can be change through PowerCLI for the whole Cluster

Checking the parameter for all the hosts in a cluster:

1

PS> Get-Cluster Cluster | Get-VMHost | Select-Object @{Name="ESXi";Expression={$_.Name}}, @{Name="ATS Heartbeat";Expression={$_ | Get-AdvancedSetting -Name "VMFS3.UseATSForHBOnVMFS5" | Select-Object -ExpandProperty Value}}

Disabling ATS Heartbeat for the whole cluster:

1

PS> Get-Cluster Cluster | Get-VMHost | Foreach-Object {$_ | Get-AdvancedSetting -Name "VMFS3.UseATSForHBOnVMFS5" | Set-AdvancedSetting -Value 0 -Confirm:$false}

Disabling DelayedACK

Most of the time, when you are using an iSCSI storage, it is advised to disable DelayedAck as it can induce latency. DataCore is not making an exception here, so you will have to disable it in order to maintain a consitant latency:

To check the value:

1

PS> Get-Cluster Cluster | Get-VMHost | Select-Object @{Name="ESXi";Expression={$_.Name}}, @{Name="DelayedAck";Expression={$_ | Get-VMHostHba | Where-Object Model -match "Software" | Foreach-Object {$_.ExtensionData.AdvancedOptions | Where-Object Key -eq "DelayedAck" | Select-Object -ExpandProperty Value}}}

Disabling it for the cluster:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

#This section will get host information needed  
$HostView = Get-VMHost nameofesxhost | Get-View  
$HostStorageSystemID = $HostView.configmanager.StorageSystem  
$HostiSCSISoftwareAdapterHBAID = ($HostView.config.storagedevice.HostBusAdapter | where {$_.Model -match "iSCSI Software"}).device  
  
#This section sets the option you want.  
$options = New-Object VMWare.Vim.HostInternetScsiHbaParamValue[] (1)  
  
$options[0] = New-Object VMware.Vim.HostInternetScsiHbaParamValue  
$options[0].key = "DelayedAck"  
$options[0].value = $false  

#This section applies the options above to the host you got the information from.  
$HostStorageSystem = Get-View -ID $HostStorageSystemID  
$HostStorageSystem.UpdateInternetScsiAdvancedOptions($HostiSCSISoftwareAdapterHBAID, $null, $options)

Multipathing

The multipathing configuration is always an important topic when it comes to vSphere storage.
Two documents should be taken into account:

• The VMware Hardware Compatibility List (HCL)
• The DataCore VMware ESXi Configuration Guide

There are three different Multipathing rules by default on an ESXi (PSP):

• Fixed
• Round Robin
• Most Recently Used

So the question is: which one should I use ?

HCL VMware

On VMware side, there are two relevant points:

ESXi Configuration Guide

In the DataCore Configuration Guide for ESXi, we can see the supported version with and without ALUA:

Regarding the multipathing rule to use, the document is not very clear and let you make your own decision “DataCore does not recommend to use a particular PSP. Choose the PSP that is appropriate for your configuration."

MRU (Most Recently Used) is indicated as not tested (and we can’t find it in the VMware HCL):

And most important, MRU in association with ALUA is not supported! For MRU, Hosts must not have the ALUA setting.

In conclusion : Dot NOT use MRU in your DataCore/ESXi configurations!

Fixed was the preferred option with old DataCore releases (SANmelody), in particular with iSCSI. Since SANsymphony 9.x, we tend to use Round Robin most of the time. In association with ALUA, the configuration on ESXi side is simplified as we don’t have to set preferred path manually anymore. I will try to do a dedicated an article on the advantages of using ALUA with Round Robin.

Round Robin is only supported with ALUA (which will be set on DataCore side)

So now on, we will make sure to use Round Robin with ALUA.

Default configuration

As we saw previously, MRU should not be used with ALUA but it is the default ESXi configuration!

For each SATP (Storage Array Type Plug-ins) module, ESXi will associate a specific PSP (Path Selection Policy). The default configuration is (for ESXi 6.x):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

[root@esx01:~] esxcli storage nmp satp list
Name                 Default PSP    Description
-------------------  -------------  -----------------------------------------
VMW_SATP_MSA         VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_ALUA        VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AP  VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_SVC         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_EQL         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_INV         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_EVA         VMW_PSP_FIXED  Placeholder (plugin not loaded)
VMW_SATP_ALUA_CX     VMW_PSP_RR     Placeholder (plugin not loaded)
VMW_SATP_SYMM        VMW_PSP_RR     Placeholder (plugin not loaded)
VMW_SATP_CX          VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_LSI         VMW_PSP_MRU    Placeholder (plugin not loaded)
VMW_SATP_DEFAULT_AA  VMW_PSP_FIXED  Supports non-specific active/active arrays
VMW_SATP_LOCAL       VMW_PSP_FIXED  Supports direct attached devices

On the second row, we can see MRU is chosen for hosts configured with ALUA. This default configuration can be changed with the following command line:

1
2

[root@esx01:~] esxcli storage nmp satp set --satp VMW_SATP_ALUA --default-psp VMW_PSP_RR 
Default PSP for VMW_SATP_ALUA is now VMW_PSP_RR

It is a possibility, but it should only be considered if you have just DataCore storage, even though I recommend creating a custom rule for DataCore.

Dedicated DataCore rule creation

We can display the rule already created:

1

[root@esx01:~] esxcli storage nmp satp rule list

And then create a custom rule:

1

[root@esx01:~] esxcli storage nmp satp rule add -V DataCore -M "Virtual Disk" -s VMW_SATP_ALUA -c tpgs_on -P VMW_PSP_RR -O iops=10

The IOPS parameter is only available with Round Robin, no matter the storage used with VMware, it is rather often recommended to lower it (default value is 1000). Please refer to your storage vendor best practices guide.

DataCore ALUA configuration

Last step, during the process of creating the host in the DataCore console, you need to activate ALUA (if you are using an up-to-date version of SANsymphony it should be activated by default). You also need to chose a Preferred server:

The Preferred server is the DataCore node which is in the same “side” as your host. It is used to set a weight on both path within the same datacenter.

Enjoy :)