NFS v3 datastore and Failed to lock byte 100


Currently I use NFS v4 as a datastore. The backend of datastore is an EMC Data Domain and seems that linux nfs client has an issue with it.
The ‘nfs: server XXX not responding, timed out’ messages are often observed at kvm nodes’ logs.
So I’ve decided to switch to NFS v3 for the test purpose and faced another problem, the KVM’s VM has failed to start:

Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/101/45/deployment.15' 'one-kvm-node-01-int' 45 one-kvm-node-01-int
error: Failed to create domain from /var/lib/one//datastores/101/45/deployment.15
error: internal error: qemu unexpectedly closed the monitor: 2019-07-24T14:38:09.754413Z qemu-kvm: -drive file=/var/lib/one//datastores/101/45/disk.0,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Failed to lock byte 100
Could not create domain from /var/lib/one//datastores/101/45/deployment.15
ExitCode: 255
Failed to execute virtualization driver operation: deploy.
Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/101/45/deployment.15

NFS v4 mount options at /etc/fstab:

NFS v3 mount options at /etc/fstab:

Any assistance with a ‘Failed to lock byte 100’ issue is more than welcome :slight_smile:

Versions of the related components and OS (frontend, hypervisors, VMs):
KVM nodes run under CentOS kernel version 3.10.0-957.21.3.el7.x86_64
Opennebula version 5.8.1

Steps to reproduce:
Start the VM

Current results:
KVM VM does not start within NFS v3 datastore
Switched back to NFS v4
Expected results:
VM up and running

Seems I’ve solved this problem with the following mount options on the client (kvm nodes) side:


Also I tunned the following kernel options (on the clients):

echo 262144 > /proc/sys/net/core/rmem_max
echo 262144 > /proc/sys/net/core/wmem_max
echo 262144 > /proc/sys/net/core/rmem_default
echo 262144 > /proc/sys/net/core/wmem_default
echo '8192 524288 2097152' > /proc/sys/net/ipv4/tcp_rmem
echo '8192 524288 2097152' > /proc/sys/net/ipv4/tcp_wmem

I found this topic in google.
For other guys who also will find it: this is the issue with locking on nfs server side.
I faced with the same issue (Failed to lock byte 101) with Debian (client) -> FreeBSD (server). And the topic starter’s solution works fine to me. But it also helped me to look more carefully on the settings of my environment. And i found that i forgot to turn on lockd and statd on the server side. I have fixed that and now i can mount without adding extra options.