Wednesday, 11 February 2015

Strange NFS problem topic






I'm running VMWare ESXi 5.1 as a host.

Within that host I have several virtual machines running:

  • **NiY** My main server (CentOS release 5.10 (Final))
  • **downloadstation** My download server (Ubuntu 14.04.1 LTS)
  • And several other which are not important


I also have a HTPC which boots the xbmcbuntu iso, using TFTP, from NiY and mounts an additional NFS share for copy on write.

And downloadserver mounts a NFS share from NiY

My problem is that since i upgraded downloadserver from Ubuntu 12 to Ubuntu 14, it can no longer mount the NFS shares from NiY. The HTPC still can mount all the NFS shares.

What I have tried so far:

  • Pinging all machines from all machines by hostname and IP, this works
  • Granting all access in /etc/exports
  • Disabling iptables on NiY (downloadserver is not running iptables)
  • Using showmount to inspect the exports from NiY, this works: http://pug205.nl/stackoverflow/showmount.jpg
  • Mounting the nfs share on NiY, this works
  • Installing a fresh virtual machine with Ubuntu 12 and mount, does **not** work
  • Installing a fresh virtual machine with CentOS 7 and mount, does **not** work
  • Mount my NAS via NFS, this **does** work from **all** machines
  • Checking logs on NiY, they are not mentioning any thing related to NFS
  • logs on downloadstation contain some information


Code:


RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
FS-Cache: Netfs 'nfs' registered for caching
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
init: idmapd-mounting (/mnt/media) main process (296) killed by TERM signal
init: statd-mounting (/mnt/media) main process (297) killed by TERM signal
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
init: failsafe main process (542) killed by TERM signal
nfs: server {IP NiY} not responding, still trying


The last line keeps appearing in the logs


/etc/exports:

Code:


    #XBMC Frodo 12.2 XBMCBUNTU
    /export/XBMC/xbmcbuntu-12.2.Intel-NVIDIA *(ro,nohide,async,mp,no_root_squash,insecure,no_subtree_check)
    /export/nfsroot/xbmcbuntu-12.2.Intel-NVIDIA/{MAC ADDRESS HTPC} {IP HTPC}(rw,nohide,async,no_root_squash,insecure,no_subtree_check)

#XBMC Gotham 13.0 XBMCBUNTU
    /export/XBMC/xbmcbuntu-13.0~gotham_amd64 * (ro,nohide,async,mp,no_root_squash,insecure,no_subtree_check)
    /export/nfsroot/xbmcbuntu-13.0~gotham_amd64/{MAC ADDRESS HTPC} {IP HTPC (rw,nohide,async,no_root_squash,insecure,no_subtree_check)

#Media share
    /mnt/data/media *(rw,nohide,sync,no_root_squash,no_subtree_check)


I tried mounting with verbose logging and running Wireshark to check if anything did go to NiY (either by hostname or ip address):

Code:


root@downloadstation:~# mount -vvv -t nfs niy:/mnt/data/media /mnt/Download/
    mount: fstab path: "/etc/fstab"
    mount: mtab path:  "/etc/mtab"
    mount: lock path:  "/etc/mtab~"
    mount: temp path:  "/etc/mtab.tmp"
    mount: UID:        0
    mount: eUID:      0
    mount: spec:  "niy:/mnt/data/media"
    mount: node:  "/mnt/Download/"
    mount: types: "nfs"
    mount: opts:  "(null)"
    mount: external mount: argv[0] = "/sbin/mount.nfs"
    mount: external mount: argv[1] = "niy:/mnt/data/media"
    mount: external mount: argv[2] = "/mnt/Download/"
    mount: external mount: argv[3] = "-v"
    mount: external mount: argv[4] = "-o"
    mount: external mount: argv[5] = "rw"
    mount.nfs: timeout set for Wed Jan 28 01:30:59 2015
    mount.nfs: trying text-based options 'vers=4,addr={IP NiY},clientaddr={IP downloadstation}'


And it just hangs until i press [CTRL] + [C]

This is the output from Wireshark running on NiY with the previous mount command (only showing traffic from and to downloadstation)


Code:


No.    Time        Source              Destination        Protocol Info
        514 3.258878    IP downloadstation  IP NiY              NFS      V4 COMP Call <EMPTY> PUTROOTFH PUTROOTFH;GETFH GETFH;GETATTR GETATTR
        515 3.258898    IP NiY              IP downloadstation  TCP      nfs > 859 [ACK] Seq=1 Ack=121 Win=46 Len=0 TSV=314460502 TSER=78612280
      1135 35.288077  IP downloadstation  IP NiY              SMB      Echo Request
      1136 35.288140  IP NiY              IP downloadstation  SMB      Echo Response
      1137 35.288223  IP downloadstation  IP NiY              TCP      37709 > microsoft-ds [ACK] Seq=43 Ack=43 Win=2296 Len=0 TSV=78620288 TSER=314492532
      1681 63.319756  IP downloadstation  IP NiY              TCP      [TCP Keep-Alive] 859 > nfs [ACK] Seq=120 Ack=1 Win=229 Len=0 TSV=78627296 TSER=314460502
      1682 63.319769  IP NiY              IP downloadstation  TCP      [TCP Keep-Alive ACK] nfs > 859 [ACK] Seq=1 Ack=121 Win=46 Len=0 TSV=314520564 TSER=78612280
      1683 63.319790  IP downloadstation  IP NiY              TCP      859 > nfs [FIN, ACK] Seq=121 Ack=1 Win=229 Len=0 TSV=78627296 TSER=314460502
      1684 63.358905  IP NiY              IP downloadstation  TCP      nfs > 859 [ACK] Seq=1 Ack=122 Win=46 Len=0 TSV=314520604 TSER=78627296
      1944 78.359607  IP downloadstation  IP NiY              TCP      859 > nfs [RST, ACK] Seq=122 Ack=1 Win=229 Len=0 TSV=78631056 TSER=314520604
      1945 78.359639  IP downloadstation  IP NiY              TCP      [TCP Port numbers reused] 859 > nfs [SYN] Seq=0 Win=29200 Len=0 MSS=1460 TSV=78631056 TSER=0 WS=7
      1946 78.359654  IP NiY              IP downloadstation  TCP      nfs > 859 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=314535605 TSER=78631056 WS=7
      1947 78.359715  IP downloadstation  IP NiY              TCP      859 > nfs [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSV=78631056 TSER=314535605
      1948 78.359737  IP downloadstation  IP NiY              NFS      V4 COMP Call <EMPTY> PUTROOTFH PUTROOTFH;GETFH GETFH;GETATTR GETATTR
      1949 78.359743  IP NiY              IP downloadstation  TCP      nfs > 859 [ACK] Seq=1 Ack=121 Win=5888 Len=0 TSV=314535605 TSER=78631056
      1950 78.359760  IP downloadstation  IP NiY              NFS      V4 COMP Call <EMPTY> PUTROOTFH PUTROOTFH;GETFH GETFH;GETATTR GETATTR
      1951 78.359763  IP NiY              IP downloadstation  TCP      nfs > 859 [ACK] Seq=1 Ack=241 Win=5888 Len=0 TSV=314535605 TSER=78631056
      5308 95.447441  IP downloadstation  IP NiY              SMB      Echo Request
      5309 95.447488  IP NiY              IP downloadstation  SMB      Echo Response
      5310 95.447545  IP downloadstation  IP NiY              TCP      37709 > microsoft-ds [ACK] Seq=85 Ack=85 Win=2296 Len=0 TSV=78635328 TSER=314552692
      13721 138.455024  IP downloadstation  IP NiY              TCP      [TCP Keep-Alive] 859 > nfs [ACK] Seq=240 Ack=1 Win=29312 Len=0 TSV=78646080 TSER=314535605
      13722 138.455045  IP NiY              IP downloadstation  TCP      [TCP Keep-Alive ACK] nfs > 859 [ACK] Seq=1 Ack=241 Win=5888 Len=0 TSV=314595700 TSER=78631056
      16183 155.606860  IP downloadstation  IP NiY              SMB      Echo Request
      16184 155.606928  IP NiY              IP downloadstation  SMB      Echo Response
      16185 155.607035  IP downloadstation  IP NiY              TCP      37709 > microsoft-ds [ACK] Seq=127 Ack=127 Win=2296 Len=0 TSV=78650368 TSER=314612853


Any help?






No comments:

Post a Comment