Busy Device on Umount
I often experience a problem to umount a directory:
umount /mnt/dir umount: /mnt/dir: device is busy
There are many reasons why the device is busy. Sometimes there are processes running which have open locks on it, sometimes there are other directories mounted on top of
What are the steps to check why a directory couldn't be unmounted.
I know there are many reasons, but it's ok if you explain a specific solution.
[X] running processes on mounted volumes.
[X] another volume is mounted on top of a volume we want to unmount
[_] NFS locks the volume we want to unmount
The way to check is
fuser -vm /mnt/dir, which must be run as root. It will tell you which processes are accessing the mount point.
An alternative is
lsof /mnt/dir, which will show each open file on the mount. Again best run as root.
You can run either of these as non-root, but then the output will be limited to your processes—ones from other users will just be silently not shown, even though they will prevent unmounting the filesystem.
Watt:~# fuser -vm /mnt/Zia/src USER PID ACCESS COMMAND /mnt/Zia/src: root kernel mount /mnt/Zia/src anthony 24909 ..c.. bash anthony 25041 F.c.. gvim
The "access" field tells you how its being accessed. In this case, the kernel has it in use as a mount (duh, but unmount will be OK with only this).
bashhas it as the current working directory (will have to
cdto a different directory before unmount) and gvim both has the current directory and has a file open (will need to close that gvim).
Watt:~# lsof /mnt/Zia/src COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME bash 24909 anthony cwd DIR 0,26 12288 3527682 /mnt/Zia/src/perl (zia.vpn.home:/home/anthony/src) gvim 25041 anthony cwd DIR 0,26 12288 3527682 /mnt/Zia/src/perl (zia.vpn.home:/home/anthony/src) gvim 25041 anthony 6u REG 0,26 16384 3526219 /mnt/Zia/src/perl/.utf8.c.swp (zia.vpn.home:/home/anthony/src)
In this output, you can see the current directories for both bash and gvim (as type
DIR). You can also see which file gvim has open for write.
How to force the issue:
-koption which will send a signal (default:
SIGKILL) to each process using the mount. This is a rather forceful way to stop the mount from being busy. (And of course, be careful of what you
-loption to perform a lazy unmount. The mount will be removed from the filesystem namespace (so you won't see it under
/mnt/Zia/srcanymore, in the example) but it stays mounted, so programs accessing it can continue to do so. When the last program accessing it exits, the unmount will actually occur.
There is one final fixable cause of unmount failing, and that's an NFS server going down. Here you can use
umount -f, but you risk data loss if you do so. (The client may have cached writes that haven't been confirmed by the server yet, and those writes will be discarded. Apps, however, have already been told the write is successful.)
Note that `fuser -k` is **extremely** risky, as you'll be doing it as root and if you're not **very** sure of which processes will be killed off you can do truly spectacular damage with a careless command...
@Shadur well, hopefully you've already run it without the `-k` option, so you'll know which processes you're going to kill. But I'll add in a warning.
`fuser -vm` showed "kernel mount". had to do `systemctl stop opt.mount` instead of manual `umount`.
For some reason umount -f doesn't work for me but running umount -l works perfectly.
Thanks for the note about `umount -f`and NFS. My issue was NFS related where my dev machines changed IPs and I could not get the share removed.
One note "F...." doesn't seem to block the unmount (I thought it would), but like you have above having the "c" does. Not sure why this is.
There's an advantage to using `lsof +f /dev/device` rather than `/mountpoint`. See the same for kernel processes which need other methods.
You should use:
sudo umount -l <path>
From the man page:
- Lazy unmount. Detach the filesystem from the file hierarchy now, and clean up all references to this filesystem as soon as it is not busy anymore.
A system reboot would be expected in near future if you're going to use this option for network filesystem or local filesystem with submounts. The recommended use-case for
umount -lis to prevent hangs on shutdown due to an unreachable network share where a normal umount will hang due to a downed server or a network partition. Remounts of the share will not be possible.
⁺¹, I've no idea what silly person could downvote it. The `-l` is exactly the option to use when even `-f` doesn't work.
Another volume is mounted on top of a volume we want to unmount :
mountcommand lets you know all the mounted volumes if invoqued without arguments nor options (except
-v). You can have a list of active mountpoints by adding a bit of perl :
mount | perl -pe 's/.*on (\S+) type.*/\1/'
Then, just grep over the mointpoint from which you wish to unmount and you will know if there is mounted filesystems over this one.
mount | perl -pe 's/.*on (\S+) type.*/\1/' | grep '/mnt/dir/'
Then you have two solutions. Either unmount the filesystems, or move them with
mount --move olddir newdir(kernel >2.5.1)
Yes, thanks. /etc/mtab and /proc/mounts are also possible.
Ah that's right, I always forgot those. Let's say that typing "mount" requires less characters (but more resources to execute?)
I have an external USB storage device "permanently" mounted on a certain directory on my laptop, Sometimes the cable gets disconnected by mistake. It was a big pain to remount the device on the directory (because of "device busy") until I read this answer. Now I known to use mount --move olddir newdir. Thanks.
Processes with open files are the usual culprits. Display them:
lsof +f -- <mountpoint or device>
There is an advantage to using
/mountpoint: a mountpoint will disappear after an
umount -l, or it may be hidden by an overlaid mount.
fusercan also be used, but to my mind
lsofhas a more useful output. However
fuseris useful when it comes to killing the processes causing your dramas so you can get on with your life.
List files on
<mountpoint>(see caveat above):
fuser -vmM <mountpoint>
Interactively kill only processes with files open for writing:
fuser -vmMkiw <mountpoint>
After remounting read-only (
mount -o remount,ro <mountpoint>), it is safe(r) to kill all remaining processes:
fuser -vmMk <mountpoint>
The culprit can be the kernel itself. Another filesystem mounted on the filesystem you are trying to
umountwill cause grief. Check with:
mount | grep <mountpoint>/
For loopback mounts, also check the output of:
Anonymous inodes (Linux)
Anonymous inodes can be created by:
- Temporary files (
- inotify watches
These are the most elusive type of pokemon, and appear in
a_inode(which is undocumented in the
They won't appear in
lsof +f -- /dev/<device>, so you'll need to:
lsof | grep a_inode
For killing processes holding anonymous inodes, see: List current inotify watches (pathname, PID).
- Temporary files (
The question how to check if NFS access a directory which is going to unmount is still unanswered.
What I have is only this:
Check if nfsd is running:
Show mounted directories by clients:
showmountw/o arguments shows only client hosts even if they are off line. I assume this is a special behavior of NFS.