Tracking down where disk space has gone on Linux?
When administering Linux systems I often find myself struggling to track down the culprit after a partition goes full. I normally use
du / | sort -nrbut on a large filesystem this takes a long time before any results are returned.
Also, this is usually successful in highlighting the worst offender but I've often found myself resorting to
sortin more subtle cases and then had to trawl through the output.
I'd prefer a command line solution which relies on standard Linux commands since I have to administer quite a few systems and installing new software is a hassle (especially when out of disk space!)
@Bart thanks for taking the time to improve posts here, but may I ask you to take a little more care when adding tags? Tags aren’t for visibility, they describe the question. The disk tag isn’t appropriate here (see its tag description), and the at least two of the tags you added in this suggested edit weren’t appropriate there (Kali isn’t Debian, and there are no PPAs involved).
ncdu, an excellent command-line disk usage analyser:
Typically, I hate being asked to install something to solve a simple issue, but this is just great.
I was looking for a fast way to find what takes up disk space in an ordered way. This tool does it and it also provides sorting and easy navigation. Thank you for the reference.
You quite probably know which filesystem is short of space. In which case you can use `ncdu -x` to only count files and directories on the same filesystem as the directory being scanned.
best answer. also: `sudo ncdu -rx /` should give a clean read on biggest dirs/files ONLY on root area drive. (`-r` = read-only, `-x` = stay on same filesystem (meaning: do not traverse other filesystem mounts) )
@Alf47 Required header for what? You list only partial error. You are missing a lib dependency. Maybe try installing ncurses lib. That seems to be the usual culprit. The info is there in the build output on what your system is missing. see: https://unix.stackexchange.com/a/113493/186861
@bshea had a great suggestion, many times on AWS it's only your root filesystem that is small, everything else is an EBS or EFS mount that is huge, so you only need to find and clean the root partition.
Error, can't install ncdu, E: You don't have enough free space in /var/cache/apt/archives/. :(
This is like WinDirStat for Linux users - absolutely perfect for evaluating disk consumption and treating out-of-control scenarios.
Don't go straight to
du /. Use
dfto find the partition that's hurting you, and then try
One I like to try is
# U.S. du -h <dir> | grep '[0-9\.]\+G' # Others du -h <dir> | grep '[0-9\,]\+G'
because it prints sizes in "human readable form". Unless you've got really small partitions, grepping for directories in the gigabytes is a pretty good filter for what you want. This will take you some time, but unless you have quotas set up, I think that's just the way it's going to be.
As @jchavannes points out in the comments, the expression can get more precise if you're finding too many false positives. I incorporated the suggestion, which does make it better, but there are still false positives, so there are just tradeoffs (simpler expr, worse results; more complex and longer expr, better results). If you have too many little directories showing up in your output, adjust your regex accordingly. For example,
is even more accurate (no < 1GB directories will be listed).
If you do have quotas, you can use
to find users that are hogging the disk.
`grep '[0-9]G'` contained a lot of false positives and also omitted any decimals. This worked better for me: `sudo du -h / | grep -P '^[0-9\.]+G'`
@jchavannes `-P` is unnecessary for this expression because there's nothing specific to Perl there. Also, `-P` isn't portable to systems that don't have the GNU implementation.
Ahh. Well having a carat at the beginning will remove false positives of directories which have a number followed by a G in the name, which I did.
In case you have really big directories, you'll want `[GT]` instead of just `G`
Is there a tool that will continuously monitor disk usage across all directories (lazily) in the filesystem? Something that can be streamed to a web UI? Preferably soft-realtime information.
For a first look, use the “summary” view of
du -s /*
The effect is to print the size of each of its arguments, i.e. every root folder in the case above.
GNU (Linux, …):
du --max-depth 3
BSD (macOS, …):
du -d 3
This will limit the output display to depth 3. The calculated and displayed size is still the total of the full depth, of course. But despite this, restricting the display depth drastically speeds up the calculation.
Another helpful option is
-h(words on both GNU and BSD but, once again, not on POSIX-only
du) for “human-readable” output (i.e. using KiB, MiB etc.).
Great anwser. Seems correct for me. I suggest `du -hcd 1 /directory`. -h for human readable, c for total and d for depth.
You can also run the following command using
~# du -Pshx /* 2>/dev/null
-soption summarizes and displays total for each argument.
hprints Mio, Gio, etc.
x= stay in one filesystem (very useful).
P= don't follow symlinks (which could cause files to be counted twice for instance).
Be careful, the
/rootdirectory will not be shown, you have to run
~# du -Pshx /root 2>/dev/nullto get that (once, I struggled a lot not pointing out that my
/rootdirectory had gone full).
Edit: Corrected option -P
Finding the biggest files on the filesystem is always going to take a long time. By definition you have to traverse the whole filesystem looking for big files. The only solution is probably to run a cron job on all your systems to have the file ready ahead of time.
One other thing, the x option of du is useful to keep du from following mount points into other filesystems. I.e:
du -x [path]
The full command I usually run is:
sudo du -xm / | sort -rn > usage.txt
-mmeans return results in megabytes, and
sort -rnwill sort the results largest number first. You can then open usage.txt in an editor, and the biggest folders (starting with /) will be at the top.
"finding biggest takes long time.." -> Well it depends, but tend to disagree: doesn't take that long with utilities like `ncdu` - at least quicker than `du` or `find` (depending on depth and arguments)..
I always use
du -sm * | sort -n, which gives you a sorted list of how much the subdirectories of the current working directory use up, in mebibytes.
You can also try Konqueror, which has a "size view" mode, which is similar to what WinDirStat does on Windows: it gives you a viual representation of which files/directories use up most of your space.
Update: on more recent versions, you can also use
du -sh * | sort -hwhich will show human-readable filesizes and sort by those. (numbers will be suffixed with K, M, G, ...)
For people looking for an alternative to KDE3's Konqueror file size view may take a look at filelight, though it's not quite as nice.
That's only Konqueror 3.x though - the file size view _still_ hasn't been ported to KDE4.
I use this for the top 25 worst offenders below the current directory
# -S to not include subdir size, sorted and limited to top 25 du -S . | sort -nr | head -25
This command did the trick to find a hidden folder that seemed to be increasing in size over time. Thanks!
By default, on my system, 'du -S' gives a nice human readable output. You get a plain number of bytes for small files, then a number with a 'KB' or 'MB' suffix for bigger files.
@Siddhartha If you add `-h`, it will likely change the effect of the `sort -nr` command - meaning the sort will no longer work, and then the `head` command will also no longer work
At a previous company we used to have a cron job that was run overnight and identified any files over a certain size, e.g.
find / -size +10000k
You may want to be more selective about the directories that you are searching, and watch out for any remotely mounted drives which might go offline.
One option would be to run your du/sort command as a cron job, and output to a file, so it's already there when you need it.
du -ch --max-depth=2 .
and I change the max-depth to suit my needs. The "c" option prints totals for the folders and the "h" option prints the sizes in K, M, or G as appropriate. As others have said, it still scans all the directories, but it limits the output in a way that I find easier to find the large directories.