How to search text throughout entire file system?

  • Assuming that the grep tool should be used, I'd like to search for the text string "800x600" throughout the entire file system.

    I tried:

    grep -r 800x600 /

    but it doesn't work.

    What I believe my command should do is grep recursively through all files/folders under root for the text "800x600" and list the search results.

    What am I doing wrong?

    And by "it doesn't work" you mean exactly what? Does it not print any output, hangs or print lots of `Permission denied` errors? Did you run it as root or a normal user?

    I'm getting some traction, first of all I was in my user home directory trying to run the command. So now I've cd / out to root. Next I tried the same command as above and I'm getting a lot of Permission denied errors. Ok, so now I try sudo grep -r 800x600 / and then I get a /proc/sysrq-trigger: Input/output error

    Hmm, don't know why it wouldn't work. You may ignore access errors by doing `grep -r 800x600 / 2>/dev/null`. You can also try running it as root.

  • I normally use this style of command to run grep over a number of files:

    find / -xdev -type f -print0 | xargs -0 grep -H "800x600"

    What this actually does is make a list of every file on the system, and then for each file, execute grep with the given arguments and the name of each file.

    The -xdev argument tells find that it must ignore other filesystems - this is good for avoiding special filesystems such as /proc. However it will also ignore normal filesystems too - so if, for example, your /home folder is on a different partition, it won't be searched - you would need to say find / /home -xdev ....

    -type f means search for files only, so directories, devices and other special files are ignored (it will still recurse into directories and execute grep on the files within - it just won't execute grep on the directory itself, which wouldn't work anyway). And the -H option to grep tells it to always print the filename in its output.

    find accepts all sorts of options to filter the list of files. For example, -name '*.txt' processes only files ending in .txt. -size -2M means files that are smaller than 2 megabytes. -mtime -5 means files modified in the last five days. Join these together with -a for and and -o for or, and use '(' parentheses ')' to group expressions (in quotes to prevent the shell from interpreting them). So for example:

    find / -xdev '(' -type f -a -name '*.txt' -a -size -2M -a -mtime -5 ')' -print0 | xargs -0 grep -H "800x600"

    Take a look at man find to see the full list of possible filters.

    Note that `-xdev` will exclude _all_ other filesystems, not just special ones. (e.g., if you have `/home` mounted as a separate partition, it won't be searched.)

    I tried running each one but both return an error -- `find: paths must precede expression: /`

    @Level1Coder my mistake, I had the -xdev option in the wrong spot. I've edited the answer to correct this. @cjm Good catch, meant to say that but forgot by the time I reached the end!

    Great, both are working now but I find that the first method is extremely slow. I ran both in 2 different terminals and the efficient one was done in ~1min and the inefficient method was slowly chugging along, I didn't even allow it to finish and just ^C out. Now that this works, is it possible to make it a bit more specific, such as only search text within files that are of filetype .txt, less than 2MB and modified within the last 5 days?

    @Level1Coder, thanks for your comments. I've edited the answer to remove the inefficient version, since it's obviously rubbish :) And yes, it can be made more specific - again, I've edited the answer so that it covers your example.

    @Richard, thank you for your excellent answer and concise explanation.

    Note: When regular expressions aren't required 'fgrep' is significantly faster than 'grep', which will make a big difference if you're searching a large tree.

    Note on @NathanKidd 's note: fgrep is deprecated, it's the same as `grep -F`

    You can avoid going through `xargs` with maybe better efficiency by doing `find / -xdev -type f -exec grep -H '800x600' +`.

    @Totor, thanks for the suggestion - actually my first version of this answer did that, but it turned out very slow on medium and large finds. Passing `-exec` to `find` will cause it to spawn a `grep` command for every single file it sees - creating then destroying thousands of `grep` commands this way kills performance. `xargs` starts fewer `grep` commands so has better performance.

    No, the `+` sign at the end of the `find` command actually does the same thing as `xargs`: it spawns one `grep` process with several arguments.

License under CC-BY-SA with attribution

Content dated before 6/26/2020 9:53 AM