How to add a newline to the end of a file?

  • Using version control systems I get annoyed at the noise when the diff says No newline at end of file.

    So I was wondering: How to add a newline at the end of a file to get rid of those messages?

    Nice solution down below that sanitizes all files recursively. Answer by @Patrick Oscity

    Going forward, text editors often have options to ensure there's a trailing newline which you and your collaborators could use to keep clean.

  • To recursively sanitize a project I use this oneliner:

    git ls-files -z | while IFS= read -rd '' f; do tail -c1 < "$f" | read -r _ || echo >> "$f"; done
    

    Explanation:

    • git ls-files -z lists files in the repository. It takes an optional pattern as additional parameter which might be useful in some cases if you want to restrict the operation to certain files/directories. As an alternative, you could use find -print0 ... or similar programs to list affected files - just make sure it emits NUL-delimited entries.

    • while IFS= read -rd '' f; do ... done iterates through the entries, safely handling filenames that include whitespace and/or newlines.

    • tail -c1 < "$f" reads the last char from a file.

    • read -r _ exits with a nonzero exit status if a trailing newline is missing.

    • || echo >> "$f" appends a newline to the file if the exit status of the previous command was nonzero.

    You can also do it like this if you want to just sanitize a subset of your files: `find -name \*.java | while read f; do tail -n1 $f | read -r _ || echo >> $f; done`

    @StéphaneChazelas good suggestions, will try to incorporate this into my answer.

    @PerLundberg you can also pass a pattern to `git ls-files` which will still save you from editing files that are not tracked in version control.

    @StéphaneChazelas adding the `IFS= ` to unset the separator is good to preserve surrounding whitespace. The null terminated entries are only relevant if you have files or directories with a newline in their name, which seems kind of far fetched, but is the more correct way to handle the generic case, I agree. Just as a small caveat: the `-d` option to `read` is not available in POSIX sh.

    Yes, hence my *zsh/bash's*. See also my use of `tail -n1 < "$f"` to avoid problems with file names that start with `-` (`tail -n1 -- "$f"` doesn't work for the file called `-`). You may want to clarify that the answer is now zsh/bash specific.

    Actually, `{ tail -c1 | read -r _ || echo; } < "$f" >> "$f"` would be better to avoid adding the newline if the file can't be open for reading (for instance because it doesn't exist or is write-only). Reading one byte of that last line instead of the full line would be enough.

    @StéphaneChazelas all valid points. Thank you.

    For me, `touch - && tail -n1 -- "-"` is working fine. Are you sure this is an issue?

    The GNU, busybox, ast-open, Solaris `tail` implementations at least (not {Free,Net,Open}BSD's) treat `-` as meaning standard input instead of the file called `-` in the current directory (as allowed but not required by POSIX).

    @StéphaneChazelas I tried to incorporate all of your suggestions, but tried to omit curly braces to make it easier to read left-to-right. You seem to know your shell stuff, so I'd be happy about final remarks :) Cheers!

    How do you make this ignore binary files and only operate on text files?

    @AaronFranke replace `git ls-files -z` with `git grep -zIl ''`

    Awesome, that works. Now, how do I exclude specific file extensions?

    @AaronFranke you could pipe through a `grep -v` or you could take a look at the man page of git-grep. It has some examples for excluding files. https://git-scm.com/docs/git-grep#_examples Should be smth like `git grep -zIl '' -- ':!*.xml' ':!*.json'`

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM