Sort based on the third column

  • I'm facing a huge 4-columns file. I'd like to display the sorted file in stdout based on its 3rd column:

    cat myFile | sort -u -k3
    

    Is that enough to perform the trick?

    Note that you can write this as `sort -u -k3 < myFile`.

    As `sort -u -k3 myFile`, even

  • sort -k 3,3 myFile
    

    would display the file sorted by the 3rd column assuming the columns are separated by sequences of blanks (ASCII SPC and TAB characters in the POSIX/C locale), according to the sort order defined by the current locale.

    Note that the leading blanks are included in the column (the default separator is the transition from a non-blank to a blank), that can make a difference in locales where spaces are not ignored for the purpose of comparison, use the -b option to ignore the leading blanks.

    Note that it's completely independent from the shell (all the shells would parse that command line the same, shells generally don't have the sort command built in).

    -k 3 is to sort on the portion of the lines starting with the 3rd column (including the leading blanks). In the C locale, because the space and tab characters ranks before all the printable characters, that will generally give you the same result as -k 3,3 (except for lines that have an identical third field),

    -u is to retain only one of the lines if there are several that sort identically (that is where the sort key sorts the same (that's not necessarily the same as being equal)).

    cat is the command to concatenate. You don't need it here.

    If the columns are separated by something else, you need the -t option to specify the separator.

    Given example file a

    $ cat a
    a c c c
    a b ca d
    a b  c e
    a b c d
    

    With -u -k 3:

    $ echo $LANG
    en_GB.UTF-8
    
    $ sort -u -k 3 a
    a b ca d
    a c c c
    a b c d
    a b  c e
    

    Line 2 and 3 have the same third column, but here the sort key is from the third column to the end of line, so -u retains both. ␠ca␠d sorts before ␠c␠c because spaces are ignored in the first pass in my locale, cad sorts before cc.

    $ sort -u -k 3,3 a
    a b c d
    a b  c e
    a b ca d
    

    Above only one is retained for those where the 3rd column is ␠c. Note how the one with ␠␠c (2 leading spaces) is retained.

    $ sort -k 3 a
    a b ca d
    a c c c
    a b c d
    a b  c e
    $ sort -k 3,3 a
    a b c d
    a c c c
    a b  c e
    a b ca d
    

    See how the order of a b c d and a c c c are reversed. In the first case, because ␠c␠c sorts before ␠c␠d, in the second case because the sort key is the same (␠c), the last resort comparison that compares the lines in full puts a b c d before a c c c.

    $ sort -b -k 3,3 a
    a b c d
    a b  c e
    a c c c
    a b ca d
    

    Once we ignore the blanks, the sort key for the first 3 lines is the same (c), so they are sorted by the last resort comparison.

    $ LC_ALL=C sort -k 3 a
    a b  c e
    a c c c
    a b c d
    a b ca d
    $ LC_ALL=C sort -k 3,3 a
    a b  c e
    a b c d
    a c c c
    a b ca d
    

    In the C locale, ␠␠c sorts before ␠c as there is only one pass there where characters (then single bytes) sort based on their code point value (where space has a lower code point than c).

    columns are `blank`-separated that may include other characters in addition to space and tab depending on locale.

    Nice, +1. Could you explain what the `3,3` does? Why not just `3`?

    @terdon, see expanded description with examples.

    @J.F.Sebastian, you're right, answer updated.

    Ah, to make it only sort on the 3rd, not the rest of the line, thanks.

    Wonderful option :) .. I didn't know about that! .. thanks for the tip.

    Where is the original file `a` ??? Why is it not included? Why was it rejected to include it ???

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM