Generating list of manually installed packages and querying individual packages

  • I'd like to get a list of packages installed manually by apt or aptitude and be able to find out whether a foobar package was installed manually or automatically. Is there any neat way of doing that from the command line?

    See this answer on unix.stackexchange.com for a solution that filters out stock packages.

    Really good solution that excludes packages installed by default: Ubuntu list explicitly installed packages

    Careful on answers' comments here. People aren't saying way more packages show up, but they are forgetting that there are dependencies packages installed from a manual installed.

    Why tf isn't this part of the program itself?? absolutely shocking UI

  • jmiserez

    jmiserez Correct answer

    6 years ago

    You can use either of these two one-liners. Both yield the exact same output on my machine and are more precise than all solutions proposed up until now (July 6, 2014) in this question.

    Using apt-mark:

    comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)
    

    Using aptitude:

    comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)
    

    Very few packages still fall through the cracks, although I suspect these are actually installed by the user, either right after the installation through the language localization setup or e.g. through the Totem codec installer. Also, the linux-header versions also seem to accumulate, even though I've only installed the non version-specific metapackage. Examples:

    libreoffice-help-en-gb
    openoffice.org-hyphenation
    gstreamer0.10-fluendo-mp3
    linux-headers-3.13.0-29    
    

    How does it work:

    1. Get the list of manually installed packages. For aptitude, the additional sed strips out remaining whitespace at the end of the line.
    2. Get the list of packages installed right after a fresh install.
    3. Compare the files, only output the lines in file 1 that are not present in file 2.

    Other possibilities don't work as well:

    • Using the ubuntu-14.04-desktop-amd64.manifest file (here for Ubuntu 14.04) instead of /var/log/installer/initial-status.gz. More packages are shown as manually installed even though they are not.
    • Using apt-mark showauto instead of /var/log/installer/initial-status.gz. apt-mark for example doesn't include the xserver-xorg package, while the other file does.

    I used various other StackExchange posts as references, however none work as well as the above solution:

    Both list more packages than the above solution.

    EDIT: What to do if you've upgraded from a previous release:

    If you've upgraded Ubuntu from one release to the next, you will probably need to adjust this process. In that case, I would check the manifest file of the newer release (see above) in addition to the initial-status.gz file from the current release. You can easily do that by just adding another comparison. Using just the manifest file will not work, as the manifest file unfortunately does not contain everything that the initial_status.gz file does (I checked).

    This didn't work for me because `/var/log/installer/initial-status.gz` is missing. Also I want to know if this is depending on apts marking of `manual` or not?

    Alas there is no manifest for server versions.

    I ran the `showmanual` command (below). And use `comm` to compare the two(sorted) lists. The `showmanual` result gave me 1,840 _more_ unique packages from `apt-mark showmanual` not shown by this method. NO packages were unique to the output of this `comm`-command. I think it is more interesting to record that for my PC, **894** packages are listed in Both results. Not sure why there's such a Vast discrepancy. Some (many?) packages seem to be release specific. Others like XOrg, GTK components and `lib*` things could be updates. _Anyway_ this answer is a very good start.

    I just compared your solutions with `apt-mark showmanual`. It's interesting how many differences are visible. your list has 238 packages while showmanual returns 1717 packages. Of the 2179 installed packages, there are only 223 in both lists, 15 are only in yours (examples: nodejs, lightdm) and 223 are only in showmanual (examples: xserver-xorg, ubuntu-desktop). It feels like your list is more helpful, but without knowing where these differences come from, it's not easy to decide... (but I'm quite sure I installed nginx and lightdm manually...) [sorry will that I just wrote the same ;)]

    Is there a way to get a concise list of what packages ***I*** installed? I expect a list with 30-ish items, but it's full of stuff like 1) things I've never ever even heard of (like `zenity`), 2) Things I've heard of but consider core functionality (like `acpi-support`), 3) stuff that's 100% useless to me (like ten `printer-driver-xxx` while I don't even have a printer) and 4) stuff that's almost surely installed by another app and not by me (like `thunderbird-locale-en`)

    And today I learned about 'comm -23'! Thanks for that!

    Manifest files can be downloaded from http://releases.ubuntu.com/

  • In newer versions of the package apt, there is also the apt-mark command

    apt-mark showmanual
    

    Exactly the answer I was searching for.

    This shows way more packages than I have installed manually.

    @Umang You are right. I would say this wasn't like this when I wrote this answer. There is no reason on my system to consider `linux-image-3.11.0-*-generic` etc as manual

    @Umang maybe this will help you http://askubuntu.com/questions/432743/how-are-packages-classified-in-apt-mark-showauto-showmanual, but the answer is not accepted. Fact is, that many packages of a fresh installation are already marked as manual. But there are still some strange things. To stay with my example: `linux-image-3.13.0-24-generic` is manual but the current `linux-image-3.13.0-27-generic` is automatic. It seems that an upgrade of a referencing package (in this case `linux-image-generic`, which changed the dependencies), the manual mark is automatically set

    @DanielAlder some packages of a fresh installation should be marked as manual. If no packages marked as manual, the entire system could be deleted with `apt-get autoremove`. This is definitely not what you want.

    @AntonK And this explains why I had 5 kernels marked as manual?

    They should add `apt-mark showreallymanual --in-fact-manual` command option

    If "manually installed" means "installed by the user after initial OS installation", this is not a correct answer.

  • For Ubuntu 16.04, check out the log file /var/log/apt/history.log.

    For example:

    zgrep 'Commandline: apt' /var/log/apt/history.log /var/log/apt/history.log.*.gz
    

    It's not perfect, but it's pretty good at making it clear exactly what I installed by hand. Put a -B 1 on the grep to see when it was installed.

    Example output

    Commandline: apt install postgresql-9.5-plv8
    Commandline: aptdaemon role='role-install-file' sender=':1.85'
    Commandline: apt install task
    Commandline: apt autoremove
    Commandline: apt install atom
    Commandline: apt upgrade
    Commandline: apt-get install asciinema
    Commandline: apt install iperf3
    Commandline: apt upgrade
    Commandline: apt-get install chromium-browser
    Commandline: apt install joe cpanminus build-essential postgresql libdbd-pg-perl libcrypt-openssl-bignum-perl libcrypt-openssl-rsa-perl libio-socket-ssl-perl libnet-ssleay-perl libssl-dev
    Commandline: aptdaemon role='role-commit-packages' sender=':1.2314'
    Commandline: apt install git
    Commandline: apt install sqlite
    Commandline: apt install whois
    Commandline: apt install libdbd-pg-perl
    Commandline: apt install perl-doc
    Commandline: apt upgrade
    

    Not sure if this picks up aptitude or not. It doesn't seem to pick up installs from the Ubuntu Software desktop app.

    Of all the answers over the years, this is the only one that even comes close in 18.04 server.

    Don't forget that logs get deleted from time to time...

  • apt-mark showauto | grep -iE '^foobar$' will output "foobar" if the package was installed automatically, nothing otherwise.

    aptitude search '!~M ~i' will list the packages that were not installed automatically. It's a pity aptitude won't be part of the default install on Ubuntu Desktop starting from 10.10.

    `aptitude search` shows ALL packages not just the ones that are manually installed (I assume that's what the OP wanted)

    @Oli: look into aptitude search patterns; the pattern I'm using there should do exactly what the OP wants.

    I *ran* it. It shows a whole load of packages that aren't installed.

    Something isn't right with this, I'm using `aptitude search '!~M ~i'` and it lists 1043 packages. There is no way I installed that many packages manually.

    This definitely doesn't work as requested, prints pre-installed packages as well.

    @GraemeDonaldson & Irfy: I think there's a bug in Ubuntu(?), under some conditions it lists system packages as packages installed manually. I'm seeing the same on my system (check with synaptic), where I have packages like "accountservice" listed as manually installed and I am certain I did not install it manually.

    On my system, `aptitude search '!~M ~i'` shows exactly 1583 lines, many more than also I expected. But so does `apt-mark showmanual`, so at least they agree.

    Unaccepting this answer, because as everyone has pointed out, it doesn't actually seem to work. However, the other answer don't seem to work either. I don't need a solution right now (this is a very old question), but to keep this accurate, I will accept any answer that actually works well.

  • The following script will print out all the packages that are not set to automatic install and hence were installed manually:

    #!/usr/bin/python
    
    try:
        import apt_pkg
    except ImportError:
        print "Error importing apt_pkg, is python-apt installed?"
        sys.exit(1)
    
    apt_pkg.init()
    STATE_FILE = apt_pkg.config.find_dir("Dir::State") + "extended_states"
    auto = set()
    tagfile = apt_pkg.TagFile(open(STATE_FILE))
    while tagfile.step():
        pkgname = tagfile.section.get("Package")
        autoInst = tagfile.section.get("Auto-Installed")
        if not int(autoInst):
            auto.add(pkgname)
    print "\n".join(sorted(auto))
    

    it is based on how apt-mark prints out the automatically installed packages.

    Kudos to you fine sir. This actually works, in contrast to the accepted answer.

    show just a couple of package for me --- definitely missing a lot of them.

    Same here, definitely missing manually installed packages, right after I installed them.

    Using `sys.exit(1)` without `import sys` might result in an error in newer versions of python. Either `import sys` or use `exit(1)`.

  • To get a list of all packages (not installed, installed by user or installed by default, across all PPAs), apt employs the following method:

    apt list [option]

    The possible options useful for this are:

    --installed to display only the packages that are installed on the system (out of some 50,000+)

    --manual-installed to list the packages that were explicitly installed by a command, either directly, or as dependencies.

    Alternatively, you could do:

    apt list --manual-installed | grep -F \[installed\] to get a list of packages that resulted from user commands and their dependencies only, and to get additional information on them such as version and architecture supported (x86, x86_64, amd64, all, etc.)

  • As several people have commented, apt-mark showmanual seems to be a bit buggy (and I reported it as bug 727799). When I'm using it, it actually reports a lot of stuff that isn't even logged in /var/lib/apt/extended_states (where this is supposed to be stored) and apt-get isn't logging things as installed in /var/lib/apt/extended_states (just in /var/lib/dpkg/status). The python script by txwikinger above draws from /var/lib/apt/extended_states directly but if you're using it today the syntax might not work (mine just started generating errors with Kubuntu 13.10). Updated syntax is:

    #!/usr/bin/python
    import sys
    
    try:
        import apt_pkg
    except ImportError:
        print "Error importing apt_pkg, is python-apt installed?"
        sys.exit(1)
    
    apt_pkg.init()
    STATE_FILE = apt_pkg.config.find_dir("Dir::State") + "extended_states"
    auto = set()
    tagfile = apt_pkg.TagFile(open(STATE_FILE))
    while tagfile.step():
        pkgname = tagfile.section.get("Package")
        autoInst = tagfile.section.get("Auto-Installed")
        if not int(autoInst):
            auto.add(pkgname)
    print "\n".join(sorted(auto))
    

    For me this was a very short list of 5 items which doesn't seem to be accurate either.

    Using `sys.exit(1)` without `import sys` might result in an error in newer versions of python. Either `import sys` or use `exit(1)`.

  • I would like to give a GUI solution.

    enter image description here

    1. Open Synaptic Package Manager

    2. Go to Status

    3. Click Installed (manual)

    It will give the list of packages installed manually by apt or aptitude.

    Unfortunately I could not find any option in Custom Filters to find out whether a foobar package was installed manually or automatically.

    If the package is under Installed but not under Installed (manual) then it was Installed automatically. If the package is under Installed (manual) then it was installed manually.

  • If no one gives you a nice answer using a apr-something command you can do it the hard way. Apt-get stores its info in /var/lib/apt/extended_states. Any file that is installed automatically will be added to this file. If you install a package already in this file manually, the package will remain in this file but with Auto-installed: 0 in the second line. It's not deleted.

    Note: As expected better answers that are likely to work if file placement changes have appeared. I keep mine just in case the info on the file location is useful.

    No. I took a quick look at that file to find that liferea was marked as auto-installed. I did an `apt-get install liferea` and it didn't install but I got output that was something to the effect of "marked as manually installed". Now liferea is still in the file, except the next line has a `0` instead of a `1`. Also, you should change your regex pattern to `" foobar$"` instead of just `foobar`.

    That's true. My fault, in my system there is no line with 0, but it should be a rare happening. I update the answer just in case.

  • After googling a lot, I've managed to assemble this script. It works alright for me:

    # List of all packages currently installed
    current=$(dpkg -l | awk '{print $2}' | sort | uniq)
    
    # List of all packages that were installed with the system
    pre=$(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort | uniq)
    
    # List of packages that don't depend on any other package
    manual=$(apt-mark showmanual | sort | uniq)
    
    # (Current - Pre) ∩ (Manual)
    packages=$(comm -12 <(comm -23 <(echo "$current") <(echo "$pre")) <(echo "$manual") )
    
    for pack in $packages; do
        packname=$(echo $pack | cut -f 1 -d ":")
        desc=$(apt-cache search "^$packname$" | sed -E 's/.* - (.*)/\1/')
        date=$(date -r /var/lib/dpkg/info/$pack.list)
    
        echo "# $desc"
        echo "# $date"
        echo "sudo apt-get install $pack"
        echo -e ""
    done
    

    You could use `sort -u` instead of `sort | unique`. As `apt-mark`does not display architecture, you should strip it from dpkg's output before the set operations (or use `dpkg-query -W -f='${Package}\n'`). Besides, dpkg may list some packages that are not installed currently. As for "desc", you could use `dpkg-query -W -f='# ${binary:Summary}\n' $pack, which is faster.

    Oh, `apt-mark`may display architecture for some packages, but not for so many as `dpkg -l`.

    `apt-cache search` is slow. Getting a list of installed dates in advance using something like https://help.ubuntu.com/community/ListInstalledPackagesByDate might be more efficient

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM