How to backup settings and list of installed packages
If I want to make a backup of everything I have done since the fresh install of Ubuntu, what are the possible options? What all should I backup? I want to get all the settings that I changed, all the packages I installed, etc.
Be wary of this bug though: https://bugs.launchpad.net/ubuntu/+source/policykit-desktop-privileges/+bug/1240336 I tried the most upvoted solution and never got it working because of that bug. (This might be better suited as a comment, but I don't have enough reputation to post it as a comment)
be aware also that if you are restoring in a newer version of the OS, some settings may be incomplete or even conflict, having a possibility to make some applications malfunction.
A quick way of backing up a list of programs is to run this:
dpkg --get-selections > ~/Package.list sudo cp -R /etc/apt/sources.list* ~/ sudo apt-key exportall > ~/Repo.keys
It will back them up in a format that dpkg can read* for after your reinstall, like this:
sudo apt-key add ~/Repo.keys sudo cp -R ~/sources.list* /etc/apt/ sudo apt-get update sudo apt-get install dselect sudo dselect update sudo dpkg --set-selections < ~/Package.list sudo apt-get dselect-upgrade -y
* You may have to update dpkg's list of available packages or it will just ignore your selections (see this debian bug for more info). You should do this before
sudo dpkg --set-selections < ~/Package.list, like this:
apt-cache dumpavail > ~/temp_avail sudo dpkg --merge-avail ~/temp_avail rm ~/temp_avail
Settings and Personal Data
Before you reinstall, you should probably back up the settings from some of your programs, this can easily be done by grabbing folders from /etc and all the content from your user directory (not just the stuff you can see in nautilus!):
rsync --progress /home/`whoami` /path/to/user/profile/backup/here
After you reinstall, you can restore it with:
rsync --progress /path/to/user/profile/backup/here /home/`whoami`
So all together as a pseudo-bash script.
This assumes there is only one user on the machine (remove
/'whoami'otherwise) and that you used the same username on both installs (modify dest. of rsync otherwise).
dpkg --get-selections > ~/Package.list sudo cp -R /etc/apt/sources.list* ~/ sudo apt-key exportall > ~/Repo.keys rsync --progress /home/`whoami` /path/to/user/profile/backup/here ## Reinstall now rsync --progress /path/to/user/profile/backup/here /home/`whoami` sudo apt-key add ~/Repo.keys sudo cp -R ~/sources.list* /etc/apt/ sudo apt-get update sudo apt-get install dselect sudo dpkg --set-selections < ~/Package.list sudo dselect
The last line of the reinstall script, "sudo dselect", contradicts the last line in the second script, "sudo apt-get dselect-upgrade -y". Also, running "dselect" opens some text-based GUI, where it is not clear how to continue. Is this a typo?
I ran the reinstall script, and got into an infinite loop: [email protected]:~$ sudo dpkg --set-selections < ~/Package.list [email protected]:~$ sudo apt-get dselect-upgrade -y ... After this operation, 451 MB of additional disk space will be used. Get:1 http://il.archive.ubuntu.com/ubuntu/ precise/main libkeyutils1 i386 1.5.2-2 [7,716 B] Get:2 http://il.archive.ubuntu.com/ubuntu/ precise/main libkeyutils1 i386 1.5.2-2 [7,716 B] ......
It tries forever to get "libkeyutils1". I tried to remove "libkeyutils1" from Package.list, but it didn't help.
It is likely a dependency of another package so removing it from the packages.list wont fix the fact that another package will attempt to install it. I recommend you start a new question for this and link back here for context.
This will not work on ubuntu 12.10. it says ``dpkg: warning: package not in database at line XXX: xrdp`` And there is no package by name "dselect"
Try doing a `sudo apt-get upgrade` because both of those packages exist in 12.10 as per packages.ubuntu.com. I will add that line into the pseudo-script
@Huckle: in 13.04 there is a package dselect but the error is still same... ex: `dpkg: warning: package not in database at line 1484: vlc`
blah... to solve dpkg warning, read this: http://forums.debian.net/viewtopic.php?f=17&;t=79006#p432478
Remember to back up and re-add the sources.d directory -- ppa's especially will pile there source info into separate files in there. The re-install will fail without them.
ssh files are stored in ~/.ssh so as long as rsync recurses into dot-directories the ssh files will be taken. If it doesn't recurse into directories starting with a dot, there's likely a switch that will make it do so.
Restored. Was disappointed as all DesktopEnv related settings (launcher, animation, compiz tweaks, desktop background) were not restored. Commoness: All reside in dconf! Solution: Within the GUI, `Log Out` via the gear icon in the main menu bar. Then switch to a CLI via `CTRL-ALT-F1`. There restore .config/dconf/user from your backup. Switch back to the GUI via `CTRL-ALT-F7`, log in again. Voila: All your Unity related settings are in effect again! Explanation attempt: While an active Unity GUI session the dconf file may be locked or ignored, thus you need to switch to a pure CLI mode?
@porq If you replace the configuration of a program/service that is running, you will need to stop and start that program/service. Dropping to a lower run level or restarting the computer will solve some or all of these sorts of problems.
**note:** `dpkg: warning: package not in database` errors are likely caused due insufficient file permissions - don't forget to `chmod +r /etc/apt/sources.list* -R` after copying them to the target machine!
Sorry but there is is an error when you save the apt-key : When saving them you should use $ sudo apt-key exportall > ~/Repo.keys definitely not apt-key add which is used for restoring ...
Who is this for: users that have normal regular use of their computer, that have done minimal or no configuration outside their home folder, did not mess up startup scripts and services. A user that wants to have his software restored to how it was when he installed it with all customizations being done and kept in their home folder.
Who this will not fit for: servers geeks, power users with software installed by source (restoring the package list might break your system), users that have changed the startup script of some application to fit better their needs. Caution: there is a big chance any modifications outside home will be over written.
Backup your current packages and user settings
Once you are ready with your system and happy with the software installed you can get a list of the installed packages using the command
dpkg --get-selectionsand save the output in to a log file.
backupfolder in your home
sudo mkdir ~/backup
Save the currently installed packages list
dpkg --get-selections > ~/backup/installed_packages.log
Make a backup of your apt sources file
sudo cp /etc/apt/sources.list ~/backup/sources.bak
and a copy of your apt's list of trusted keys
sudo apt-key exportall > ~/backup/repositories.keys
Make a backup of your home folder with the integrated backup tool in Ubuntu, Deja-dup.
- Set the backup folder
- Verify that it will make a backup of your home folder and add any folders your might not want, like for instance the
- On the overview tab press Make Backup Now to backup your home folder
- The backup tool will ask if you want to add a password to your backup, your choice. Your backup will start after you press Continue
When this is done you will have a backup of your packages and configuration files relative to your user. If when necessary you will be able to restore your files from the
Restoring your backup
Start by restoring the sources file from the backup made
sudo cp ~/backup/sources.bak /etc/apt/sources.list
The backed-up keys
sudo apt-key add ~/backup/repositories.keys
Update your sources lists
sudo apt-get update
Restore the packages from the saved
sudo dpkg --clear-selections sudo dpkg --set-selections < ~/backup/installed_packages.log && sudo apt-get dselect-upgrade
sudo dpkg --clear-selectionswill mark all current packages installed for removal, that way when you restore your saved package list the packages that are not on the list will be removed from your system.
Remove your current configuration from your home creating a backup of the folder in their current state (after all, whats the use of restoring fresh files if there are other there that can affect the configuration?)
mkdir ~/.old-gnome-config/ && mv ~/.gnome* ~/.old-gnome-config/ && mv ~/.gconf* ~/.old-gnome-config/ && mv ~/.metacity ~/.old-gnome-config/ && mv ~/.cache ~/.old-gnome-config/ && mv ~/.dbus ~/.old-gnome-config/ && mv ~/.dmrc ~/.old-gnome-config/ && mv ~/.mission-control ~/.old-gnome-config/ && mv ~/.thumbnails ~/.old-gnome-config/ && ~/.config/dconf/* ~/.old-gnome-config/
After this is done restore the backup created with Deja-dup.
- Open the backup tool and on the overview tab press restore
- Follow the prompts to restore the files to their original positions
After this is done you will have your packages back to the saved selection, our configuration restored and hopefully a working desktop, all of that without installing a single extra application and using the Ubuntu default tools.
The only thing left is to do some clean up and check that everything is working.
Something went wrong, my desktop is gone
There are a few posts in Ask Ubuntu that can guide you in case something when wrong and you need to hard reset your desktop. If something when wrong and you find your self needing to do so, please have a look at these posts:
Backups take some planning and there are several viable strategies. You will have to decide which method works best for you.
Be careful about "best way" , what works best for one person may not be best for another.
At the end of the day, the "best method" is one that has been tested and known to work. You need to test your backup strategy BEFORE you need it
One way is to simply copy and compress an image of your partitions. You can do this with several tools, anything from dd to partimage to clonezilla
The advantage of this strategy is that it is (relatively) easy and very complete. The disadvantage is that the back up images are large.
You can make smaller backups by only backing up data and settings. There are several tools to do this, everything from dd to tar to rsync.
The key here is to know what you need to back up.
Advantage - Backups will be smaller.
Disadvantage - This is a manual method, so it is easy to forget about a critical file (/etc/passwd).
A list of what to back will always need to be reviewed to make sure the list is complete.
1) List of installed packages
Package list # Create a text list of an existing installation of all apt-get installed packages # in order to re-install on a newly installed distro
# make the list [old distro] sudo dpkg --get-selections > packages
You would then perform a fresh install, and restore your packages. The following commands also update all the packages on your system (so restore and full update all at once).
# Now put them back on the new distro [new distro] sudo dpkg --set-selections < packages [new distro] sudo apt-get dselect-upgrade
2) Data. Generally this would be /home . Most user data and customization's are going to be in your users home directory. If you save data in other locations, include that (for example /media/data).
3) system settings . Here is where there is going to be some variation. Personally if I edit ANY system setting, I keep a copy of the original configuration file and my custom file in /root. So if I edit
/etc/fstabfor example, keep a copy in /root/etc/fstab and original in /root/etc/fstab.orig
You will also need
/etc/hosts(you may need more in /etc, if I forgot something I will add it)
On a server you may need to include
/var/wwwor other data directories.
I understand this takes some effort, so, it may be easier to include all of
4) A copy of your disk partition table.
sudo fdisk -l > fdisk.bak
You can use this information to restore your partition table if you replace your hard drive.
5) A copy of your MBR
sudo dd if=/dev/sda of=MBR.bak bs=512 count=1
You would then restore with
sudo dd if=MBR.bak of=/dev/sda bs=512 count=1
6) Other files/directories - Depending on your system and customization you may need to include additional directories. Considerations might include
/usr/local, /usr/share`, and/or your .desktop files. Anything on your system you downloaded or customized outside of apt-get or software manager.
7) Put all that into an archive
sudo tar -cvpzf backup.tar.gz /home /root /etc ./MBR.bak ./fsdisk.bak ./packages
As an alternate to tar, you can use rsync.
Another option is to use NFS or Samba to back up data.
You can automate backups by writing a backup script and running it (daily / hourly) with cron.
1) Using a live CD, restore your partitions using gparted or fdisk from the information in fdisk.bak
2) Install Ubuntu.
3) Boot your new install, update your packages.
4) Restore your data and settings
tar -xvpzf /home/test/backup.tar.gz -C /
There are many additional strategies for backup, some for backing up your home directory only, some graphical.
It is not a backup unless it is tested. This is most important when you are NOT using an image of your root partition.
Test restoring your system in a VM, a spare computer, or a spare partition or hard drive.
Installed Packages Selection using
APT-Clone. This package can be used to clone/restore the packages on a apt based system. It will save/restore the packages, sources.list, keyring and automatic-installed states. It can also save/restore no longer downloadable packages using dpkg-repack.
APT-Clone is used by
ubiquity(Ubuntu installer) for upgrade process.
sudo apt-get install apt-clone
sudo apt-clone clone path-to/apt-clone-state-ubuntu-$(lsb_release -sr)-$(date +%F).tar.gz
sudo apt-clone restore path-to/apt-clone-state-ubuntu.tar.gz
Restore to newer release:
sudo apt-clone restore-new-distro path-to/apt-clone-state-ubuntu.tar.gz $(lsb_release -sc)
Home Data, Configuration
Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.
sudo apt-get install duplicity
duplicity full path-to/source_folder/ file:///path_to/duplicity_backups/
or incremental backup (It will backup only the difference from last backup):
duplicity incremental path-to/source_folder/ file:///path_to/duplicity_backups/
duplicity restore file:///path_to/duplicity_backups/ path-to/target_folder/
APT-Clone is much better than the `dpkg --get-selections` solution because: 1. it preserves all repositories information 2. it keeps track of what packages were automatically installed 3. it allows to repack locally installed DEB files. This should be the accepted answer!
This is great! I'm hoping these work with 16.04, it's exactly what I'm looking for. But I'm wondering, will Apt-Clone also remove packages that aren't in the restored configuration?
@XToro, No it it will not remove. (btw, if it was removing packages by default it will damages the upgrade, because most of upgrades have new packages installed and some dropped)
@Sneetsher That's a shame. I would actually like if it removed all packages not on the list and then install the ones listed along with its dependencies. My issue with why I want to use it is to remove unwanted packages after playing around with things like installing new DE's. There's always tons of leftovers after removing the DE.
Although this question has been answered for a while, I noticed that nobody mentioned etckeeper. Run
apt-get install etckeeper, configure your VCS in
/etc/etckeeper/etckeeper.conf, and you will from then on have much better control of your /etc directory. This can also be used to restore your configurations within the /etc directory.
For instance, (assuming you configured git as your VCS) back up your
/etc/.gitdirectory. Any time you need to restore your configurations onto a fresh /etc, you copy your backed-up /etc/.git directory into the fresh /etc directory. You now have many options for managing your restoration:
- View differences
- Keep all differences in the "fresh" directory in a git branch
git checkout -b new; git add -A; git commit -m 'new etc'
- Unconditionally revert to your backup
git clean -f
- View differences
For those wanting a nice. neat GUI...
All you need is a backup directory, stored locally or in the cloud.
Aptikwill backup PPAs, downloaded packages, software selections, application settings and themes and icons. Very useful.
You can install it through the ppa:
sudo apt-add-repository –y ppa:teejee2008/ppa sudo apt-get update sudo apt-get install aptik
Hope this helps out :)
`sudo apt-add-repository –y ppa:teejee2008/ppa` should be `sudo apt-add-repository ppa:teejee2008/ppa`. Only 1 argument accepted.
This is the most REALISTIC option to someone that wants to do a selectable, system-wide and user backup. Btw, the current version supports much more settings than what is shown on that screenshot.
How to run aptik as gui? I've just installed it with commands provided and it appears to run as a console utility, non-available in Show Applications menu. You may fix aptik repo in your post, btw.
To create a backup list of all your installed programs: http://savvyadmin.com/backup-and-restore-package-lists-in-ubuntu/
You will obviously need to backup your software sources that some of those installed packages are from: Backup Software Sources
As for your program settings, most of those are in hidden (start with a
.) folders and files in your home folder. I would just backup all of them.
If storage space isn't a concern, using dd or dc3dd will backup absolutely everything on the target disk or partition you designate. You don't want to try this on a mounted drive, so you'd do this from a bootable USB or DVD (or your install media choosing "Try Ubuntu"):
sudo dd if=/dev/<source> of=/path/<target>.img
Where is the disk or partition that you wish to backup and target is the backup filename (often the same) (
Substitute dc3dd for dd if you want a progress report. To access specific files in the backup you can typically mount the resulting .img file as a loop device by:
mount -o loop,ro,offset=32256 filename.img /mnt/dirThis works on drives with a single partition where you have backed up the entire drive.
mount -o loop,ro filename.img /mnt/dirif you backed up a partition.
You can restore the entire disk or partition by swapping the if= and of= parameters as in:
sudo dd if=/path/<source>.img of=/dev/<target>
Where is the image file that you wish to restore and <target> is the drive or partition you wish to restore it to.
PROS: Easy to backup and easy to restore everything. Unlike some other solutions doesn't really rquire much in the way of planning since you are backing up everything you don't have to worry about whether you will need it or not.
CONS: Time consuming (computer time, not yours) and not suitable for daily backup (no incremental options)
Source: experience; I use this approach to backup client systems prior to beginning work and have never lost a bit of client data.
I want to have ready-to-restore apps and their data, also offline.
That defeats the purpose of a "format-and-install". If you want to do that, just do an upgrade from your existing Ubuntu installation.
Since "installation" in Ubuntu is as simply as going into the Software Center (or Synaptic or
apt-get), and most programs are themselves updated frequently, there's no real point in backing up the programs, reinstalling Ubuntu and then installing from the backups instead of just getting the latest versions from the repositories.
As long as your data and program settings are backed up, you should be fine. If you still want a comprehensive list of everything in your system so you know if a package is missing, just go into the terminal and type
dpkg --list > mypackages.txt, and put that text file somewhere safe -- it contains every package ever installed on your current system.
I'll assume it's a new HDD, SSD or new system or you had on an old Ubuntu. Now you want to upgrade to Ubuntu 12.04 (Precise Pangolin)?
I'll assume you detest installing them all again or reconfiguring? No need to.
Open the terminal by pressing Ctrl + Alt + T, and run the following command:
sudo dpkg --get-selections > app-backup-list.txt
Then cp the text file to your home directory by:
sudo cp app-backup-list.txt /home/username
Next run the backup application from the systems settings in the side bar.
/hometo Ubuntu One, a flash stick, a floppy (oops scratch that) or burn onto a CD, or other HDD in system or external HDD drive or use whatever backup program you want.
Then reverse the backup using restore, pick in the backup program you used, and now \home is back.
Lastly, reverse the
sudo dpkg --get-selections < app-backup-list.txt sudo apt-get -y update sudo apt-get dselect-upgrade
Restoring home, gets all settings for your applications. I installed a SSD, and copied all the files, but I still did a backup just in case.
PS: This avoids, actually backing up, massive apps' machine code, etc. Make sure your download folder is clean. Not full of machine code or videos, and then burn the
downloadfolder to a DVD using k3B.