How to create a local APT repository?

  • I would like to build my own local repository on my LAN, so that machines on the LAN can update and upgrade from it. I want to download the packages and store them on my local server so that I can update, upgrade, install, etc, from it without using the internet.

    I don't think it is a duplicate. What maythux wants to accomplish is create his own repository server for use with aptitude. What Keryx does is replace aptitude as package manager and create external sources for packages.

  • Correct answer

    8 years ago

    *To make an offline Repository Over LAN *

    Install a Local Apache Webserver

    # apt-get install apache2
    

    By default, Debian's Apache package will set up a website under /var/www on your system. For our purposes, that's fine, so there's no reason to do anything more. You can easily test it by pointing your favorite browser at http://localhost You should see the default post-installation web page which is actually stored in /var/www/index.html


    Create a Debian Package Repository Directory

    chose to create a directory /var/www/debs for this. Under it, you should create "architecture" directories, one for each architecture you need to support. If you're using just one computer (or type of computer), then you'll only need one -- typically "i386" for 32-bit systems or "amd64" for 64 bit. If you are using some other architecture, I'll assume you probably already know about this. Now just copy the ".deb" package files for a given architecture into the appropriate directories. If you now point your favorite web browser at http://localhost/debs/amd64 (for example) you'll see a listing of the packages for 64 bit systems.


    Create a Packages.gz file

    Now we need to create a catalog file for APT to use. This is done with a utility called "dpkg-scanpackages". Here's the commands I use to update the AMD64 packages on my LAN:

    # cd /var/www/debs/
    
    # dpkg-scanpackages amd64 | gzip -9c > amd64/Packages.gz
    




    Make the repository known to APT

    Now the only thing left to do is to let APT know about your repository. You do this by updating your /etc/apt/sources.list file. You'll need an entry like this one:

    deb http://localhost/debs/ amd64/

    I used the actual hostname of my system instead of localhost -- this way the code is the same for all of the computers on my LAN, but localhost will do just fine if you are running just one computer.
    Now, update APT:

    # apt-get update
    

    Adding that line to /etc/apt/sources.list will break updates when not in the LAN, won't it?

    For Ubuntu 16.04, you might need to replace `/var/www/debs` in this answer with `/var/www/html/debs`. Or you will need extra steps to manually edit your apache configuration in `/etc/apache2`

  • From the Ubuntu Help wiki:

    There are 4 steps to setting up a simple repository for yourself

    1.Install dpkg-dev
    2.Put the packages in a directory
    3.Create a script that will scan the packages and create a file apt-get update can read
    4. Add a line to your sources.list pointing at your repository

    Install dpkg-dev

    Type in a terminal

    sudo apt-get install dpkg-dev
    

    The Directory

    Create a directory where you will keep your packages. For this example, we'll use /usr/local/mydebs.

    sudo mkdir -p /usr/local/mydebs
    

    Now move your packages into the directory you've just created.

    Previously downloaded Packages are generally stored on your system in the /var/cache/apt/archives directory. If you have installed apt-cacher you will have additional packages stored in its /packages directory.

    The Script update-mydebs

    It's a simple three liner:

    #! /bin/bash
     cd /usr/local/mydebs
     dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
    

    Cut and paste the above into gedit, and save it as update-mydebs in ~/bin. (the tilde '~' means your home directory. If ~/bin does not exist, create it: Ubuntu will put that directory in your PATH. It's a good place to put personal scripts). Next, make the script executable:

    chmod u+x ~/bin/update-mydebs
    
    How the script works:
    

    dpkg-scanpackages looks at all the packages in mydebs, and the output is compressed and written to a file (Packages.gz) that apt-get update can read (see below for a reference that explains this in excruciating detail). /dev/null is an empty file; it is a substitute for an override file which holds some additional information about the packages, which in this case is not really needed. See deb-override(5) if you want to know about it.

    Sources.list

    add the line

    deb file:/usr/local/mydebs ./
    

    to your /etc/apt/sources.list, and you're done.

    CD Option

    You can burn the directory containing the debs to a CD and use that as a repository as well (good for sharing between computers). To use the CD as a repository, simply run

    sudo apt-cdrom add
    

    Using the Repository

    Whenever you put a new deb in the mydebs directory, run

    sudo update-mydebs
    sudo apt-get update
    

    Now your local packages can be manipulated with Synaptic, aptitude and the apt commands: apt-get, apt-cache, etc. When you attempt to apt-get install, any dependencies will be resolved for you, as long as they can be met.

    Badly made packages will probably fail, but you won't have endured dpkg hell.

    Could you explain the syntax on the line `dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz`. What's `/dev/null` doing there. I read the man page too, but it wasn't quite clear.

    @blade19899 I need a small bit of clarification, please. I want a repository with just a few select packages in it, not every package I ever touched. Am I correct that this technique will give me that ability? The goal here is to have a repository that a software installation group can use on an isolated LAN, far away from the temptations to apt-get the unneeded.

    @WesMiller I think u need I just edited his post!

    @blade19899 I'm sorry, I don't understand your answer.

    @WesMiller u need BigSack I just edited his post for grammar issues(I think its bin awhile) this is not my answer, but BigSack's

    @BigSack When do you run `update-mydebs`? Shouldn't it be run before `apt-get update` each time a system update is done?

    Adding the repo from command line: `sudo apt-add-repository "deb file:///usr/local/mydebs ./"`

    Always wanted to know how to create depositories, but never had the need to do it before. I'm going to bookmark this to try some day to test. In a broad sense I know how depositories work, but only in the sense of http requests.

    Note that if you use `deb file:/usr/local/mydebs ./` in your `/etc/apt/sources.list`, you may face the problem *E: The repository 'file:/home/path Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default*. To solve the problem use `deb [trusted=yes] file:/usr/local/mydebs ./`.

    Should the new line in `sources.list` go at the top or the bottom? Or does it not matter?

  • Creating an Authenticated Repository

    I've had a look at the answers here and on other sites and most have the (IMHO big) disadvantage that you're setting up an unauthenticated repository. This means you need to run apt-get with --allow-unauthenticated to install packages from it. This can be a security risk, especially in scripts where the packages you're installing might not all be from your local repository.

    Note that I haven't covered here how to make it available over the LAN, but that's fairly generic config using Apache or nginx (see the other answers here).

    Setup the repo directory

    mkdir /home/srv/packages/local-xenial
    cd /home/srv/packages/local-xenial
    

    Then add a line like this to sources.list:

    deb file:/home/srv/packages/local-xenial/ ./
    

    Adding and Removing Packages

    remove packages

    rm /home/srv/packages/local-xenial/some_package_idont_like
    

    add packages

    cp /some/dir/apackage.deb /home/srv/packages/local-xenial
    

    now run the following script which generates the Packages, Release and InRelease files and signs them with your gpg private key:

    #!/bin/bash
    
    if [ -z "$1" ]; then
           echo -e "usage: `basename $0` DISTRO
    where DISTRO is the Ubuntu version codename (e.g. 14.04 is trusty)\n
    The way to use this script is to do the changes to the repo first, i.e. delete or copy in the .deb file to /srv/packages/local-DISTRO, and then run this script\n
    This script can be run as an unprivileged user - root is not needed so long as your user can write to the local repository directory"
    else
        cd /srv/packages/local-"$1"
    
        # Generate the Packages file
        dpkg-scanpackages . /dev/null > Packages
        gzip --keep --force -9 Packages
    
        # Generate the Release file
        cat conf/distributions > Release
        # The Date: field has the same format as the Debian package changelog entries,
        # that is, RFC 2822 with time zone +0000
        echo -e "Date: `LANG=C date -Ru`" >> Release
        # Release must contain MD5 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
        echo -e 'MD5Sum:' >> Release
        printf ' '$(md5sum Packages.gz | cut --delimiter=' ' --fields=1)' %16d Packages.gz' $(wc --bytes Packages.gz | cut --delimiter=' ' --fields=1) >> Release
        printf '\n '$(md5sum Packages | cut --delimiter=' ' --fields=1)' %16d Packages' $(wc --bytes Packages | cut --delimiter=' ' --fields=1) >> Release
        # Release must contain SHA256 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
        echo -e '\nSHA256:' >> Release
        printf ' '$(sha256sum Packages.gz | cut --delimiter=' ' --fields=1)' %16d Packages.gz' $(wc --bytes Packages.gz | cut --delimiter=' ' --fields=1) >> Release
        printf '\n '$(sha256sum Packages | cut --delimiter=' ' --fields=1)' %16d Packages' $(wc --bytes Packages | cut --delimiter=' ' --fields=1) >> Release
    
        # Clearsign the Release file (that is, sign it without encrypting it)
        gpg --clearsign --digest-algo SHA512 --local-user $USER -o InRelease Release
        # Release.gpg only need for older apt versions
        # gpg -abs --digest-algo SHA512 --local-user $USER -o Release.gpg Release
    
        # Get apt to see the changes
        sudo apt-get update
    fi
    

    Example Contents of conf/distributions file

    Origin: My_Local_Repo Label: My_Local_Repo Codename: xenial Architectures: i386 amd64 Components: main Description: My local APT repository SignWith: 12345ABC

    Links

    https://wiki.debian.org/RepositoryFormat

    http://ubuntuforums.org/showthread.php?t=1090731

    https://help.ubuntu.com/community/CreateAuthenticatedRepository

    @Phillip your edit used `date -Rc`, I corrected it to `date -Ru` assuming that's what you meant from the edit description

    Thanks, I only recently started getting warnings from apt about this due to the date generated being in the local TZ and not UTC. I fixed it in my own script but forget to edit it here

    @KevinJohnson I've updated the main answer now with an example of that file from my local apt repo

  • You can also setup local source server by nginx and reprepro:

    1. Install debian packages

      sudo apt-get install reprepro nginx 
      
    2. make directories for reprepro and edit it

      sudo mkdir -p /srv/reprepro/ubuntu/{conf,dists,incoming,indices,logs,pool,project,tmp}
      
      $ cd /srv/reprepro/ubuntu/
      $ sudo chown -R `whoami` . # changes the repository owner to the current user
      

      /srv/reprepro/ubuntu/conf/distributions

      Origin: Your Name
      Label: Your repository name
      Codename: karmic
      Architectures: i386 amd64 source
      Components: main
      Description: Description of repository you are creating
      SignWith: YOUR-KEY-ID
      

      /srv/reprepro/ubuntu/conf/options

      ask-passphrase
      basedir .
      
    3. Include it in reprepro, build it

      $ reprepro includedeb karmic /path/to/my-package_0.1-1.deb \
      # change /path/to/my-package_0.1-1.deb to the path to your package
      
    4. Config nginx:

      /etc/nginx/sites-available/vhost-packages.conf

      server {
        listen 80;
        server_name packages.internal;
      
        access_log /var/log/nginx/packages-access.log;
        error_log /var/log/nginx/packages-error.log;
      
        location / {
          root /srv/reprepro;
          index index.html;
        }
      
        location ~ /(.*)/conf {
          deny all;
        }
      
        location ~ /(.*)/db {
          deny all;
        }
      }
      
    5. Optimize bucket size:

      /etc/nginx/conf.d/server_names_hash_bucket_size.conf

      server_names_hash_bucket_size 64;
      

    Reference to Install Guide Link

    Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.

    @elprup: you must have forgotten to update that answer :)

    reprepor does not support multiple versions of the same package. It sounds strange, but this is the way reprepro works

  • The instructions in @BigSack's answer and Ubuntu's official wiki post didn't work for me on Ubuntu 18.04, until I made these two changes:

    1. Generate a plain, uncompressed Packages file (when executing this, the working directory must be where all packages are located)

      cd /usr/local/mydebs
      dpkg-scanpackages -m . > Packages
      
    2. Add the following entry in /etc/apt/sources.list

      deb [trusted=yes] file:/usr/local/mydebs ./
      
  • You might want to take a look at apt-mirror and apt-cacher .

    Here is a guide on how to install and use it.

    A 5-Year old answer of yours is being flagged as LQ. If it would be deleted, go to meta and ask for an undelete. I voted to **keep open**, but it needs an edit! **;-)**

  • There are several reasons you may want to create a local repository. The first is that you want to save on bandwidth if you have multiple Ubuntu machines to update. For example if you had 25 Ubuntu machines that all needed updating at least once a week, you would significantly save bandwidth because you could do all but the repository locally.

    Most organizations have decent bandwidth for their network gateways but this bandwidth is a precious commodity that needs to be used wisely.

    Many organizations still have routers with 10MB or 100MB limits at the gateway but 1 GB network connections internally so bandwidth could be better used internally. The second reason for creating your own repository is that you can control what applications are loaded on your internal Ubuntu machines.

    You can remove any applications your organization does not want to use on the local network from the repository that updates the machines. Even better, you can create a test box and test applications and versions before you allow them to roll out into your network assuring security and stability.

    You first have to setup a mirror, to do that you need to Just press Ctrl+Alt+T on your keyboard to open Terminal. When it opens, run the command below.

    apt-get install apt-mirror 
    

    Once you have your set up apt-mirror you can start your download of the repository with this command.

    apt-mirror /etc/apt/mirror.list1

    Read on

    1Source:Create an Ubuntu Repository

    sorry, link is dead

  • To make an offline local Repository
    1. make a dir accessible (atleast by root)

    sudo mkdir /var/my-local-repo

    1. copy all the deb files to this directory.
    2. scan the directory

    sudo dpkg-scanpackages /var/my-local-repo /dev/null > /var/my-local-repo/Packages

    1. add the local repository to sources

    echo "deb file:/var/my-local-repo ./" > /tmp/my-local.list

    sudo mv /tmp/my-local.list /etc/apt/sources.list.d/my-local.list

    sudo apt-get update

    The more or less same thing is also on official wiki: Repositories/Personal - Community Help Wiki

  • I tried to use apt-rdepends like in the selected answer, but when I tried to install the package from my local repository, it complained about missing dependencies.

    apt-rdepends wasn't listing some of the dependencies for my package. I suspect it has something to do with the fact, that apt-cache show shows multiple records for it.

    Instead I used apt-cache depends, and that did the trick:

    Getting a recursive list of dependencies

    apt-cache depends <packagename> -i --recurse
    

    -i: important dependencies only --recurse: recursive

    Turn it into a digestible list

    • Removing symbols & spaces: | tr -d "|,<,>, "
    • Removing Depends: & PreDepends: | sed -e 's/^Depends://g' | sed -e 's/^PreDepends://g'
    • Sorting the list: | sort
    • Only unique values: | uniq > list.txt

    Complete command:

    apt-cache depends <packagename> -i --recurse | tr -d "|,<,>, " | sed -e \
    's/^Depends://g' | sed -e 's/^PreDepends://g' | sort | uniq > list.txt
    

    Download the packages

    for i in $( cat list.txt ); do apt-get download $i; done;
    

    Scan for the packages and turn it into Packages.gz

    dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
    

    Might be a good idea to reference which answer you're talking about...

  • I have done using apt-mirror.

    Its good but you need to have more hard drive space as it will be syncing with repos server.

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM