Mirror a PPA repositories using aptly – PHP (ppa:ondrej/php)

This is a simple example of how to mirror a PPA repository to a local server. The Ubuntu PPA to mirror is ppa:ondrej/php, which offers the user different PHP version generally not available in the Ubuntu installation. Of course, the user should be very careful about adding PPA repositories, because they are exactly what the abbreviation stands for Personal Package Archives.

If you want to know how to install and a brief description of what is aptly you may want to read our previous article – Install aptly under Ubuntu 18 LTS with Nginx serving the packages and the first steps

What we are going to do – this is what you need to have a mirror of an external application repository:

1. Install aptly in Ubuntu 18 LTS
2. Create a mirror in aptly
3. Create a snapshot of the mirror created before
4. Publish the snapshot to be used in other servers.

and at the last step there is an example how to use the mirror in your local machines.

STEP 1) Install aptly in Ubuntu 18.04 LTS.

As mentioned already you may follow our article on the subject – Install aptly under Ubuntu 18 LTS with Nginx serving the packages and the first steps. The following steps are based on this installation!
The aptly home directory is in “/srv/aptly”. We use the “aptly” user and change to it to manipulate the aptly installation.
Change the user to aptly, because under this user the mirror process will happen.

root@srv ~ # su - aptly
aptly@srv:~$

STEP 2) Create a mirror in aptly.

Prepare the keys (aptly needs to have the Ubuntu keys in its trustedkeys keyring):

aptly@srv:~$ gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver pool.sks-keyservers.net --recv-keys 4F4EA0AAE5267A6C
gpg: requesting key E5267A6C from hkp server pool.sks-keyservers.net
gpg: key E5267A6C: public key "Launchpad PPA for Ondxc5x99ej Sur�" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)

Here we’ve used the method to obtain the key from a GPG KEY server, but the key can be downloaded directrly from the original repository as suggested in the error message below.
If you are not sure where to download the key you could always just try to create the mirror ( in fact, this is in STEP 3) ) and get the error for missing key and how to obtain the key:

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=false -with-udebs=false php-bionic http://ppa.launchpad.net/ondrej/php/ubuntu bionic main
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Fri 27 Mar 2020 08:31:12 AM UTC using RSA key ID E5267A6C
gpgv: Can't check signature: public key not found

Looks like some keys are missing in your trusted keyring, you may consider importing them from keyserver:

gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver pool.sks-keyservers.net --recv-keys 4F4EA0AAE5267A6C

Sometimes keys are stored in repository root in file named Release.key, to import such key:

wget -O - https://some.repo/repository/Release.key | gpg --no-default-keyring --keyring trustedkeys.gpg --import

Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/Release...
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/Release.gpg...
gpgv: Signature made Fri 27 Mar 2020 08:31:12 AM UTC using RSA key ID E5267A6C
gpgv: Can't check signature: public key not found

Looks like some keys are missing in your trusted keyring, you may consider importing them from keyserver:

gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver pool.sks-keyservers.net --recv-keys 4F4EA0AAE5267A6C

Sometimes keys are stored in repository root in file named Release.key, to import such key:

wget -O - https://some.repo/repository/Release.key | gpg --no-default-keyring --keyring trustedkeys.gpg --import

ERROR: unable to fetch mirror: verification of detached signature failed: exit status 2

Two commands are needed for the aptly mirror:

1. create – create the mirror.
2. update – download the repository contents locally.

aptly@srv:~$ aptly mirror create -architectures=i386,amd64 -with-sources=false -with-udebs=false php-bionic http://ppa.launchpad.net/ondrej/php/ubuntu bionic main
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Fri 27 Mar 2020 08:31:12 AM UTC using RSA key ID E5267A6C
gpgv: Good signature from "Launchpad PPA for Ondxc5x99ej Sur�"

Mirror [php-bionic]: http://ppa.launchpad.net/ondrej/php/ubuntu/ bionic successfully added.
You can run 'aptly mirror update php-bionic' to download repository contents.
aptly@srv:~$ aptly mirror update php-bionic
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/InRelease...
gpgv: Signature made Fri 27 Mar 2020 08:31:12 AM UTC using RSA key ID E5267A6C
gpgv: Good signature from "Launchpad PPA for Ondxc5x99ej Sur�"
Downloading & parsing package files...
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/main/binary-i386/Packages.gz...
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/dists/bionic/main/binary-amd64/Packages.gz...
Building download queue...
Download queue: 714 items (262.89 MiB)
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/pool/main/p/php-gearman/php-gearman_2.0.6+1.1.2-7+ubuntu18.04.1+deb.sury.org+1_i386.deb...
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/pool/main/p/php7.3/php7.3-mysql_7.3.16-1+ubuntu18.04.1+deb.sury.org+1_amd64.deb...
.....
.....
Downloading http://ppa.launchpad.net/ondrej/php/ubuntu/pool/main/p/php7.1/php7.1-tidy_7.1.33-14+ubuntu18.04.1+deb.sury.org+1_amd64.deb...

Mirror `php-bionic` has been successfully updated.

Aptly will download 714 items (262.89 MiB) to create the local mirror. If something is not downloaded successfully, it will be reported at the end and to download the packages just execute the same command update again.

Here is a listing with all the mirrors:

aptly@apt3:~$ aptly mirror list           
List of mirrors:
 * [php-bionic]: http://ppa.launchpad.net/ondrej/php/ubuntu/ bionic

To get more information about mirror, run `aptly mirror show `.

And information for the mirror:

aptly@srv:~$ aptly mirror show php-bionic
Name: php-bionic
Archive Root URL: http://ppa.launchpad.net/ondrej/php/ubuntu/
Distribution: bionic
Components: main
Architectures: i386, amd64
Download Sources: no
Download .udebs: no
Last update: 2020-04-09 07:05:41 UTC
Number of packages: 714

Information from release file:
Acquire-By-Hash: yes
Architectures: amd64 arm64 armhf i386 ppc64el s390x
Codename: bionic
Components: main
Date: Fri, 27 Mar 2020  8:31:11 UTC
Description:  Ubuntu Bionic 18.04

Label: ***** The main PPA for supported PHP versions with many PECL extensions *****
Origin: LP-PPA-ondrej-php
Suite: bionic
Version: 18.04

STEP 3) Create a snapshot of the mirror

Now create the snapshot of the synchronized mirror. The snapshot will be published in the next step. We choose the name of the snapshot as the name of the mirror but you may want to create the snapshots with a date suffix.

aptly@srv:~$ aptly snapshot create php-bionic from mirror php-bionic

Snapshot php-bionic successfully created.
You can run 'aptly publish snapshot php-bionic' to publish snapshot as Debian repository.

Here is the detail information for the newly created snapshot.

aptly@srv:~$ aptly snapshot list
List of snapshots:
 * [php-bionic]: Snapshot from mirror [php-bionic]: http://ppa.launchpad.net/ondrej/php/ubuntu/ bionic

To get more information about snapshot, run `aptly snapshot show `.
aptly@srv:~$ aptly snapshot show php-bionic
Name: php-bionic
Created At: 2020-04-09 07:10:25 UTC
Description: Snapshot from mirror [php-bionic]: http://ppa.launchpad.net/ondrej/php/ubuntu/ bionic
Number of packages: 714
Sources:
  php-bionic [repo]

STEP 5) Publish the snapshot of the official mirror.

The last step to be able to use the local “official mirror” (in fact, you are going to use its snapshot and if you update the local mirror with an update command above you will continue to use the repository as it looks before the update). So updating the mirror must trigger a new snapshot and then a publish command (probably with a different name, a date suffix?).

aptly@srv:~$ aptly publish snapshot -architectures=i386,amd64 -component=main php-bionic php    
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:

Snapshot php-bionic has been successfully published.
Please setup your webserver to serve directory '/cdn/aptly/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
  deb http://your-server/php/ bionic main
Don't forget to add your GPG key to apt with apt-key.

You can also use `aptly serve` to publish your repositories over HTTP quickly.

STEP 6) Use the local mirror of ppa:ondrej/php

To use the mirror you’ve just published first, the key of this aptly (repository) server should be imported and then add the repository to the Ubuntu’s configuration files. As mentioned in the beginning, we installed the aptly software following our previous article – Install aptly under Ubuntu 18 LTS with nginx serving the packages and the first steps, so the public key of our aptly server is located in the /srv/aptly/.aptly/public/key.pub accessed from the web from “https://aptly.example.com/key.pub“.
Login in your client machine and import the key:

root@srv2:~# wget http://apt.example.com/key.pub
root@srv2:~# apt-key add ./key.pub
OK

Then replace the official repositories in /etc/apt/sources.list with the new ones:

deb https://aptly.exmaple.com/php bionic main

And update to download the metafiles and upgrade or install packages. apt program will access only your local repositories.

root@srv2:~# apt update
.....
Get:8 http://apt3.cdn-project.info/php bionic/main i386 Packages [78.4 kB]
Get:9 http://apt3.cdn-project.info/php bionic/main amd64 Packages [78.4 kB]
.....
Fetched 36,8 MB in 32s (1160 kB/s)                                                                                                                                         
Reading package lists... Done
Building dependency tree       
Reading state information... Done
384 packages can be upgraded. Run 'apt list --upgradable' to see them.

You see the accessed URLs for the metafiles and the downloads would be from the new local PHP repository.

The post Mirror a PPA repositories using aptly – PHP (ppa:ondrej/php) first appeared on Any IT here? Help Me!.