Updating NetBSD pkgsrc packages and the base system

Tags: netbsd, unix, pkgsrc

Allow building of system and pkgsrc as non-root user

The NetBSD wiki1 provides instructions for building the sytem as a non root user. One of the options is to change the permissions on /usr/src so that you can maintain your source trees in the standard /usr/src directory as a non-root user. I have followed the same approach for /usr/pkgsrc also. The following needs to be undertaken once to set the permissions as writable for a non-root user:

# chown -R <username>:wsrc /usr/src /usr/pkgsrc
# usermod -G wsrc <username>

Building the base system and keeping it up to date

Detailed instructions for building the base system2 and keeping it updated3 are provided on the NetBSD website. You should check out the detailed instructions provided there rather than rely on the summary information provided here.

The first step is to obtain the sources for the base system. There are several options. I chose to use CVS. It is necessary to set the CVSROOT environment variable which tells the CVS program where to fetch the files from:

$ export CVSROOT="anoncvs@anoncvs.NetBSD.org:/cvsroot"

I also added same export command to my shell ~/.profile file.

I am running the NetBSD development branch on my Raspberry Pi 4. To obtain the sources for NetBSD-current:

$ cd /usr
$ cvs checkout -A -P src

To update the sources later issue cvs update -A -dP in the /usr/src directory. Before you try to rebuild the system check that the most version of -current actually built successfully for your machine architecture4.

To manually build the base system from the updated sources the NetBSD website suggests the following5:

$ cd /usr/src
$ ./build.sh -O ../obj -T ../tools -U -u distribution  # To build the userland.
$ ./build.sh -O ../obj -T ../tools -U -u kernel=GENERIC64  # To build the kernel.
$ cd ../obj/sys/arch/<ARCH>/compile/GENERIC64/ # To install the kernel (substitute your machine architecture), in my case "evbarm"

To find out what your machine architecture is type uname -m. For the Raspberry Pi 4 this is evbarm. Typing uname -a will provide more information including the date the kernal was built and the version number. This can be useful to check that the running kernel version after an update is the version you are expecting for example.

Backup the old kernel and copy the new kernel into place before rebooting

$ su
# mv /netbsd /netbsd.old
# cp netbsd /netbsd
# shutdown -r now

Assuming the new kernel boots successfully, install the new userland:

$ cd /usr/src
$ su
# ./build.sh -O ../obj -T ../tools -U install=/

If the new kernel doesn’t boot you will likely need to connect to the Pi via a serial connection so that you can interupt the boot process and select the old kernel to boot boot netbsd.old and copy the old kernel back from /netbsd.old to /netbsd.

Once you’ve installed the new kernel and userland use the postinstall utility to check and fix things that can be easily automated:

# /usr/sbin/postinstall -s /usr/src check
# /usr/sbin/postinstall -s /usr/src fix

Then update the system and configuration files using the etcupdate utility:

#  /usr/sbin/etcupdate -s /usr/src

There are also automated build and upgrade options if you would prefer.

Build problems

Running -current in particular can be a rocky ride and breakages are common. Even with a build that is known to work there are numerous local reasons why a userland or kernel compilation may not be successful. A great deal of useful information is presented in the /usr/src/UPDATING and /usr/src/BUILDING files in the source tree. A problem commonly encountered is stale files hanging around in source or building directories. Running a make cleandir in the relevant directory may do the trick or running the build.sh commands above without the -u option may be worth a try. When run with the -u option the build system only builds tools which have changed since they were built previously. Additionally, the /usr/src/UPDATING files provide a listing for a shell script which can be run to clean out the whole source tree ready for a full rebuild.

Obtaining the pkgsrc sources

To fetch the pkgsrc -current branch6 run:

$ cd /usr && cvs -q -z2 -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P pkgsrc

Find software to install from pkgsrc

You can search the pkgsrc tree to find which directory the software you want to install is located7:

find /usr/pkgsrc -type d -name "python"

or issue the command8 (this may take a while first time around):

# cd /usr/pkgsrc/
# make search key='python'

from the /usr/pkgsrc directory.

Building a package

To build a package9 cd into the relevant directory under the pkgsrc tree. You can see what’s in the source directory which a quick ls and read a description of the package with cat DESC. Then issue make to build the various components of a package, make install to actually install software and make clean to clean out all the build files in the work directory. A make clean-depends does the same but for all the dependicies which were built automatically.

$ cd /usr/pkgsrc/sysutils/sysbuild
$ make
$ make install
$ make clean
$ make clean-depends

If you want to download all the source and dependencies at once before you start building then issue the following command:

$ make fetch-list | sh

To find out the options affect how a program is built10:

$ make show-options

To change options:

$ make PKG_OPTIONS.<package_name>="-option1 option2" install clean

Listing an option enables it. Listing an option with a “-” before it disables the option. To make these option changes permanent for every future build or upgrade of this package, put a similar line in /usr/pkg/etc/mk.conf:

 . PKG_OPTIONS.<package_name>=-option1 option2

Updating pkgsrc

To update the pkgsrc sources via CVS 11 use the following command:

$ cd /usr/pkgsrc && cvs update -dP

Finding installed packages which need updating and update them including dependencies

pkg_rolling-replace replaces packages one by one after sorting the packages by dependencies to avoid most duplicate rebuilds1213. Once pkg_rolling-replace is installed you can update the packages through the following steps.

$ pkg_rolling-replace -u

Find out of date packages on your system

Install pkgtools/lintpkgsrc and run lintpkgsrc with the “-i” argument to check if your packages are up-to-date, e.g.

$ lintpkgsrc -i

You can then use make update to update the package on your system and rebuild any dependencies.

Update a specific package

To upgrade a port or package; cd into the relevant directory for the package in the pkgsrc tree and then issue make update as root. This also rebuilds any dependencies.

Removing a package

Cd into the relevant pkgsrc subdirectory and make deinstall.

What happens when pkgsrc upgrades go wrong

Sometimes bad stuff happens and you can’t fix it. The pkg_admin tool offers a number of utilty functions which can help fix issues with pkgsrc. For example:

# pkg_admin rebuild-tree

will rebuild the dependency tree and is worth a try to fix package dependency resolution inconsistences. The man page is well worth a careful read if you get stuck.

In extremis you could delete all your packages and start again. Get a list of packages installed14:

# pkg_info -a > pkglist

Remove all the packages:

# pkg_info -a | sed 's/ .*//' | tail -r | while read p ; do pkg_delete $p ; done

And reinstall just those you need:

# cat pkglist | (while read pp ; do cd /usr/pkgsrc/$pp ; make && make install ; done)

  1. https://wiki.netbsd.org/guide/updating/#index6h3↩︎

  2. https://www.netbsd.org/docs/guide/en/part-compile.html↩︎

  3. https://www.netbsd.org/docs/guide/en/chap-updating.html↩︎

  4. https://releng.netbsd.org/cgi-bin/builds.cgi↩︎

  5. https://www.netbsd.org/docs/guide/en/chap-updating.html↩︎

  6. https://www.netbsd.org/docs/pkgsrc/getting.html#getting-via-cvs↩︎

  7. https://opensource.com/article/19/11/pkgsrc-netbsd-linux↩︎

  8. https://www.dragonflybsd.org/docs/howtos/HowToPkgsrc/↩︎

  9. https://www.netbsd.org/docs/pkgsrc/using.html#how-to-build-and-install↩︎

  10. https://www.dragonflybsd.org/docs/howtos/HowToPkgsrc/↩︎

  11. https://www.netbsd.org/docs/pkgsrc/getting.html#uptodate-cvs↩︎

  12. https://www.dragonflybsd.org/docs/howtos/HowToPkgsrc/↩︎

  13. https://wiki.netbsd.org/pkgsrc/how_to_upgrade_packages/#index7h2↩︎

  14. https://wiki.netbsd.org/pkgsrc/how_to_upgrade_packages/#index7h2↩︎