First, some explanation re: ‘stage3.5’. I had planned on calling this a ‘stage4’, but there are some confusion regarding the term ‘stage4’, even amongst Gentoo users. To prevent howls of protests that might possibly be raised by Gentoo graybeards, I decided to call this a ‘stage3.5’ instead.

Okay, what the heck is a ‘stage3.5’? The numbering implied that it’s a continuation of ‘stage3’. And in fact, it is. It’s a tarball containing:

• A customized /etc/make.conf
• A customized /etc/portage/ (accepted keywords, package-specific use flags, rsync excludes, etc)
• An updated portage
• A @world that’s gone through emerge -uDN and revdep-rebuild

In other words, all the stuffs just before emerging the kernel sources.

Here’s how:

Like I mentioned, do everything but stop just before you emerge the kernel sources. The reason why: Upgrading the kernel is perhaps the most invasive thing you can do on a Linux system (on a par with upgrading gcc and/or glibc). By not emerging the kernel sources, you have a ‘clean-ish slate’ to build the kernel upon.

(In addition, the kernel sources add considerably to the stage3.5 tarball’s size).

After reaching that state, the next thing is to tarball the whole / tree. You’ll want to not backup the /proc, /sys, /boot, and most of the /usr/portage tree. You’ll also want to backup the ‘original’ /dev (instead of the /dev overlaid by the BootCD’s /dev).

The easiest way is to temporarily ‘escape’ from the chroot-ed environment (piece of cake if you’ve started screen before you chroot-ed: just do a ‘C-a c’) and bind-mount /mnt/gentoo to /mnt/stage3.5 (remember to mkdir /mnt/stage3.5 first).

mount --bind /mnt/gentoo /mnt/stage3.5

this will ‘prune’ non-same-filesystem directories (e.g., /boot, /proc, /sys, and the overlaid /dev), resulting in a / tree identical to stage3’s but with customized content.

(Important note: If you plan on running the Gentoo system as a guest VM on top of XenServer, now is the perfect time to create the necessary hvc* and xvd* devices in /mnt/stage3.5/dev, if you haven’t created them before it gets overlaid by udev’s /dev — see my previous post ;) )

Next, you want to exclude most of the usr/portage tree, but not all. Which ones? The ones in the portage-latest tarball, and the files in usr/portage/distfiles. Easy way:

tar tf portage-latest.tar.xz |
    awk -F "/" '$2!="" {print "usr/portage/"$2}' |
    sort -u > ~/exclude.portage
echo 'usr/portage/distfiles/*' >> ~/exclude.portage
echo portage-latest.tar.xz >> ~/exclude.portage

this excludes the files in usr/portage, the files in usr/portage/distfiles and the category directories under usr/portage. No need to specify the 2nd level directories (i.e., the package-name directories).

Finally, since we have to make the tarball inside /mnt/stage3.5 (the tmpfs of the BootCD might not be big enough), we’ll want to exclude the stage3.5 tarball itself. Do the following:

cd /mnt/stage3.5
export stage35name=your_stage3.5_tarball_name
tar c -J --exclude "$stage35name" -X ~/exclude.portage -f $stage35name *

• Note 1: if you want a ‘progress indicator’, add --checkpoint=.200 right after the -J. This will print out a dot every 200 files. Change the number to your liking)

• Note 2: I personally name my stage3.5 tarballs like this: stage3.5_hostname_YYYYMMDD-HHMM.tar.xz.

• Note 3: Append a printf "\007" to alert you when stage3.5 is complete.

Here’s an example:

export stage35name=stage3.5_MYMACHINE_$(date +%Y%m%d-%H%M).tar.xz
tar c -J --checkpoint=.200 --exclude "$stage35name" -X ~/exclude.portage -f $stage35name * && printf "\nDONE.\007\n"

After tar completes, you can pull the file using scp and put it on a different system. Don’t forget to unmount the stage3.5 directory! ( cd ~ ; umount /mnt/stage3.5 )

Done! Now when you need to create a Gentoo system with the same USE flags, you no longer have to waste too much time waiting for emerge -uDN @world to complete. (You’ll still want to do it to bring the stage3.5 files up-to-date, though. But instead of tens of packages needing to be emerged/updated, you’ll see maybe only 3 or 4 packages need to be updated — provided your stage3.5 tarball is not too old).

To use the stage3.5 tarball, just follow the Gentoo installation handbook, but replacing the stage3 tarball with the stage3.5 tarball. Then go straight to the step where you emerge your kernel sources, and pick up from there.

There! My tip on making a ‘stage3.5’ tarball. Hopefully this can be of some use to you.

Feel free to point out any errors :)

Until my next dispatch, then. Keep strong!

Update 1: I’ve made a slight mistake with the awk command during extraction of portage-latest. The $2 should not be empty. The version above has been fixed.

Update 2: Added an example tar command I use.

Update 3: Forgot to exclude the portage-latest tarball. Fixed.