Jake's Tech Blagh

[Updated for OpenBSD 4.4]



I got my Soekris 5501 box via UPS a while ago. This is my guide to setting it up with OpenBSD on CompactFlash. Storing portions of the filesystem in ram, to minimize writes on CF card and to maximize speed. With this setup, I get less login latency and faster boot times than I got from my old beige box Intel P3 500Mhz system. yay! :)

We're going to install OpenBSD on our soekris 5501 via pxeboot. Let's start by configuring inetd, dhcpd and /tftpboot on an already up and running OpenBSD machine (connected to the same internal network as our soekris).

Hero we go;

Uncomment these two lines in inetd.conf:

17 tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot
18 tftp dgram udp6 wait root /usr/libexec/tftpd tftpd -s /tftpboot

Create /tftpboot and copy pxeboot and bsd.rd there:

mkdir /tftpboot
cp /usr/mdec/pxeboot /tftpboot
cp /bsd.rd /tftpboot

In /etc/dhcpd.conf add this line (under 'option range x.x.x.x - x.x.x.x'):

filename "pxeboot";

Create a boot.conf file which will on boot change serial port speed to 19200 baud and standard output terminal to com0:

echo "stty com0 19200" > /tftpboot/boot.conf
echo "set tty com0" >> /tftpboot/boot.conf
echo "boot tftp:/bsd.rd" >> /tftpboot/boot.conf

Now we're done with our pxeboot configuration. Time to restart dhcpd and inetd:

kill `cat /var/run/inetd.pid`
/usr/sbin/inetd
kill `ps aux | grep '_dhcp' | grep dhcpd | awk '{print $2}'`
/usr/sbin/dhcpd [Interface on internal network]

Connect serial- and network cable and power on your soekris box. When connected via serial console (at 19200 baud), Press ctrl +P then write 'boot f0' to pxeboot:

comBIOS ver. 1.33c 20080626 Copyright (C) 2000-2008 Soekris Engineering.

net5501

0512 Mbyte Memory CPU Geode LX 500 Mhz

Pri Mas SanDisk SDCFX-8192 LBA Xlt 996-255-63 8003 Mbyte

3 Seconds to automatic boot. Press Ctrl-P for entering Monitor.

[Press ctrl + P]

> boot f0

Intel UNDI, PXE-2.0 (build 082)
Copyright (C) 1997,1998,1999 Intel Corporation
VIA Rhine III Management Adapter v2.43 (2005/12/15)

CLIENT MAC ADDR: 00 00 24 CA 98 EC
CLIENT IP: 192.168.1.11 MASK: 255.255.255.0 DHCP IP: 192.168.1.1
GATEWAY IP: 192.168.1.1
probing: pc0 com0 com1 pci pxe![2.1] mem[639K 511M a20=on]
disk:
net: mac 00:00:24:ca:98:ec, ip 192.168.1.11, server 192.168.1.1
>> OpenBSD/i386 PXEBOOT 2.02
switching console to com0
com0: changing speed to 19200 baud in 5 seconds, change your terminal to match!

com0: 19200 baud
booting tftp:bsd.rd: 4780308+874136 [52+178240+163973]=0x5b821c
entry point at 0x200120a
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2008 OpenBSD. All rights reserved. http://www.OpenBSD.org

OpenBSD 4.4 (RAMDISK_CD) #857: Tue Aug 12 17:31:49 MDT 2008
deraadt@i386.openbsd.org:/usr/src/sys/arch/i386/compile/RAMDISK_CD
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD" 586-class) 500 MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX
real mem = 536440832 (511MB)
avail mem = 512524288 (488MB)
mainbus0 at root

...

(I)nstall, (U)pgrade or (S)hell?

While in 'disklabel' during install, create these partitions:

a: / = 40% of your CF disk space
b: swap = alocate '1' sector, just so OpenBSD don't complain during boot. We don't want it to swap to CF.
d: /mfs = alocate 200MB, it will hold your /dev, /etc, /root and /var ram-backup. Otherwise you will lose info on reboot and power failure.
e: /home = 40% of your CF diskspace.

Install these packages:

[X] bsd
[X] bsd.rd
[ ] bsd.mp
[X] base43.tgz
[X] etc43.tgz
[X] misc43.tgz
[X] comp43.tgz
[X] man43.tgz
[ ] game43.tgz
[ ] xbase43.tgz
[ ] xetc43.tgz
[ ] xshare43.tgz
[ ] xfont43.tgz
[ ] xserv43.tgz

When asked about 'default console', answer yes and enter 19200:

Change the default console to com0? [no] y
Available speeds are: 9600 19200 38400 57600 115200
Which one should com0 use? (or 'done') [9600] 19200

After you've complete the installation script, reboot.
Your soekris machine should now automatically start OpenBSD.


Let's configuration rsync and our MFS (Memory filesystem).

Our goal is to mount '/' read only, to preserve CF writes.
To do this we need to create four memory filesystems and rsync them to our '/mfs' backup at regular intervals and before reboot.

'/dev' need write access since users have to own their tty on login.
'/var' need write access for logs, sockets, pids and db files.
'/etc' need write access since resolv.conf might get changed via dhclient.
'/root' need write access for login and should not exist as a symbolic link.

Let's prepare for readonly '/' and fill the MFS backup folder;

Move '/tmp' and replace it with a link to '/var/tmpinroot':

mv /tmp /var/tmpinroot
ln -s /var/tmpinroot/ /tmp

Create '/dev' inside the MFS backup folder and make devices:

mkdir -p /mfs/dev
cp -p /dev/MAKEDEV /mfs/dev/
cd /mfs/dev/ && ./MAKEDEV all

Copy contents of /root to the MFS backup folder:
('/root' can be symbolic linked like we did '/tmp', but then /etc/security will complain during execution of 'crontab daily'.)

cp -Rp /root /mfs/

Copy contents of '/etc' to the MFS backup folder:

cp -Rp /etc /mfs/

Link '/usr' whatis database to '/var':

rm /usr/local/man/whatis.db
rm /usr/share/man/whatis.db
mkdir -p /var/usr/local/man/
mkdir -p /var/usr/share/man/
ln -s /var/usr/local/man/whatis.db /usr/local/man/whatis.db
ln -s /var/usr/share/man/whatis.db /usr/share/man/whatis.db

Rebuild whatis database:

sh /etc/weekly

Copy contents of '/var' to the MFS backup folder:

cp -Rp /var /mfs/

Setup a MFS rsync script.

Put this script in /bin/mfs-sync:

#!/bin/sh

MFS='/dev/wd0d' # MFS backup partiton.

sudo mount -uw $MFS && mount | grep $MFS
printf "Synchronizing in memory root with root backup ... "
sudo /usr/local/bin/rsync -vhaz --delete /root/ /mfs/root/
printf "Synchronizing in memory etc with etc backup ... "
sudo /usr/local/bin/rsync -vhaz --delete /etc/ /mfs/etc/
printf "Synchronizing in memory var with var backup ... "
sudo /usr/local/bin/rsync -vhaz --delete -f "- *.sock" -f "- **/empty/dev/log" /var/ /mfs/var/
sudo mount -ur $MFS && mount | grep $MFS

( mfs-sync will filter /var so it does not sync socket files, since they lose their context when mfs mounted. New system sockets will be created on boot. So you won't be getting messages like this; pax: /mfs/./cron/tabs/.sock skipped. Sockets cannot be copied or extracted )

Give script execution rights:

chmod +x /bin/mfs-sync

Install rsync:

export PKG_PATH=ftp://ftp.eu.openbsd.org/pub/OpenBSD/4.4/packages/i386/
pkg_add rsync

Add mfs-sync to rc.shutdown, to have it automatically run before shutdown and reboot:

echo "/bin/mfs-sync" >> /etc/rc.shutdown

Add mfs-sync to crontab weekly, to have it automatically run once a week:

echo "/bin/mfs-sync >> /var/log/mfs-sync 2>&1" >> /etc/weekly

/etc/daily - is executed once a day, at 1:30 AM.
/etc/weekly - is executed once a week, every Saturday at 3:30 AM.
/etc/monthly - is executed the first day of the month, at 5:30 AM.

MFS partitons are defined in 512 byte sectors, so in order to create a 50MB /var, 10MB /etc, 5MB /dev and 5MB /root partiton, we need to do some simple calculations.

( 2^13 bit / 8 = 1024 byte
1024^2 = 1048576 byte = 1 MiB
0.000001 (mega) * 1048576 = 1.048576 => The MB to MiB multiplier. )

1.048576 * 50 MB = 52,4288 MiB = 52428800 byte
52428800 byte / 512 byte = 102400 'sectors'

1.048576 * 10 MB = 10,48576 MiB = 10485760 byte
10485760 byte / 512 byte = 20480 'sectors'

1.048576 * 5 MB = 5,24288 MiB = 5242880 byte
5242880 byte / 512 byte = 10240 'sectors'

Define four (swap) MFS file systems in /etc/fstab below the '/mfs' entry, using the 'sectors' we just calculated above. Change '/', '/mfs' and '/home' to readonly. Add 'softdep' and 'noatime' to speed up filesystem access:

/dev/wd0a / ffs ro,noatime,softdep 1 1
/dev/wd0d /mfs ffs ro,noatime,softdep 1 2
swap /root mfs rw,nosuid,-P=/mfs/root,-s=10240 0 0
swap /var mfs rw,nosuid,-P=/mfs/var,-s=102400 0 0
swap /dev mfs rw,nosuid,-P=/mfs/dev,-s=10240,-i=128 0 0
swap /etc mfs rw,nosuid,-P=/mfs/etc,-s=20480 0 0
/dev/wd0e /home ffs rw,noatime,softdep 1 2

( 5 MiB FFS partition will give you 640 inodes, an inode every (2^13) 8192 bytes default. But '/dev' has 1295 devices/files in OpenBSD 4.4. So that creates a problem, which we solve by using the -i=128 flag. It will tell mount_mfs to reserve an inode every (2^7) 128 bytes instead. So now we won't run out of inodes :)

Execute MFS sync script.

/bin/mfs-sync

mount MFS partitons:

mount /dev
mount /root
mount /etc
mount /var

mfs-sync script will give you output similar to this;

/dev/wd0d on /mfs type ffs (local, noatime)
Synchronizing in memory root with root backup.
sending incremental file list

sent 89 bytes received 12 bytes 202.00 bytes/sec
total size is 1.30K speedup is 12.85
Synchronizing in memory etc with etc backup.
sending incremental file list
resolv.conf
resolv.conf.save

sent 4.73K bytes received 82 bytes 9.62K bytes/sec
total size is 2.55M speedup is 529.66
Synchronizing in memory var with var backup.
sending incremental file list
cron/log
db/dhclient.leases.vr0
db/dhcpd.leases
log/authlog
log/daemon
log/lastlog
log/messages
log/secure
log/wtmp
run/utmp
tmp/vi.recover/
tmpinroot/
www/logs/access_log

sent 123.11K bytes received 302 bytes 49.36K bytes/sec
total size is 6.58M speedup is 53.28
/dev/wd0d on /mfs type ffs (local, noatime, read-only)


If there are no errors in /var/log and you are happy with the setup, reboot. And experience MFS and read only OpenBSD. 8-D

Tip: To read/write enable '/' execute these two:

For WRITE => mount -uw /dev/wd0a && mount | grep '/dev/wd0a'
For READ => mount -ur /dev/wd0a && mount | grep '/dev/wd0a'


See these three blogs with info regarding similar MFS setups:
http://blog.spoofed.org/2007/12/openbsd-on-soekris-cheaters-guide.html
http://surricani.blogspot.com/2007/09/openbsd-and-readonly-filesystems.html
http://stsx.xs4all.nl/www.stsx.org/openbsd/obsd-bootsoekris.html