OpenBSD on Google Compute Engine

This tutorial outlines a simple way to get OpenBSD working on GCE, utilizing only OpenBSD to create the image and send up into gcloud.

Create the OpenBSD VM

We will first need to create a bridge interface for use by the vmd service on OpenBSD. This is done so that the VM's have quick access to the network. However, you can view other options as well here.

Create the interface file for bridge0 and bring up the interface:

echo 'add em0' > /etc/hostname.bridge0
sh /etc/netstart bridge0

Then go ahead and create/modify /etc/vm.conf, and add the following text:

switch "thebridge" {
    interface bridge0
}

This will place the VM we create next on the hosts physical NIC, as if it were a separate machine.

Next, download the latest OpenBSD install iso, here is a link to an OpenBSD mirror. Save the image in your home directory, then move on.

Now we need to create the actual disk image to install OpenBSD on:

vmctl create -s 20G openbsd.qcow2

The next step is actually starting the vmd service, start our VM, and then jump into the console:

rcctl start vmd
vmctl start -m 1G -L -i 1 -n thebridge -r install66.iso -d openbsd66.qcow2 openbsd66
vmctl console openbsd66

At this point, this becomes a standard OpenBSD installation process. Go ahead and answer the questions as you normally would, except for the following questions:

# The following installation vars need to be set:
IPv4 address for interface = dhcp
Change the default console to com0 = yes
Which speed should com0 use = 115200

After the installation has completed, go ahead and reboot the VM.

Configure the OpenBSD VM

Now that we are back inside the VM, we need to patch the system, as well as update some sysctl and networking values. But first, we may need to manually configure the networking in case the VM was unable to get a DHCP lease.

If the VM was unable to acquire an IP, create your own quickly with ifconfig and route, use an IP and netmask appropriate for your network:

ifconfig vio0 192.168.0.166 netmask 255.255.255.0
route add default 192.168.0.1 

Once the networking is working, go ahead and patch the system, and update some files as shown below:

# Patch the server:
syspatch

# Install curl, required for getting metadata for VM
pkg_add curl

# Add the following lines to /etc/sysctl.conf
net.inet.ipcomp.enable=1
net.inet.tcp.mssdflt=1420

I also wrote a small script saved at /etc/rc.local to set the hostname of the server as well as your save your user's authorized_keys, using values provided by Google's metadata server. This script is ran when the server is rebooted, or can also be ran manually. This script also can be used to launch a bootstrap script, specified via custom metadata with the "bootstrap" key. You can set custom metadata for your VM's by editing the VM, then scrolling down to the custom metadata fields. For example, if you specify a key/value pair of "bootstrap" : "https://raw.githubusercontent.com/findelabs/openbsd-ansible-deploy/master/bootstraps/bootstrap_basic_gcp.sh", /etc/rc.local will execute this script at the end of startup. This script url is then saved to a file to ensure that it is only ran once. This is useful if you have custom configurations needed added to the basic image.

#!/bin/sh

# Save to /etc/rc.local, to be ran on startup

### get instance hostname ###

instance_hostname=$(/usr/local/bin/curl -s -H "Metadata-Flavor: Google"  http://metadata.google.internal/computeMetadata/v1/instance/name)
current_hostname=$(hostname)

if [ "$instance_hostname" != "" ]
then
    if [ "$instance_hostname" != "$current_hostname" ]
    then
        echo "Setting hostname to $instance_hostname"
        hostname $instance_hostname
        echo $instance_hostname > /etc/myname
        rcctl restart syslogd >/dev/null
    else
        echo "Hostname is correct"
    fi
else
        echo "Could not discover hostname"
fi

### set up user's keys ###

instance_keys=$(/usr/local/bin/curl -s -H "Metadata-Flavor: Google"  http://metadata.google.internal/computeMetadata/v1/project/attributes/ssh-keys)

if [ "$instance_keys" != "" ]
then
    echo "$instance_keys" | while read line
    do
        username="$(echo $line | cut -d: -f1)"
        user_key="$(echo $line | cut -d: -f2)"
        key_comment="$(echo $line | awk '{print $NF}')"

        if [ "$username" != "" ]
        then
            mkdir -p /home/${username}/.ssh

            if [ "$(grep -c "$user_key" /home/${username}/.ssh/authorized_keys)" == "0" ]
            then
                echo "$user_key" >> /home/${username}/.ssh/authorized_keys
                chmod 600 /home/${username}/.ssh/authorized_keys
                echo "$username: added ssh-key $key_comment"
            else
                echo "$username: ssh-key $key_comment already exists"
            fi
        fi
    done
else
    echo "No keys found"
fi

### Check for bootstrap value ###

instance_bootstrap=$(/usr/local/bin/curl -s -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/bootstrap)

if [ "$instance_bootstrap" != "" ]
then
    if [ "$(echo "$instance_bootstrap" | grep -c 'was not found')" != "1" ]
    then
        if [ ! -f /var/log/bootstrap ] || [ "$(grep -c "$instance_bootstrap" /var/log/bootstrap)" == "0" ]
        then
            echo "Bootstrap: starting"
            export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
            ftp -V -o - $instance_bootstrap | sh
            echo $instance_bootstrap >> /var/log/bootstrap
        else
            echo "Bootstrap: completed"
        fi
    else
        echo "Bootstrap: metadata not found"
    fi
else
    echo "Bootstrap: empty curl"
fi

Convert the OpenBSD Disk Image

This should be all that is required to get a basic OpenBSD VM to work on Google Cloud. Now we need to package up the disk and send it up.

We first need to convert the qcow2 image to a raw disk, which will inflate the image to the full image size, so make sure that your system has the space available to use.

# Install qemu-3.1.0
pkg_add qemu-3.1.0

# Convert the qcow2 image to raw
qemu-img convert -f qcow2 -O raw openbsd.qcow2 disk.raw

Please note that the output file MUST be disk.raw, as Google is expecting this as a filename.

Now this is a large file, let's shrink that down with tar. BSD tar sometimes has issues with large files, but gtar is able to handle at least 20G, probably more:

# Install gtar 
pkg_add gtar

# Tar up disk.raw
gtar -Sczvf openbsd.tar.gz disk.raw

This may take some time, so be patient. After the tar command completes, we will need to install the Google SDK, so that we have the tools required to upload this file.

Upload to Google Cloud

We will first need to install bash, as the Google package expects it. And we will need to make some symlinks, since Google has hard-coded in the path for bash as /bin/bash.

# Install required software
pkg_add bash curl python3

# link bash to /bin/bash for the scripts
ln -s /usr/local/bin/bash /bin/bash

# Link python3 to /usr/local/bin/python
ln -s /usr/local/bin/python /usr/local/bin/python3

# Download and run Google's install
curl https://sdk.cloud.google.com | /usr/local/bin/bash

When the installation asks if you would like to setup scripts for auto-complete and PATH, say yes. Once the installation is complete, we will need to initialize gcloud, and ensure that one more link is created for gsutil to work:

# Switch to bash
bash

# Initialize gcloud
gcloud init

# Create a link for gsutil
ls -s /home/$USER/google-cloud-sdk/bin/bootstrapping/gsutil.py /home/$USER/google-cloud-sdk/bin/gsutil

Next up, we need to transfer the new tar file up into the Google Cloud Storage, into a bucket attached to your project. If you do not already have a bucket, navigate to the Storage section, and click on Create Bucket, or simply search for the string "create bucket".

# Copy the tar.gz up into the cloud
gsutil cp openbsd.tar.gz gs://<my-bucket-name>/openbsd.tar.gz

Once the upload is complete, we will need to use gcloud to create an image to boot from the tar.gz. You will need to ensure that the file can be accessed by the user that gcloud was initialized with.

# Create the image
gcloud compute --project <my-project-name> images create <image name, eg. openbsd-65-amd64> --source-uri gs://<my-bucket-name>/openbsd.tar.gz

Once the image creation is complete, you should see a new image available to create VM's from. I will continue to revise this doc if new requirements arise that I come aware of. There are some google services running on standard GCE Linux images that I am working on porting over to OpenBSD. But due to some of their requirements, not all may work.

Note: On OpenBSD 6.5, you will need to specify an mtu of 1460 in the hostname file for the interface. However, that value must be removed in 6.6, as it causing issues with dhclient. I also have noticed that including IPv6 autoconf along with dhcp in the hostname.vio0 file can be problematic.