Go to https://djwong.dreamwidth.org/

Bye Bye!

2017-04-13T04:50:36Z

NetworkManager/resolveconf -> networkd/resolved

2016-05-27T19:38:01Z

Miscellaneous notes from converting Ubuntu 16.04 from NetworkManager/resolveconf to systemd-{networkd,resolved}.

In 14.04, system had the following /etc/network/interfaces:

auto br0
iface br0 inet dhcp
bridge_ports be0

and the following /etc/udev/rules/70-net-persistent.rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="XX:XX:XX:XX:XX:XX", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="be0"

But now that we've upgraded to 16.04, we have choices: stay with the old ifupdown, or migrate to NM or systemd-networkd. ifupdown still works just fine, though it may go away some day, doesn't have a convenient UI, etc. So let's try NetworkManager, which has a convenient CLI:

# nmcli con add type con-name br0 ifname br0
# nmcli con modify br0 bridge.stp no
# nmcli con modify br0 ipv6.addr-gen-mode stable-privacy
# nmcli con add type 
# nmcli con add type bridge-slave con-name enp1s0f0 ifname enp1s0f0 master br0
# nmcli con up enp1s0f0

Unfortunately, it kills dhclient when tearing down the bridge prior to going to sleep and never restarts it after resume. The end result is that when the IPv4 lease expires there's nothing to renew the DHCP and IPv4 stops working an hour or so after resume.

So maybe let's try networkd. First, we set up the NIC in /etc/systemd/network/enp1s0f0.network:

[Match]
Name=enp1s0f0

[Network]
Bridge=br0

Next, we establish a bridge device in /etc/systemd/network/br0.netdev:

[NetDev]
Name=br0
Kind=bridge
MACAddress=XX:XX:XX:XX:XX:XX

Note that we're cloning the NIC's MAC address to the bridge for convenience. Finally, we need to actually configure the bridge (/etc/systemd/network/br0.network):

[Match]
Name=br0

[Network]
DHCP=ipv4

[DHCP]
UseDomains=true

This sets up the bridge to request IPv4 information via DHCP, and to use the domain provided by the server as a default search domain. Next, we disable NM and resolvconf in favor of the systemd replacements:

# systemctl stop NetworkManager
# systemctl disable NetworkManager
# systemctl stop resolvconf
# systemctl disable resolvconf
# systemctl enable systemd-networkd
# systemctl start systemd-networkd
# systemctl enable systemd-resolved
# systemctl start systemd-resolved
# ln -sf ../run/systemd/resolve/resolv.conf /etc/resolv.conf

Reboot for changes to take effect.

Create a Bare git Repo to Cache a Remote Repo, w/ Tracking Branches and stgit

2015-08-26T21:34:44Z

Use case: Somewhere across the Internet is a git repo that we want to cache locally. We should be able to clone this cache-repo to do our development work, but the bare repo should only ever pull from its upstream. The flow looks like this: upstream repo <-- bare cache repo <-- work repo.

First, set up the bare repo:
$ git clone --bare git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git e2fsprogs.git

Next, tell git that we want to fetch all heads.

$ cd e2fsprogs.git
$ git config remote.origin.fetch 'refs/heads/*:refs/heads/*'

Check out the working copy.

$ cd ..
$ git clone -l -s  e2fsprogs.git e2fsprogs

Create a remote tracking branch.

$ cd e2fsprogs
$ git branch --track for-next origin/for-next
$ git checkout for-next

Initialize stgit.
$ stg init

Stupid Recipe for Creating LVM2 SSD Cache

2015-05-03T06:05:52Z

Silly recipe for creating a SSD cache with LVM without wasting space on the SSD device:


# pvcreate /dev/slowdisk
# pvcreate /dev/fastdisk
# vgcreate plum_disk_cache /dev/slowdisk /dev/fastdisk
# lvcreate -n origin plum_disk_cache -l $size_of_slowdisk /dev/slowdisk
# lvcreate -n cache plum_disk_cache -l $size_of_fastdisk /dev/fastdisk
# lvconvert --type cache --cachemode writeback --cachepool plum_disk_cache/cache plum_disk_cache/origin --chunksize 512k -v --poolmetadataspare n
# lvreduce plum_disk_cache/cache -l -6 (or however many it claims to need)
# lvconvert --type cache --cachemode writeback --cachepool plum_disk_cache/cache plum_disk_cache/origin --chunksize 512k -v --poolmetadataspare n

A somewhat clumsy alternative method is this:

(Create VG, allocate "origin" LV across slow disks, add fastdisk to VG...)


# vgs plum_disk_cache -o vg_free_count
 Free
67236
# lvcreate --type cache -n cache plum_disk_cache/origin /dev/fastdisk -l 67236 --cachemode writeback --chunksize 512k --poolmetadataspare n; done
# lvchange --cachepolicy smq plum_disk_cache/origin

(Replace 67236 with successive values until it succeeds, being careful to remove any LVs it erroneously creates while trying to set up the cache.)

Remapping Keyboard Keys with udev hwdb

2014-12-24T07:23:51Z

Here's how to remap globally keyboard keys with udev's hwdb database, since I couldn't find a single document that documents all the steps correctly (for Ubuntu 14.04).

Motivation: We want some key on some type of keyboard (say F15) to some other pre-defined function (play/pause song) without globally remapping F15 on all keyboards, because that will break all keyboards that actually have a play/pause button.

Overview: We need to find the keyboard scan code for the key, find out the name of the key that we want to turn it into, then compose a mapping (for /just/ that keyboard!) file. Finally, kick udev into recognizing our new mapping.

Retrieve the keyboard scan code. This can be done by firing up the evtest(1) command.

Select the device you want to monitor (we'll pick the MS Natural 4000).

Press the key (here, the zoom out slider). So far, the evtest output looks like this:
# evtest No device specified, trying to scan all of /dev/input/event* Available devices: /dev/input/event0: Power Button <snip> /dev/input/event11: Logitech USB Optical Mouse /dev/input/event12: Microsoft Natural® Ergonomic Keyboard 4000 /dev/input/event13: Microsoft Natural® Ergonomic Keyboard 4000 /dev/input/event14: HID 05ac:820a /dev/input/event15: HID 05ac:820b /dev/input/event16: Apple, Inc. IR Receiver /dev/input/event17: PWC snapshot button Select the device event number [0-17]: 13 Input driver version is 1.0.1 Input device ID: bus 0x3 vendor 0x45e product 0xdb version 0x111 Input device name: "Microsoft Natural® Ergonomic Keyboard 4000" Supported events: Event type 0 (EV_SYN) <snip> Testing ... (interrupt to exit) Event: time 1419404672.824579, type 4 (EV_MSC), code 4 (MSC_SCAN), value c022e Event: time 1419404672.824579, type 1 (EV_KEY), code 153 (KEY_F14), value 1 Event: time 1419404672.824579, -------------- SYN_REPORT ------------ Event: time 1419404672.928509, type 4 (EV_MSC), code 4 (MSC_SCAN), value c022e Event: time 1419404672.928509, type 1 (EV_KEY), code 153 (KEY_F14), value 0 Event: time 1419404672.928509, -------------- SYN_REPORT ------------

The keyboard scan code is at the end of the first line in the SYN_REPORT. For the key we want (the zoom out slider), the scan code is 0xC022E.

Now we need to construct a hardware ID for the keyboard. sysfs will tell us; remembering the number 13 from step 3, we can look it up:
# ls /sys/class/input/input13 lrwxrwxrwx 1 root root 0 Dec 23 22:34 /sys/class/input/input13 -> ../../devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1.3/1-1.3:1.1/0003:045E:00DB.0002/input/input13/

From this device, we see that the keyboard is USB, the vendor ID is 0x045E (Microsoft) and the product ID is 0x00DB (MS Natural 4000). This enables us to build a hwdb selector: keyboard:usb:v045Ep00DB*. NOTE: As of systemd 205(?) the selector format changed to evdev:input:b0003v045Ep00DB*.

Now look up the symbolic name of the key event that we want to fire when we press the zoom in key on the keyboard. That list can be found at http://hal.freedesktop.org/quirk/quirk-keymap-list.txt . The key we want is "Previous Song" and its symbolic name is "previoussong".

Construct the hwdb file /lib/udev/hwdb.d/99-ms-natural.hwdb (the .hwdb extension is critical!) with the following contents:
# Microsoft Natural Ergonomic Keyboard 4000 evdev:input:b0003v045Ep00DB* KEYBOARD_KEY_c022e=previoussong # KEY_ZOOMOUT

Regenerate the udev hwdb database: udevadm hwdb --update

Reboot the system. Occasionally, running udevadm trigger and replugging the device after a few seconds will work.

Verify correct operation by re-running evtest:
Event: time 1419405734.888026, type 4 (EV_MSC), code 4 (MSC_SCAN), value c022e Event: time 1419405734.888026, type 1 (EV_KEY), code 165 (KEY_PREVIOUSSONG), value 1 Event: time 1419405734.888026, -------------- SYN_REPORT ------------ Event: time 1419405735.008025, type 4 (EV_MSC), code 4 (MSC_SCAN), value c022e Event: time 1419405735.008025, type 1 (EV_KEY), code 165 (KEY_PREVIOUSSONG), value 0 Event: time 1419405735.008025, -------------- SYN_REPORT ------------

That should do it!

Encrypted root and /boot on AppleEFI on Ubuntu 14

2014-08-23T05:55:39Z

Hi all,

Here's how you install Ubuntu 14 with a LUKS-encrypted / and /boot partition when using Apple's "quirky and fun" EFI:

Insert a USB stick, and boot the installer into "Try Ubuntu" mode (you need to be able to start an xterm before rebooting).

Install Ubuntu. On the hard disk, you'll want to create a 64M empty partition, a 256M /boot (unencrypted), and a LUKS container with LVM for /. Format the USB stick as an EFI system partition and let the installer put files there.

BEFORE rebooting the installer, change the HDD's empty 64M partition's type code (call it /dev/sda1) to Apple Boot (gdisk type code AB00) and format it HFS+.

Mount the new / to /mnt, and the new /boot to /mnt/boot. (The installer might just leave these two filesystems mounted at the end of the installation; I don't remember.

Mount the new Apple Boot partition (on the HDD). We'll say you mounted it to /mnt/boot/efi.
touch /mnt/boot/efi/mach_kernel

mkdir -p /mnt/boot/efi/EFI/ubuntu/ /mnt/boot/efi/System/Library/CoreServices/

cd /mnt/boot/efi/EFI/ubuntu/

ln -s ../../System

touch mach_kernel

chroot /mnt/ grub-install -v (in theory grub2 knows how to generate Mac-compatible files, though it puts them in the wrong location)

Reboot to Mac OSX.

diskutil mount /dev/disk0s1

Open Preferences, Startup Disk, select the Ubuntu install.

Reboot the system to prove that Linux boots correctly.

apt-get purge grub-efi-amd64-signed (the signed grub image does not have LUKS/cryptodisk support)

Copy the contents of /boot somewhere, and note the device (say /dev/sda2 for this example).

cryptsetup luksFormat /dev/sda2 -c aes-xts-plain64
DO NOT install LVM here; grub2 refuses to allow FS writes to LVM volumes, which is needed for recordfail.

Create an entry in /etc/crypttab for the new LUKS container. We'll assume you called the device-mapper node "boot_crypt".

/etc/init.d/cryptdisks-early start (prove that crypttab works)

mkfs.ext4 /dev/mapper/boot_crypt -L boot (or whatever filesystem you want here)

Edit /etc/fstab to point /boot to /dev/mapper/boot_crypt.

Copy the saved files from step 17 into the new /boot.

echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub

grub-install -v (verify that grub-mkimage is called with luks/cryptodisk modules included)

update-grub

Reboot to Mac OSX.

diskutil mount /dev/disk0s1

Open Preferences, Startup Disk, select the Ubuntu install. Again.

Reboot. Grub should now prompt to unlock the disk before showing the boot menu. For extra credit, set up LUKS keys for automount, since the only thing unencrypted on your HDD is grub, and (in theory) you could stash the rootfs LUKS keys in the initramfs. Macs don't support SecureBoot, at least not the 2012 models.

N.B. The Apple firmware will try to boot whatever the HFS+ startup file points to; using efibootmgr does no good here. If you rewrite the boot.efi file the safe way (write boot.efi.new, mv boot.efi.new boot.efi) you'll have to set the startup disk in OSX again, because the "startup file" is really an extent map in the HFS+ superblock. Quite possibly OSX simply copies the contents of boot.efi into the blocks pointed to by the startup file when a Startup Disk is set.

Encrypted root and /boot on regular UEFI on Ubuntu 14

2014-08-23T05:37:14Z

Hi all,

Here's how you install Ubuntu 14 with a LUKS-encrypted / and /boot partition when using regular UEFI (i.e. not Apple's insane firmware):

Install Ubuntu. You'll want to create a 256M /boot (unencrypted), a LUKS container with LVM for /.

Disable SecureBoot. Hopefully I will figure out how to fix this later.

Boot the system to prove that it boots correctly.

apt-get purge grub-efi-amd64-signed (the signed grub image does not have LUKS/cryptodisk support)

Copy the contents of /boot somewhere, and note the device (say /dev/sda2 for this example).

cryptsetup luksFormat /dev/sda2 -c aes-xts-plain64
DO NOT install LVM here; grub2 refuses to allow FS writes to LVM volumes, which is needed for recordfail.

Create an entry in /etc/crypttab for the new LUKS container. We'll assume you called the device-mapper node "boot_crypt".

/etc/init.d/cryptdisks-early start (prove that crypttab works)

mkfs.ext4 /dev/mapper/boot_crypt -L boot (or whatever filesystem you want here)

Edit /etc/fstab to point /boot to /dev/mapper/boot_crypt.

Copy the saved files from step 5 into the new /boot.

echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub

grub-install -v (verify that grub-mkimage is called with luks/cryptodisk modules included)

update-grub

Reboot. Grub should now prompt to unlock the disk before showing the boot menu. For extra credit, set up LUKS keys for automount, since the only thing unencrypted on your HDD is grub, and (in theory) you could stash the rootfs LUKS keys in the initramfs. If I figure out how to make this work with SecureBoot I'll update this post.

Encrypted root and /boot on legacy BIOS on Ubuntu 14

2014-08-23T05:24:33Z

Hi all,

Here's how you install Ubuntu 14 with a LUKS-encrypted / and /boot partition when using legacy BIOS:

Install Ubuntu. You'll want to create a 256M /boot (unencrypted), a LUKS container with LVM for /.

Boot the system to prove that it boots correctly.

Copy the contents of /boot somewhere, and note the device (say /dev/sda1 for this example).

cryptsetup luksFormat /dev/sda1 -c aes-xts-plain64
DO NOT install LVM here; grub2 refuses to allow FS writes to LVM volumes, which is needed for recordfail.

Create an entry in /etc/crypttab for the new LUKS container. We'll assume you called the device-mapper node "boot_crypt".

/etc/init.d/cryptdisks-early start (prove that crypttab works)

mkfs.ext4 /dev/mapper/boot_crypt -L boot (or whatever filesystem you want here)

Edit /etc/fstab to point /boot to /dev/mapper/boot_crypt.

Copy the saved files from step 3 into the new /boot.

echo 'GRUB_ENABLE_CRYPTODISK=y' >> /etc/default/grub

grub-install -v (verify that grub-mkimage is called with luks/cryptodisk modules included)

update-grub

Reboot. Grub should now prompt to unlock the disk before showing the boot menu. For extra credit, set up LUKS keys for automount, since the only thing unencrypted on your HDD is grub, so in theory you could stash the key for the rootfs LUKS keys in the initramfs.

Whacking AppleEFI into fast-booting Ubuntu 14.04 via grub2

2014-04-20T04:10:03Z

Same instructions as http://djwong.livejournal.com/66501.html, but with a few notes:

The (Kubuntu) installer isn't going to want to use your "Apple Boot" partition as a EFI boot partition. Stick in a GPT-formatted USB stick with a properly typecode'd and FAT-formatted FS, and have it use that as the EFI system partition. When the installer finishes, copy the files it put in /EFI/ubuntu onto the HFS+ partition. Make sure the grub.cfg file does NOT have execute permissions, because for some reason grub2 won't read +x files. Copy /EFI/ubuntu/grubx64.efi and grub.cfg to /System/Library/CoreServices/boot.efi, then do the whole OSX blessing rigamarole.

Basically you're letting the regular UEFI grub2 installer stuff files onto the system, then copying them the special OSX locations that AppleEFI knows about, and blessing it as if it were OSX.

Whacking AppleEFI into fast-booting Ubuntu 12.04 via grub2

2013-09-29T06:05:56Z

Let's face it folks, Apple's firmware isn't EFI1 and it isn't UEFI2 either. It's a weird "hybrid", i.e. mess. I posted quite some time ago about how to make AAPLEFI find grub2-efi, but it's slow because the firmware scans all boot devices looking for bootx64.efi files, and it's not easily recoverable if your machine forgets one of the pieces it needs. What you really want is fastboot mode where the EFIVARS retains the location of the bootloader. This is /fairly/ simple if you're already booted with BIOS/CSM.

(Hopefully you left enough space to create a EFI boot partition somewhere. It can be quite small.)

1. Format disk (say /dev/sdX) with GPT label. Use gdisk.
2. Create smallish partition (64MB) with type AB00 and name 'Recovery HD'.
3. Format partition as HFS+.
4. Run blkid -o udev /dev/sdXY to ensure that you got a HFS+ with a UUID. The partition must have a UUID for the AAPLEFI to recognize it.
4. mkdir /efiboot and make an fstab entry to automount the small partition.
5. Mount partition.
6. mkdir -p /efiboot/EFI/grub/ /efiboot/EFI/boot/ /efiboot/System/Library/CoreServices/
7. apt-get install grub-efi-amd64
8. grub-install /dev/sdX
9. ln /efiboot/EFI/grub/grub.efi /efiboot/EFI/boot/bootx64.efi
10. update-grub
11. touch /efiboot/mach_kernel
12. ln /efiboot/EFI/grub/grub.efi /efiboot/System/Library/CoreServices/boot.efi
13. Dump the following into /efiboot/System/Library/CoreServices/SystemVersion.plist:


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>ProductBuildVersion</key>
	<string></string>
	<key>ProductCopyright</key>
	<string></string>
	<key>ProductName</key>
	<string>Ubuntu</string>
	<key>ProductUserVisibleVersion</key>
	<string>12.04</string>
	<key>ProductVersion</key>
	<string>12.04</string>
</dict>
</plist>

14. Reboot to MacOS.
15. diskutil mount /dev/diskXsY
16. bless --folder=/Volumes/some_volume --file=/Volumes/some_volume/efi/grub/grub.efi --setBoot. Or you can use the Startup Disk Preference panel to select the disk.
17. Reboot to Linux. Hopefully it comes up super-fast; if not, it'll eventually find the bootx64.efi.
18. When you're satisfied, go back to gdisk and tell it to recreate protective MBR. This erases your legacy grub2 installation.

Whacking UEFI into booting Ubuntu 12.04 via grub2

2013-09-29T05:56:52Z

Want to use UEFI 2.x to load grub2-efi and then the kernel? This is /fairly/ simple if you're already booted with BIOS/CSM.

(Hopefully you left enough space to create a EFI boot partition somewhere. It can be quite small.)

1. Format disk (say /dev/sdX) with GPT label. Use gdisk.
2. Create smallish partition (64MB) with type EF00 and name 'EFI System Partition'.
3. Format partition as FAT32.
4. mkdir /efiboot and make an fstab entry to automount the small partition.
5. Mount partition.
6. mkdir -p /efiboot/EFI/grub/ /efiboot/EFI/boot/
7. apt-get install grub-efi-amd64
8. grub-install /dev/sdX
9. cp /efiboot/EFI/grub/grub.efi /efiboot/EFI/boot/bootx64.efi
10. update-grub
11. Reboot. Enter F1 setup.
12. Create boot option for fsX:\EFI\grub\grub.efi named Ubuntu, then exit setup and boot it. If your UEFI won't let you create boot options, then system will find bootx64.efi from step 9 and use that instead.
13. When you're satisfied, go back to gdisk and tell it to recreate protective MBR. This erases your legacy grub2 installation.

How to Set Up kvm MTR

2012-10-15T18:49:46Z

You are going to create a chroot on a VM host, then set up a PXE environment to net-boot the system. Let %CHROOT% be the chroot location on the host; %VM_NET_IP% be the IP address of the local VM network; %VM_NET_CIDR% be the CIDR of the local VM network; and %VM_NET_BRIDGE% be the name of the bridge that ties together the local VM network.

1. Use debootstrap to set up a chroot at %CHROOT%.
2. Add full set of Ubuntu repos to %CHROOT%.
3. Use djwong repo install.sh to install djwong package sources.
4. chroot %CHROOT% su - root
5. apt-get install djwong-mtr djwong-kernel-support
6. Install some sort of kernel.
7. Jump out to host.
8. virsh net-destroy default
9. virsh net-edit default
10. Add "<tftp root="%CHROOT%/boot/" />" as a child of the <ip> element.
11. Add "<bootp file="pxelinux.0" />" as a child of the <dhcp> element.
12. virsh net-start default
13. apt-get install nfs-kernel-server
14. Add "%CHROOT%/ %VM_NET_CIDR%(ro,sync,no_root_squash,no_subtree_check)" to /etc/exports. Do not be tempted to add "-H %VM_NET_IP%" to /etc/default/nfs-kernel-server; this will result in NFS3 being disabled due to braindead nfs init script!
15. /etc/init.d/nfs-kernel-server restart
16. ufw enable
17. ufw allow ssh
18. ufw allow cups
19. ufw allow in on %VM_NET_BRIDGE%
20. Create a VM with e1000 (not virtio-net) NIC, set to boot PXE.
21. Start VM.

How to Trick a Mac into Booting EFI Grub

2012-06-23T05:19:58Z

Make sure that /dev/disk0s1 is FAT formatted, has EFI grub installed in EFI/grub/, and its GPT partition type is EF00 (EFI System Partition).


# mount -t msdos /dev/disk0s1 /mnt
# bless --mount /mnt --setBoot --file /mnt/EFI/grub/grub.efi
# umount /mnt

How to Create a Signed Personal Package Archive (PPA) with Signed Packages

2012-04-03T00:15:17Z

It's tinfoil hat time! Let's say you run Debian or Ubuntu, maintain your own package archive, and want to make it harder for someone to tamper with that archive and trick you into installing malicious packages. If you want to sign your Debian packages and apt archive, here's a quick guide to how you set this up. It's important to have a signed archive, because the archive contains package metadata (md5sums of the .deb files, etc) that are used by the receiving apt program to sanity-check the incoming packages. Signing the metadata (if the private keys are secure) prevents other people from modifying the archive contents; signing the packages themselves ensures that the .deb files haven't been tampered with. In theory it's only necessary to provide a signed archive.

First, use gpg to generate two key pairs -- one keypair on the package builder machine, and another pair on machine where the package archive will live. Let's call the pairs pkg_priv, pkg_pub, ppa_priv, and ppa_pub.
Let pkg_keyid=last 4 hunks of gpg --fingerprint -k ${pkg_priv}

On the package builder, do this:


apt-get install debsigs debsig-verify
gpg --armor --export ${pkg_priv} > ${pkg_pub_file}

For each package built, do:


debsigs --sign=origin -v ${debfile} -k ${pkg_priv}

On the package archive box, do this:


apt-get install reprepro
mkdir ${repodir}
add "SignWith: ${ppa_priv}" to ${repodir}/conf/distributions
gpg --armor --export ${ppa_priv} > ${ppa_pub_file}

To inject a package into the archive, do this:


reprepro -S main includedeb ${distro} ${debfile}

On the client, do this to add the package archive's gpg public key to apt:


apt-key add ${ppa_pub_file}

On the client, do this to make dpkg verify the signatures of individual package files:


mkdir -p /usr/share/debsig/keyrings/${pkg_keyid}/ /etc/debsig/policies/${pkg_keyid}
gpg --no-default-keyring --keyring /usr/share/debsig/keyrings/${keyid}/debsig.gpg --import ${pkg_pub_file}
cat > /etc/debsig/policies/${pkg_keyid}/djwong.pol << ENDL
<?xml version="1.0"?>
<!DOCTYPE Policy SYSTEM "http://www.debian.org/debsig/1.0/policy.dtd">
<Policy xmlns="http://www.debian.org/debsig/1.0/">

<Origin Name="djwong" id="${pkg_keyid}" Description="Packages by djwong"/>

<Selection>
<Required Type="origin" File="debsig.gpg" id="${pkg_keyid}"/>
</Selection>

<Verification MinOptional="0">
<Required Type="origin" File="debsig.gpg" id="${pkg_keyid}"/>
</Verification>

</Policy>
ENDL
mv /usr/bin/debsig-verify /usr/bin/debsig-verify.real    # really this should be dpkg-divert
cat > /usr/bin/debsig-verify << ENDL
#!/bin/sh

/usr/bin/debsig-verify.real $*
RESULT=$?
if [ $RESULT -eq 10 ]; then
	echo 'Passing unsigned package.'
	exit 0
fi
exit $RESULT
ENDL
remove no-debsig from /etc/dpkg/dpkg.cfg

This sets up a policy such that packages signed with the package builder's private key must be verified before dpkg will install them. The strange activity in /usr/bin "patches" the debsig-verify program so that unsigned packages (none of the Ubuntu/Debian package files themselves are signed) don't fail.

If you're particularly paranoid, the "package archive" steps should be done on a private machine and the contents later rsync'd up to the real archive host, so that neither private key ever has to be on a well-known system. If you don't perform this step, attackers can modify the archive, though they won't be able to tamper with the individual .deb files.

N.B. debsign signatures are NOT compatible with dpkg-sig signatures!

How to Build a (powerpc) Cross-Compiler in Ubuntu 12.04

2012-03-27T02:20:04Z

This is a rough script of what I did to build a powerpc cross compiler in Ubuntu 12.04. In theory it should work for just about any other machine type, though I wouldn't bet money on that. (FYI: They have armel packages in the repos for some reason...)

$ mkdir packages

# binutils
$ apt-get source binutils
$ export DEB_PACKAGE_BUILD_OPTS=parallel=8
$ export TARGET=powerpc-linux-gnu
$ cd binutils*
$ dpkg-buildpackage -b -uc -us -rfakeroot 2>&1 | tee ../binutils.build
$ cd ..
$ mv binutils-powerpc-linux-gnu_*.deb packages
$ dpkg -i packages/binutils-powerpc-linux-gnu_*.deb

# libc6/libgcc cross insanity
$ cd fakepackages
* Download the following *POWERPC* packages:libc6-ppc64 libc6-dev-ppc64 linux-libc-dev libc6 libc6-dev libgcc1
$ apt-get install dpkg-cross
$ dpkg-cross -M -a powerpc -b *.deb -X tzdata -X libc-dev-bin -X libc-bin -X multiarch-support -X gcc-4.6-base
$ dpkg -i libgcc1-powerpc-cross_*.deb libc6-dev-powerpc-cross_*.deb libc6-dev-ppc64-powerpc-cross_*.deb libc6-powerpc-cross_*.deb linux-libc-dev-powerpc-cross_*.deb libc6-ppc64-powerpc-cross_*.deb
* Fix up the paths in /usr/powerpc-linux-gnu/lib/libpthread.so /usr/powerpc-linux-gnu/lib/libc.so /usr/powerpc-linux-gnu/lib64/libpthread.so /usr/powerpc-linux-gnu/lib64/libc.so
$ cp libc6-dev-powerpc-cross_*.deb libc6-dev-ppc64-powerpc-cross_*.deb libc6-powerpc-cross_*.deb linux-libc-dev-powerpc-cross_*.deb libc6-ppc64-powerpc-cross_*.deb ../packages/
$ cd ..

# gcc
$ apt-get source gcc-4.6
$ #export DEB_CROSS=yes				# Not needed, picked up automatically (I guess)
$ #export DEB_CROSS_NO_BIARCH=yes			# NO!  this disables ppc64 compiler!
$ export GCC_TARGET=powerpc
$ export LD_LIBRARY_PATH=/lib/x86_64-linux-gnu	# Fixes dh script error at the end of build
$ cd gcc-4.6*
$ fakeroot debian/rules control
$ dpkg-buildpackage -b -uc -us -rfakeroot 2>&1 | tee ../gcc.build
$ cd ..
$ mv *.deb packages/
$ dpkg -i packages/cpp-4.6-powerpc-linux-gnu_*.deb packages/gcc-4.6-multilib-powerpc-linux-gnu_*.deb packages/gcc-4.6-powerpc-linux-gnu_*.deb packages/gcc-4.6-powerpc-linux-gnu-base_*.deb packages/lib64gcc1-powerpc-cross_*.deb packages/lib64gomp1-powerpc-cross_*.deb packages/libgcc1-powerpc-cross_*.deb packages/libgomp1-powerpc-cross_*.deb

# eglibc (you can skip this part)
$ apt-get source eglibc
$ cd eglibc-*
$ dpkg-buildpackage -b -uc -us -rfakeroot -B -apowerpc 2>&1 | tee ../eglibc.build
$ cd ..
$ mv *.deb fakepackages/
* FIXME: Somehow produce libc6-ppc64 packages?
* Rerun the libc6/libgcc cross insanity step above.

# Now you should have cross compiler packages in packages/

Flashing Disk Firmware to SBC (SCSI/SAS/FC) Disks

2012-03-06T19:59:03Z

# sg_write_buffer -vvvvv -m 5 -I FIRMWARE_FILE /DISK/DEVICE

djwong @ 2012-02-02T23:56:00

2012-02-03T07:56:43Z

Meme time: Leave me a one-word comment about your day that starts with the third letter of your LJ username.

Splitting Your Sound Outputs into Multiple Zones and Connecting Them to MPD

2012-01-09T07:17:17Z

One of my basement computers has one of these Intel High-Definition Audio sound cards. There are 6 analog mini jacks on the back, 5 of which are outputs and 1 which is an input. The outputs are: stereo line out, stereo rear out, stereo side out, center, and subwoofer (LFE). Given that I have two ultra-quiet speaker zones in my house (rooms where I want to be able to listen to computer music without the sound pollution of a computer), wouldn't it be neat to route two *different* sound sources to the two stereophonic outputs that are connected to the two zones?

My living room and my master bedroom each have a set of stereo speakers. I've snaked audio cables throughout the house so that both sets can be driven by a computer that lives in the basement. The living room is connected to stereo line out, and the bedroom is connected to the stereo rear out. It is my goal for this computer to have Music Player Daemon (mpd) installed, so I can control the sound outputs remotely (yes, Android/iOS controllers exist).

Hardware Support

The first step is to figure out what hardware configurations the hardware supports. The board has a RealTek ALC889A sound card, which the Linux snd-hda-intel driver can talk to. We can use ALSA utilities for this:

# aplay -L
null
    Discard all samples (playback) or generate zero samples (capture)
default:CARD=Intel
    HDA Intel, ALC889A Analog
    Default Audio Device
front:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    Front speakers
surround40:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    4.0 Surround output to Front and Rear speakers
surround41:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=Intel,DEV=0
    HDA Intel, ALC889A Analog
    7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=Intel,DEV=0
    HDA Intel, ALC889A Digital
    IEC958 (S/PDIF) Digital Audio Output

Okay, so this looks like a fairly boring piece of audio hardware. By default it'll drive just stereo line out, but the card can also handle various surround modes ("surround40" ... "surround71"). The "iec958" profile is for the digital audio outputs on the board, which I do not use. Unfortunately, these profiles are not quite what we want, because the software designers assumed that there is one (possibly surround) sound stream (referred to as a "sound input" from ALSA's perspective) coming in from the software mixer, and all channels of this stream should be mapped to one of the output channels. What we want to build is a system with multiple sound "inputs", and we want each "input" to be mapped to a specific output channel without intermixing. For two stereo zones, the 4.0 surround profile is useful, because it has four output channels (front left, front right, rear left, rear right). We can send one "input" (stream) to the front pair and a second "input" stream to the rear pair.

To test the 4.0 surround, plug stereo headphones (or speakers, or whatever) into the line out (green) and rear out (black) ports on the machine. Then run this:


# speaker-test -c4 -l1 -twav -D surround40

speaker-test 1.0.22

Playback device is surround40
Stream parameters are 48000Hz, S16_LE, 4 channels
WAV file(s)
Home directory /storage/home/djwong not ours.
Rate set to 48000Hz (requested 48000Hz)
Buffer size range from 32 to 262144
Period size range from 16 to 131072
Using max buffer size 262144
Periods = 4
was set period_size = 65536
was set buffer_size = 262144
 0 - Front Left
 1 - Front Right
 3 - Rear Right
 2 - Rear Left
Time per period = 9.568956

This program will play a test message through all four channels, so you can verify that this profile indeed has four separate channels.

Software Support

Okay, so it's cool that all four channels work. But, how do we convince programs to play to the "correct" set of speakers? We could configure each program to send its sound to the correct channel (front or rear)... but that's tricky. Do you want to have to communicate to all programs that they should send a stereo signal only to the front? Or only to the rear? That's annoying. Instead, let's use a software mapper (PulseAudio) to create two virtual stereo sound devices (PA sinks) which will be mapped to the corresponding audio output channels (on the -surround40 ALSA input). Then we just tell the program which PA sink it should play to.

The PulseAudio sound system can set up this virtualization. First, install PulseAudio and then ask it for the list of default sound sinks:

# pacmd list-sinks
Welcome to PulseAudio! Use "help" for usage information.
>>> 1 sink(s) available.
  * index: 0
	name: <alsa_output.pci-0000_00_1b.0.analog-stereo>
	driver: <module-alsa-card.c>
	flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
	state: RUNNING
	suspend cause: 
	priority: 9959
<snip>

Notice how the name says "-stereo", not "-surround40" or something like it? PA has picked the default ALSA profille, and as aplay -L just told us, the default ALSA profile for this card is stereo out, not 4.0 surround. Let's change that by adding stuff to /etc/pulse/daemon.conf:


default-sample-channels = 4
log-level = error

Now kill all the pulseaudio daemons and re-query the PA sinks:

>>> 1 sink(s) available.
  * index: 0
	name: <alsa_output.pci-0000_00_1b.0.analog-surround-40>
	driver: <module-alsa-card.c>
	flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
	state: RUNNING
	suspend cause: 
	priority: 9959
	volume: 0:  43% 1:  43% 2:  43% 3:  43%
	        0: -22.00 dB 1: -22.00 dB 2: -22.00 dB 3: -22.00 dB
	        balance 0.00
	base volume: 100%
	             0.00 dB
	volume steps: 65537
	muted: no

Remember that name field, we're going to need it to set up the virtual sound sinks. The sound sink configuration goes in /etc/pulse/default.pa:


load-module module-remap-sink sink_name=downstairs remix=no master=alsa_output.pci-0000_00_1b.0.analog-surround-40 channels=2 master_channel_map=front-left,front-right channel_map=front-left,front-right
load-module module-remap-sink sink_name=upstairs remix=no master=alsa_output.pci-0000_00_1b.0.analog-surround-40 channels=2 master_channel_map=rear-left,rear-right   channel_map=front-left,front-right

What does this do? The first line loads a module named "module-remap-sink". This module is like device-mapper for sound devices. It creates a new sink named "downstairs". The "remix=no" clause tells PA not to remix the inputs to all output channels. The "master=" clause links the new sink to a lower-level sink, which in this case is a piece of hardware named "alsa_output.pci-0000_00_1b.0.analog-surround-40". The name should be familiar. Moving along, "channels=2" identifies the new sink as being a stereo output. "channel_map" is the list of channels that this sink accepts; the choice of "front-left,front-right" means that the sink will appear to be a basic stereo line output sink. Finally, the "master_channel_map" clause maps (for "downstairs") the sink's front left and right channels to the hardware's front and right channels. Notice that in the "upstairs" sink configuration, the sink's front left and right channels are mapped to the hardware's REAR left and right channels. Hence, stereo sounds sent to "upstairs" line out will be remapped to the rear output jacks on the computer, which is exactly what we want. You can run speaker-test -Dupstairs to verify this.

Here's what PA lists for sinks now:

# pacmd list-sinks
Welcome to PulseAudio! Use "help" for usage information.
>>> 3 sink(s) available.
  * index: 0
	name: <alsa_output.pci-0000_00_1b.0.analog-surround-40>
	driver: <module-alsa-card.c>
	flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
	state: RUNNING
	suspend cause: 
	priority: 9959
	volume: 0:  43% 1:  43% 2:  43% 3:  43%
	        0: -22.00 dB 1: -22.00 dB 2: -22.00 dB 3: -22.00 dB
	        balance 0.00
	base volume: 100%
	             0.00 dB
	volume steps: 65537
	muted: no
	current latency: 66.94 ms
	max request: 36 KiB
	max rewind: 1378 KiB
	monitor source: 0
	sample spec: s16le 4ch 44100Hz
	channel map: front-left,front-right,rear-left,rear-right
	             Surround 4.0
	used by: 1
	linked by: 2
	configured latency: 105.00 ms; range is 0.50 .. 4000.36 ms
	card: 0 <alsa_card.pci-0000_00_1b.0>
	module: 4
	properties:
		alsa.resolution_bits = "16"
		device.api = "alsa"
		device.class = "sound"
		alsa.class = "generic"
		alsa.subclass = "generic-mix"
		alsa.name = "ALC889A Analog"
		alsa.id = "ALC889A Analog"
		alsa.subdevice = "0"
		alsa.subdevice_name = "subdevice #0"
		alsa.device = "0"
		alsa.card = "0"
		alsa.card_name = "HDA Intel"
		alsa.long_card_name = "HDA Intel at 0xfc000000 irq 44"
		alsa.driver_name = "snd_hda_intel"
		device.bus_path = "pci-0000:00:1b.0"
		sysfs.path = "/devices/pci0000:00/0000:00:1b.0/sound/card0"
		device.bus = "pci"
		device.vendor.id = "8086"
		device.vendor.name = "Intel Corporation"
		device.product.id = "3a3e"
		device.product.name = "82801JI (ICH10 Family) HD Audio Controller"
		device.form_factor = "internal"
		device.string = "surround40:0"
		device.buffering.buffer_size = "1411328"
		device.buffering.fragment_size = "705664"
		device.access_mode = "mmap+timer"
		device.profile.name = "analog-surround-40"
		device.profile.description = "Analog Surround 4.0"
		device.description = "Internal Audio Analog Surround 4.0"
		alsa.mixer_name = "Realtek ALC889A"
		alsa.components = "HDA:10ec0885,1458a102,00100101"
		module-udev-detect.discovered = "1"
		device.icon_name = "audio-card-pci"
    index: 1
	name: <downstairs>
	driver: <module-remap-sink.c>
	flags: DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
	state: SUSPENDED
	suspend cause: IDLE 
	priority: 1000
	volume: 0: 100% 1: 100%
	        0: 0.00 dB 1: 0.00 dB
	        balance 0.00
	base volume: 100%
	             0.00 dB
	volume steps: 65537
	muted: no
	current latency: 0.00 ms
	max request: 18 KiB
	max rewind: 689 KiB
	monitor source: 2
	sample spec: s16le 2ch 44100Hz
	channel map: front-left,front-right
	             Stereo
	used by: 0
	linked by: 0
	configured latency: 0.00 ms; range is 0.50 .. 4000.36 ms
	module: 14
	properties:
		device.master_device = "alsa_output.pci-0000_00_1b.0.analog-surround-40"
		device.class = "filter"
		device.description = "Remapped Internal Audio Analog Surround 4.0"
		device.icon_name = "audio-card"
    index: 2
	name: <upstairs>
	driver: <module-remap-sink.c>
	flags: DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY
	state: RUNNING
	suspend cause: 
	priority: 1000
	volume: 0: 100% 1: 100%
	        0: 0.00 dB 1: 0.00 dB
	        balance 0.00
	base volume: 100%
	             0.00 dB
	volume steps: 65537
	muted: no
	current latency: 66.65 ms
	max request: 18 KiB
	max rewind: 689 KiB
	monitor source: 3
	sample spec: s16le 2ch 44100Hz
	channel map: front-left,front-right
	             Stereo
	used by: 1
	linked by: 1
	configured latency: 105.00 ms; range is 0.50 .. 4000.36 ms
	module: 15
	properties:
		device.master_device = "alsa_output.pci-0000_00_1b.0.analog-surround-40"
		device.class = "filter"
		device.description = "Remapped Internal Audio Analog Surround 4.0"
		device.icon_name = "audio-card"

Looks good right now...

MPD

Put this into the MPD configuration file:

audio_output {
        type            "pulse"
        name            "pulseaudio confusion"
        sink            "upstairs"
}

This tells this MPD to send all sound to the "upstairs" PulseAudio sink that we created above. You probably want to create a second MPD config file (and run a second instance) to talk to the "downstairs" sink. Then you can use the MPD clients to control upstairs and downstairs, and to play different streams on each. Problem solved!

Enabling T10 Protection Information on SCSI Disks

2011-07-14T00:18:07Z

I intend to ramble about the T10 data integrity enhancement standards that are beginning to appear in new hardware.

Introduction

Some newer SBC-3 (ok, SCSI) compliant hard drives support what is known as T10 Protection Information (PI) or Data Integrity Fields (DIF). This field adds 8 bytes of integrity information to every sector written to the disk. Four bytes are used to record the lower 32-bits of the LBA (to guard against writes going to the wrong sector address); two bytes are used to record the CRC16 of the data block; and the final two bytes are an "application tag" that user programs can record to the disk. As of July 2011 there are a scant few drives that support this feature; there are even fewer HBAs (qla8xxx, lpfc, mpt2sas) that know how to deal with the new SBC commands to use this feature.

There are two parts to the T10 PI scheme--DIF and DIX. DIF requires that the disk media have an extra 8 bytes per sector to store integrity data, and extends the old READ/WRITE commands to accept the (lba, crc, apptag) tuple. The DIX spec specifies an interface for the OS software to pass (lba, crc, apptag) tuples to the HBA for relaying onto the disk. If an HBA supports DIF and not DIX, then it's understood that the HBA generates the (lba, crc, apptag) tuple during a write and verifies them during a read; if it supports both DIF and DIX, then the OS is responsible for providing the tuple during write and verifying it during read.

To use this feature, we need some software and hardware support to be in place. First, we need a disk that supports the T10 DIF extensions. Second, we need a HBA that supports DIF and (optionally) DIF. If the HBA supports DIX, then third, we need a software stack that knows how to pipe the DIF data to upper layers of software. Linux 3.0 knows how to do this internally, though interfaces to expose the protection information to application programs are still being debated.

How Do I Check for Hardware Support?

Load the disk and then read sysfs files:

# sysfs_dump /sys/class/scsi_host/host*/prot*
/sys/class/scsi_host/host12/prot_capabilities : 0
/sys/class/scsi_host/host12/prot_guard_type : 0
/sys/class/scsi_host/host14/prot_capabilities : 7
/sys/class/scsi_host/host14/prot_guard_type : 1
/sys/class/scsi_host/host15/prot_capabilities : 17
/sys/class/scsi_host/host15/prot_guard_type : 2

# sysfs_dump /sys/block/*/integrity/*
/sys/block/sdg/integrity/format : T10-DIF-TYPE1-IP
/sys/block/sdg/integrity/read_verify : 1
/sys/block/sdg/integrity/tag_size : 2
/sys/block/sdg/integrity/write_generate : 1

prot_capabilities is a bitset indicating the HBA's level of DIF/DIX support:
1<<0: DIF level 1
1<<1: DIF level 2
1<<2: DIF level 3
1<<3: DIX level 0 (no PI interface)
1<<4: DIX level 1
1<<5: DIX level 2
1<<6: DIX level 3

prot_guard_type indicates the CRC that the controller supports:
1<<0: CRC16
1<<1: IP checksum

read_verify indicates that the system supports verifying the PI during reads.
tag_size indicates that the application tag size is 2 bytes.
write_generate indicates that the system supports sending PI data along with data blocks during writes.

If your controller doesn't have the prot_* sysfs files, it doesn't know anything about DIF or DIX. If prot_capabilities is less than 7, then the controller doesn't know about DIX. If the disk doesn't have the integrity/ knobs, then it is not advertising DIF support. This could be due to lack of support, or the disk may need to be formatted with PI enabled.

How Do I Enable DIF on a Disk?

Run these command to enable Type 1 DIF:
# sg_format --pfu=0 --fmtpinfo=2 --format -vvvvv -w /dev/sdX # echo 1 > /sys/block/sdX/device/delete; echo "- - -" | tee /sys/class/scsi_host/*/scan # dd if=/dev/zero of=/dev/sdX bs=65536k # echo 1 > /sys/block/sdX/device/delete; echo "- - -" | tee /sys/class/scsi_host/*/scan
The precise values for --pfu and --fmtpinfo can be found in the latest SBC3 draft. A short summary follows:

FMTPINFO PFU Description
0        0   Type 0 protection, i.e. none.
2        0   Type 1 protection: 4 bytes LBA, 2 bytes application tag, 2 bytes CRC.
3        0   Type 2 protection: 4 bytes LBA, 2 bytes application tag, 2 bytes CRC,
             EXCEPT for the first logical block of each command; that block gets
             6 bytes LBA and 2 bytes CRC.
3        1   Type 3 protection: 6 bytes application tag, 2 bytes CRC.

(Not all drives support all types of PI.)

After all that, your drive should be ready to go with T10 PI enabled. In theory, the stable dirty page write feature I introduced into Linux 3.0 should make it so that you can read and write to a T10 PI disk without issues.

GPT and MBR partition tables, part 2

2011-05-25T08:40:55Z

Tonight I head into the horrible world of EFI GPT vs. MS-DOS MBR and all the insanity that is dealing with A**** and I** firmware design choices. All this to see if I could enable the emergency text console on my iMac, and also to test overriding bad BIOS choices to enable high-performance disk controllers.

I remember grousing on LJ years ago about my troubles working through all the insane bugs and design problems found on modern-day EFI Macs. To sum up the earlier post, EFI introduces the new "GUID Partition Table" format and runs it alongside the old MSDOS "Master Boot Record" format. Both can be found simultaneously on the same disk to describe the partitions on said disk, which means that one has to keep them synchronized or else all hell breaks loose as you get a different view of the disk depending on which table you read.

YUCK. This hybrid mode is a horrid hack necessary to get old versions of Windows to work, because only 32-bit Windows supports only MBR (64-bit Windows supports both). As any IT person can tell you, redundant copies that have to be kept in sync or disaster happens ... is a disaster waiting to happen.

Tonight I tried an experiment in converting my grub-efi Ubuntu installation back to grub-pc (grub for legacy boot mode). Unfortunately that totally mangled the disk, because I didn't realize that grub-pc installs itself in the first 63 sectors of the disk... right on top of the GPT table. grub-pc doesn't detect that it's about to blow away a GPT; it just blows it away.

So the first gotcha is you can lay out a maximum of 4 partitions on a GPT/MBR hybrid drive. The old MBR format only allows 4 partitions unless you chain them via extended partitions, which means you can establish up to 7. GPT, however, can handle 128. Regrettably, the GPT->MBR sync program will only sync the first four GPT partitions, and guess what? The partition editors do not warn you that partitions 5+ will not be sync'd. All your legacy boot OSes have to be in the first four partitions or they won't work. There are no warnings about this!

"Aha!" you say. "Just leave space to create an extended partition!" Well, you can't do that; gptsync just plain doesn't know what extended MBR partitions are. So you're really stuck with 4.

Once I noticed the missing GPT, I simply read the table of partition start and end sectors and recreated a GPT. Problem solved, right?

The second gotcha is that Apple's firmware only looks at the partition table's type codes to find potential OSX partitions. On a plain old MBR disk you'd simply label the HFS partition as type 0xAF and the EFI will go find something bootable. This is what it does if you don't have a GPT (see above). If you *do* have a GPT, then it'll go looking for a partition with GUID "48465300-0000-11AA-AA11-00306543ECAC" and refuse to examine the MBR. Smart, given the first gotcha.

Unfortunately, this brings us to the third gotcha. parted knows how to create a GPT table, but it does NOT know how to assign GUIDs to each partition. It can edit the labels, but not the GUIDs. Lame. That meant that I went from being able to boot rEFIt from the MBR to a Mac that wouldn't boot ANYTHING because the HFS+ partition had a "Windows/Linux data" GUID. LAME.

Ok, that just means we have to edit the GUID in the GPT. It turns out that gdisk does this. So, all I had to do was edit the GUIDs in gdisk, reinstall grub-efi-ia32 on the FAT32 EFI system partition, and reboot. Finally it works. Four hours of life down the drain for this idiocy.

Why did I bother trying to go from booting Linux via EFI back to booting via legacy? I was curious to see if I could jump out of X back into the text and/or framebuffers. Turns out that text mode works in legacy mode but has some bug where it always thinks it's in 80x25 mode (this makes it unusable) ... and AHCI is disabled. In EFI mode AHCI _is_ turned on, which enables advanced SSD features like command queueing and TRIM support. Just FYI, in both modes the framebuffer can be used to boot the system, but once X starts, the old console is gone until the next reboot.

Here's a fun tip if you find yourself stuck in legacy mode: if you add "setpci -d 8086:27c4 0x96.b=0x80" (SCRAE bit) and "setpci -d 8086:27c4 0x90.b=0x40" (AHCI/IDE mode) to grub, you can whack the SATA controller into AHCI mode before Linux starts. It even seems to stay in AHCI mode after a sleep/resume cycle. Note that the location changed on ICH10; on that, one only needs "setpci -d 8086:3a20 0x90.b=0x60" to enable AHCI. For those of you on the Internet, YMMV. This AHCI quirking is a horrid hack that could very well format your hard disk, and, really, your system vendor should give the user a way to enable AHCI mode. Most do, but some are bone-heads and set the controller to compatibility (ata-piix) mode.

Or, if you prefer, a shell script to perform the ICH10 quirk. You'd probably want to put this in the Linux initrd _before_ it loads the disk drivers.

if [ "$(cat /sys/devices/pci0000\:00/0000\:00\:1f.2/vendor)" = "0x8086" -a \
     "$(cat /sys/devices/pci0000\:00/0000\:00\:1f.2/device)" = "0x3a20" -a \
     ! -e /sys/devices/pci0000\:00/0000\:00\:1f.2/driver ]; then
	modprobe fakephp
	setpci -v -s 0:1f.2 0x90=0x60
	echo 0 > /sys/bus/pci/slots/0000\:00\:1f.2/power
	echo 0 > /sys/bus/pci/slots/0000\:00\:1f.5/power
	sleep 2
	echo 1 > /sys/bus/pci/slots/0000\:00\:00.0/power
fi

Enough for now.

Update: 15 April 2012: If you have grub2 (or any EFI bootable) on a FAT partition, use this command to bless it (i.e. set UEFI to boot it):
sudo bless --mount /mnt --setBoot --file /mnt/efi/refit/refit.efi --labelfile /mnt/efi/refit/refit.vollabel
(The labelfile is optional)

If you have an EFI bootable on a HFS+ partition, omit the --mount option and its argument.

Fix Guest Mouse Sync Problems in kvm/qemu

2009-09-19T22:07:42Z

Now that you have VMX working, you might notice that kvm/qemu's PS/2 mouse emulation sucks. The VM's mouse cursor totally does not follow the host's mouse cursor at all! That's a horrid mess; how would we clean that up? The ugly hack is to disable PS/2 mouse emulation and present a USB tablet instead:

$ kvm -usb -usbdevice tablet

Now, why is PS/2 mouse emulation messed up in the first place? It turns out that PS/2 mice do not report absolute coordinates; they only tell the OS about relative changes in position. "Two steps up" "Seventeen steps to the right" etc. This is fine with a physical mouse because the physical mouse moves in an arbitrarily defined rectangle that is separate from the screen. The mouse is not placed directly on the screen, so there is no need to know the absolute position of the physical mouse on the screen. In the case of a VM, however, we know the absolute position of the (host) mouse on the screen. Tablets provide a fixed size rectangle with absolute coordinates, and that's why they (should) work better.

Obvious caveat: The VM's graphical windowing system must know about USB and tablets. Allegedly X.org is not good at that, though Windows is.

Update: Recent Ubuntu X.org seems to work....

Picspam!

2009-06-03T02:27:57Z

/me rounded up a bunch of (old) panoramas and put them into the high-definition panorama viewer. Be sure to check out the (huge spike in memory cache when you load the) panorama previewer (click the "See All" button).

How'd they get there?

2009-05-01T08:30:51Z

Check out this Google Street View of Arastradero Park:

Happy May Day!

Forbid the Ubuntu 9.04 update manager from popping up any time there's new software

2009-04-30T17:59:08Z

Prior to 9.04, the update tool would simply post a notification icon in the tray. Now it pops up without any warning--annoying!

gconftool -s --type bool /apps/update-notifier/auto_launch false

reverts to the old behavior.

Flower Picspam!

2009-04-21T18:56:00Z

The first of the flowers (daffodils, tulips):
http://djwong.org/photography/oregon/portland/house_pics/flowers_2009/

The first of the wildflowers too:
http://djwong.org/photography/oregon/tom_mccall_point_apr_2009/