debian

It took a long time to get all the parts of the Xen support into the Linux kernel. While rudimentary Dom0-support was available since 2.6.38, support for device backends were missing. It was possible to replace this backend with a userspace implementation included in qemu, but I never tested that.

With Linux 3.0, both the traditional block backend and the network backend are available. They are already enabled in the current 3.0-rc3/-rc4 packages in experimental, so the packages can be used as Dom0 and run guests. Right now the backend modules are not loaded, so this still needs some work. Neither the init scripts loads them, because the names where in flux the last time I laid hand on it, nor does the kernel themself expose enough information to load them via udev. I think using udev to load the modules is the way to go.

This step marks the end of a five year journey. Around 2.6.16 the Xen people started to stay really close to Linux upstream. With the 2.6.18 releas this stopped and the tree was pushed in different states into Debian Etch and RHEL 5. After that, Xen upstream ceased work on newer versions completely, only changes to the now old 2.6.18 tree where done. SuSE started a forward port of the old code base to newer kernel versions and Debian Lenny released with such a patched 2.6.26. Around that time, minimal support for DomU on i386 using paravirt showed up and Lenny had two different kernels with Xen support. Since 2.6.28 this support was mature and works rather flawless since. Somehow after that, a new port of the Dom0 support, now using paravirt, showed up. This tree based on 2.6.32 is released with Debian Squeeze. After several more rounds of redefining and polishing it is now mostly merged into the core kernel.

I don't know what the future brings. We have two virtualization systems supported by Linux now. The first is KVM that converts the kernel into a hypervisor and runs systems with help of the hardware virtualization. The later one is Xen that runs under a standalone hypervisor and supports both para- and hardware virtualization. Both works, KVM is easier to use and even works on current System z hardware. It can be used by any user with hopefully enough margin of security between them. Xen's home is more suited for servers, where you don't have users at all. Both have advantages and disadvantages, so everyone have to decide what he needs, there is no "one size fits all".

After a long time, I decided to write again. I decided to start with software I wrote for my own usage that could be usefull for other people. I'll start with python-dvdvideo, a DVD-Video reader written in Python 3, and the reference tool dvd-video-backup-image, a generic DVD-Video dumper. Lets see, if this blog will see more postings in the future.

I was asked to take a look at a machine where aptitude don't even want to do the upgrade to etch. A first inspection shows some weird repositories in the source.list file and many daemons noone ever should use on that machine. I was able to do the upgrade with apt-get then.

After some time I got asked over modifications in /etc/exports. It basically included the following content:[1]

/     *(rw,async,no_root_squash)
/home *(rw,async,no_root_squash,nohide)
/usr  *(rw,async,no_root_squash,nohide)
/var  *(rw,async,no_root_squash,nohide)

The machine was taked out of service immediately. We'll have no chance to answer the question whether this was silliness or intend.

I was forced to try Ubuntu Hardy in the new university pool. The setup includes one Linux server dedicated for the pool, one Windows AD for Kerberos authentication, one Windows fileserver with user data and 20 clients. The clients are new HP machines with a Radeon Xpress 200 card.

First problem was nasty, the X server turned the display black and then crashed, leaving an unusable console behind. Even a blacklist of the radeon module does not work. Somehow the Xorg radeon driver loads the module on its own, ignoring the modprobe blacklist. Only a hard blacklist using install radoen /bin/false in the modprobe config was able to prevent this. Lets hope that the endeavors to remove much priviledges from the X server goes good.

For the homes two setups was tested:

• Homes on the Windows server via cifs, mounted via pam_mount using NTLM password.
• Homes on the Linux server via nfs version 3.

The first one just produced an error that some Gnome component was unable to lock ~/.ICEauthority. This could be worked around in the Xsession. Otherwise it was usable to slow, which may be a problem with the Windows server.

The later, plus a reinstallation with fglrx, produced first unresponsive OpenOffice.org windows and then a complete unusable Gnome desktop. I would not completely vote against a problem with fglrx or even a broken installation, but the amount of problems exceeded the threshold.

Now we will try Lenny and see if this also happens there. Especially home via nfs is not that uncommon that we can let it broken in a release.

I found a machine which is not so ancient and did some tests with Xen on it.

I found some weird opcodes in the s390 instruction set some time ago. I finally want to use it as I have access to such a machine. The result is rather slim:

.globl cu41
        .type   cu41, @function
cu41:
.L2:
        cu41    %r2,%r4
        ipm     %r1
        srl     %r1,28
        chi     %r1,3
        je      .L2
        ltr     %r1,%r1
        je      .L3
        lcr     %r1,%r1
        lgfr    %r3,%r1
.L3:
        lgr     %r2,%r3
        br      %r14

The cu41 opcode translates UTF32 to UTF8. Each "parameter" is a register pair [1], which describes address and length of the buffer. This opcode will return after a not specified number of translated characters. The rest is boiler plate code for condition code checking. The loop makes sure that anything is converted within the function unless another error occured.

Xen 3.3 added a tool called PV-GRUB. It is a GRUB (legacy aka 0.97) built against MiniOS and bootable as a PV kernel. It works fine on a Xen 3.2.1.

It is easy to use, I just set this in my config:

kernel = "/usr/local/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0)/boot/grub/menu.lst"

However it decided to not load my config as defined in the config and simply show me that:

    GNU GRUB  version 0.97  (524288K lower / 0K upper memory)

       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]

grubdom>

The reason is quite easy:

grubdom> root (hd0)
 Filesystem type unknown, using whole disk

Some debugging later, the reason is known and fixed.

grubdom> root (hd0)
 Filesystem type is ext2fs, using whole disk

grubdom> kernel /boot/vmlinuz-2.6.27-rc5-amd64

grubdom> initrd /boot/initrd.img-2.6.27-rc5-amd64

grubdom> boot
block error -1 for op 2
close blk: backend at /local/domain/0/backend/vbd/23/51712
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.27-rc5-amd64 [...]
[    0.000000] Command line:
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[...]
[    0.022093] Booting paravirtualized kernel on Xen
[    0.022101] Xen version: 3.2-1

For those who did not yet got it: Xen upstream decided to drop non-PAE x86_32 support from Xen.

As Linux-VServer is currently asleep, I thought about what may be possible only with the functionality the upstream Linux kernel provides and a helper module.

mount --rbind $newroot /
chdir /

Today I got another broken DVD. It is the special edition of The Shawshank Redemption. It is rather new and libdvdread just chokes on it. At least the old one does, while the version in sid is able to read them, thanks to a small patch.

The DVD exploits a feature of the standard. The filesystem on it (UDF) provides a size for the info files (VTS_XX_0.IFO) and the corresponding backup file (VTS_XX_0.BUP). This sizes are void, only the offsets within the files matters. The sizes are just too small and libdvdread checked that while reading them.

Thanks to the patch, libdvdread now updates its internal knowledge while parsing the file. With the fix it is possible to play the DVD, at least with mplayer. But I want to play it on my machine without optical drive. I used dvdbackup to read other DVDs but it simply failed this time.

This DVD also have broken backup files. They have different sizes, sometimes they even look empty, and contents than the real ones. With the corrent amount of force (just ignoring them) and some magic (#467075) dvdbackup copies something to disk which is playable with mplayer (libdvdread) and totem (libdvdnav).

Okay, nothing is complete. This DVD contains another title set which seems to be never referenced. It have a completely bogus info file and a 1GB menu VOB.

As I have a copy now, I'm away, watching a really great movie.

Xen support finally landed in upstream Linux. Okay, it is rather limited yet, but usable.

It supports the following:

• Unprivileged domain (domU).
• x86_32 (i386 without PAE) and x86_32p (i386 with PAE).
• Console.
• netfront and blkfront.

Changes to the old Xen patch:

• Block devices does not support takeover of hdXY and sdXY. Use xvcX (xvca, xvcb, ...) as device names.
• Console is hvc0. You must supply console=hvc0 at the command line. (Use the extra definition in the Xen domain config file.)

Yet missing things:

• Suspend, resume (also makes migration impossible).
• Ballooning.
• x86_64. Xen upstream currently waits for an unified x86 tree.
• Privileged domain (dom0).
• netback, blkback, pciback and pcifront.

Hmm. Debian fixed this problems, but Ubuntu manages to get them also:

P: Configuring package volumeid
O: cp:
O: cannot stat `/etc/fstab'
O: : No such file or directory
O:
O: dpkg: error processing volumeid (--configure):
O:  subprocess post-installation script returned error exit status 1

I finaly got some time to do development. I decided to use this for extending cdebootstrap. And it got several new features and cleanups.

Most notable change is the support to check Release files against the pgp-signature. This feature is enabled by default and uses the keyring supplied by debian-archive-keyring.

It now also includes definitions for newer Ubuntu releases and it works in my testcases. The keyring needs to be supplied manualy.

The rest are usual cleanups on the codebase and deprecation of some not longer used features.

After another debugging session I have to declare that caching of ressources with Vary headers is not working with squid 2.6. I have to check if this was working with squid 2.5 as Plone provides a predefined setup for this.

Lets see what upstream say about this.

The linux kernel includes a SCSI target infrastructure since 2.6.20. Most of the code is located in the userspace and supports iSCSI, ibm i/pSeries virtual SCSI and Xen SCSIback.

To work properly it needs a bunch of patches on top of 2.6.21-rc.

After I got it working I did some tests with bonnie. First the target:

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
                 2G           32124  17 18063   6           45815   4 324.6   0

Now with the vSCSI initiator:

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
                 2G           18513   9 16208   5           34003   3 221.4   0

And the iSCSI initiator:

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
                 2G           31631  14 12809   4           39241   5 306.3   1

As this tests was done on one OpenPower machine, all transfers uses DMA between the two systems.

The vscsi case is currently limited to 128KiB per request, maybe this is a problem.

I thought the problems with pvmove was fixed with devmapper 1.02.12, but I just run into this problem again. Not sure if there was different fixes in later versions.

pvmove suspends the devices and fails to reload tables under some conditions. This time it just blocked while the devices for /usr und /var was suspended. This means the system is dead.

Update: It seems to be related to the number of LVs to move. Only one works all the time.

It happened again. The linux-2.6 2.6.20-1 release failed for arches which we don't build snapshots for. This means that none of them was ever built in the time after I commited the whole stuff.

As this happens over and over again I consider this a real problem now. So the following arches needs new debian-kernel maintainers: alpha, hppa, and mips.

For etch, we will have apt security in place, so we can be sure that the stuff comes from the correct archive. But it is not possible to disable that checks only for one source, just for anything.

Buildds uses at least one mirror: incoming.debian.org aka ftp-master.debian.org. There are two queues, the accepted autobuild queue and the main archive. The accepted autobuild queue is not signed at all, it does not provide a Release file. The archive needs some time to generate the Packages files each dinstall run and have broken sigs during this time.

This means: buildds can't use APT security at all. And no, there is no other mechanism to ensure data integrity.

The buildd admin job is a rather dumb one. You get between 20 and 60 mails per day; most of them build logs. This logs, which are mostly sent unsecured through the public internet, have to be signed and the only available key is the personal key of the admin. So on one hand you have to make sure that the key is secure, on the other hand you have to find a way to sign a rather large amount of stuff.

For the debian-kernel archive, which I operate, and the pkg-voip/pkg-gnome/pkg-kde-extras archive, which is operated by Kilian, we decided to sign the uploads automaticaly. Each buildd get its own key and the used DAK includes a patch which restricts this keys to do uploads of only binaries of the correct arches.

This drasticaly reduces the time until a new package is uploaded; this means much less failed builds because a build dep is not yet built. Also it reduces the places where it is possible to do harm; you have to attack the buildd machine itself instead of the complete mail setup between buildd and admin.

Why is there no automatic or at least semi automatic infrastructure for LSB tests? The last published result is from the beginning of the last year and only for sarge.