The "truth" about BK and the kernel

When I say "truth" I have no idea what happened, but Larry just pulling the free version didn't quite seem to fit. There had always been tension but on the whole, there was a sort of equilibrium between the kenrel community and BitKeeper.

So I've been wondering what exactly prompted BitKeeper to pull their free version. It always seemed to be about the non-compete reverse engineering clause and something to do with OSDL, Linus' employer (if a coworker is hacking BK, they can pull your license). I knew of a few names that had caused issues on the list with Larry such as Roman Zippel, Pavel Machek and Andrea Arcangeli (who wrote the openbkweb thing a while back), but nothing that directly seemed to fit. As it turns out, it seems Andrew Tridgell started work for OSDL in January of this year, and it seems he was the straw that broke the camels back. Really, if you put a binary format in front of someone like Andrew Tridgell do you expect him to not pull it apart? Obviously there's more to the story, but I haven't seen anyone talking about it.

I know that git has been said to stand for nothing, but is it possible that it's a snide reference to Australia's favourite hacker and his lack of tact leading to the BitKeeper abortion?

update Newsforge carries an article with quotes from all concerned. If it wasn't Tridge it was going to be someone else eventually; and in the end it seems based on dollars and cents rather than code. No question, BitKeeper was really good, but what non-free software giveth it can taketh away.

Fun with floating point

You probably already know that a floating point number is represented in binary as sign * significand * radixexponent. Thus you can represent the number 20.23 with radix 10 (i.e. base 10) as 1 * .2023 * 102 or as 1 * .02023 * 103.

Since one number can be represented in different ways, we define we defined the normalised version as the one that satisfies `` 1/radix <= significand < 1``. You can read that as saying "the leftmost number in the significand should not be zero".

So when we convert into binary (base 2) rather than base 10, we are saying that the "leftmost number should not be zero", hence, it can only be one. In fact, the IEEE standard "hides" the 1 because it is implied by a normalised number, giving you an extra bit for more precision in the significand.

So to normalise a floating point number you have to shift the signifcand left a number of times, and check if the first digit is a one. This is something that the hardware can probably do very fast, since it has to do it a lot. Combine this with an architecture like IA64 which has a 64 bit significand, and you've just found a way to do a really cool implementation of "find the first bit that is not zero in a 64 bit value", a common operation when working with bitfields (it was really David Mosberger who originally came up with that idea in the kernel).

#define ia64_getf_exp(x)                                        \
({                                                              \
        long ia64_intri_res;                                    \
                                                                \
        asm ("getf.exp %0=%1" : "=r"(ia64_intri_res) : "f"(x)); \
                                                                \
        ia64_intri_res;                                         \
})


int main(void)
{

    long double d = 0x1UL;
    long exp;

    exp = ia64_getf_exp(d);

    printf("The first non-zero bit is bit %d\n", exp - 65535);
}

Note the processor is using an 82 bit floating point implementation, with a 17 bit exponent component. Thus we use a 16 bit (0xFFFF, or 65535) bias so we can represent positive and negative numbers (i.e, zero is represented by 65535, 1 by 65536 and -1 by 65534) without an explicit sign bit.

IA64 uses the floating point registers in other interesting ways too. For example, the clear_page() implementation in the kernel spills zero'd floating point registers into memory because that provides you with the maximum memory bandwidth. The libc bzero() implementation does a similar thing.

google + bloglines = cool

Google Alerts is a service that fires off an email with automated searches of Google news or new websites daily.

Bloglines has a feature where you can make an email address under your profile and any email that is received will be "blogged" and appear as any other feed ... sort of like an automated email->RSS generator.

Add them up, and you've got a really cool way of keeping up to date with things you're interested in. You'll probably end up finding articles you might otherwise never have found, amazing your friends and workmates.

The only problem is that you will need to keep your email subscriptions private in Bloglines, since you have to receive the confirmation email which has details on how to cancel the web alerts (and so does each email, for that matter).

rel v rela relocations

A relocation is simply a record that stores

an address that needs to be resolved

information on how to resolve it; i.e.

  • a symbol name (actually a pointer to the symbol table, which then gives you the actual symbol)
  • the type of relocation; i.e. what to do. (this is defined by the ABI)

ELF defines two types of relocations

typedef struct {
  Elf32_Addr    r_offset;  <--- address to fix
  Elf32_Word    r_info;    <--- symbol table pointer and relocation type
} Elf32_Rel

typedef struct {
  Elf32_Addr    r_offset;
  Elf32_Word    r_info;
  Elf32_Sword   r_addend;
} Elf32_Rela

Thus RELA relocations have an extra field, the addend.

ianw@baci:~/tmp/fptest$ cat addendtest.c
extern int i[4];
int *j = i + 2;

ianw@baci:~/tmp/fptest$ cat addendtest2.c
int i[4];

ianw@baci:~/tmp/fptest$ gcc -nostdlib -shared -fpic -s -o addendtest2.so addendtest2.c

ianw@baci:~/tmp/fptest$ gcc -nostdlib -shared -fpic -o addendtest.so addendtest.c ./addendtest2.so

ianw@baci:~/tmp/fptest$ readelf -r ./addendtest.so

Relocation section '.rela.dyn' at offset 0x3b8 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000000104f8  000f00000027 R_IA64_DIR64LSB   0000000000000000 i + 8

So, sizeof(int) == 4 and j points two ints into i (i.e. i + 8). R_IA64_DIR64LSB is just defined as SYMBOL + ADDEND which means that to fixup this relocation, we need to write to Offset the value of i (which we find and resolve) and add 8 to it.

If you try this on a 386, you will not have the explicit addend as it will use a REL relocation. You will actually have to read the memory at Offset to find the addend (it will be 8).

Having to read the memory before you do the relocation has all sorts of inefficiencies, and the only positive is that it saves space (because with RELA you have that extra field and "blank" spot waiting to be filled in the binary; with REL you keep that addend in the "blank spot"). Most modern architectures have dispensed with REL relocations all together (IA64, PPC64, AMD64).

Think outside the box

There is a story that goes around UNSW about how a large computer was having constant single bit errors. Every part was replaced but it still kept happening. Eventually, someone realised that the radar tower at the nearby Sydney airport used to sweep directly over the top floor of that building, inducing the errors; a Faraday cage was installed and all was well.

Today I read another interesting problem that came from outside the box; Rob Fowler from Rice University wrote :

I've recently had discussions with several vendors who have mentioned similar magnitudes of disk performance degradation due to the coupling of vibration between cooling fans and disks. This can dramatically increase seek time by keeping the arm from settling. In one case, upgrading a chassis fan caused disk throughput to go down by a factor of 16. The solution is careful attention to vibration damping in mountings.

When you have a problem think outside the box, literally!

on asterisks in Python

Asterisks have lots of meaning in Python. Firstly, consider in a function definition

>>> def function(arg, *vargs, **kargs):
    print arg
    print vargs
    print kargs
>>> function(1, 2,3,4,5, test1="abc", test2="def")
1
(2, 3, 4, 5)
{'test1': 'abc', 'test2': 'def'}
  • *vargs puts all left-over non-keyword arguments into a tuple called vargs.
  • **kargs puts all left-over keyword arguments into a dictionary called kargs.

On the other hand, you can use the asterisk with a tuple when calling a function to expand out the elements of the tuple into positional arguments

>>> def function(arg1, arg2, arg3):
    print arg1, arg2, arg3
>>> args = (1,2,3)
>>> function(*args)
1 2 3

You can do a similar thing with keyword arguments and a dictionary with the double asterisk operator

>>> def function(arg1=None, arg2=None):
     print arg1, arg2
>>> dict = {"arg1":"1", "arg2":"2"}
>>> function(**dict)
1 2

Changing the default emacs font

in your .Xdefaults file add a line

emacs*font:6x12

You can replace the 6x12 with pretty much anything you see when clicking shift-left mouse. Make sure you merge your changes with xrdb -merge .Xdefaults.

The problem with adding something like (set-face-font 'default "6x10") to .emacs is that if you start up emacs with -nw for no window mode you'll get errors about not being able to setup the font.

How to reparent a CVS tree

You can either edit all the CVS/Root files by hand or with your own script, or more easily use the cvschroot command from the cvsutils package (predictably in a Debian package called cvsutils).

This post brought to you via cvschroot :)