An interesting extract from the latest IA32 SDM (18.20.5)

The TSC, IA32_MPERF, and IA32_FIXED_CTR2 operate at the same, maximum-resolved frequency of the platform, which is equal to the product of scalable bus frequency and maximum resolved bus ratio.

For processors based on Intel Core microarchitecture, the scalable bus frequency is encoded in the bit field MSR_FSB_FREQ[2:0] at (0CDH), see Appendix B, "Model-Specific Registers (MSRs)". The maximum resolved bus ratio can be read from the following bit field:

• If XE operation is disabled, the maximum resolved bus ratio can be read in MSR_PLATFORM_ID[12:8]. It corresponds to the maximum qualified frequency.
• IF XE operation is enabled, the maximum resolved bus ratio is given in MSR_PERF_STAT[44:40], it corresponds to the maximum XE operation frequency configured by BIOS.

In summary, TSC increment = (scalable bus frequency) * (maximum resolved bus ratio). This implies the TSC is incrementing based on some external bus source (any hardware engineers explain what happened for Core here?), and is a departure from simply assuming that the TSC increments once for each CPU cycle.

The interesting bit is that if XE operation is disabled, the bus ratio is assumed to be the maximum qualified frequency. This seems to mean that if you overclock your CPU and your processor is running at higher than the qualified frequency, attempts to measure the CPU speed by counting TSC ticks over a given time may yeild the wrong results (well, will yield the rated result; i.e. the speed of the processor you bought out of the box).

While interesting, this divergence is probably has little practical implications because using the TSC for benchmarking is already fraught with danger. You have to be super careful to make sure the compiler and processor don't reschedule things around you and handle other architectural nuances. If you need this level of information, you're much better using the right tools to get it (my favourite is perfmon2).

If you go for an ultrasound or some other imaging procedure, they may give you a CD with the images that requires some overly complicated and under-featured Windows viewer. Chances are these images are in DICOM format, which is like the AVI of the medical world.

Your first clue will be that file might report the file as an unoptimised QuickTime movie, e.g.

$ file ./QMAG0001
./QMAG0001: Apple QuickTime movie (unoptimized)

After figuring out the file type wasn't actually anything to do with QuickTime, I tried some of the many different tools and methods to convert this to something viewable. Unfortunatley, the DICOM viewer in GIMP and ImageMagick (probably the same thing?) didn't like the files at all, and neither did a range of other tools. I finally managed to do it with the dcm2pnm tool from the Debian `dcmtk <http://packages.debian.org/sid/dcmtk>`_ package -- just point it at the file and it spits out a PNM which is easily converted by all graphics tools.

You can also encapsulate a series of images in a DICOM file, like a little movie. dcm2pnm extracts these easily, but requires the --all-frames options. An ffmpeg recipe to turn these extracted files into a more easily viewable movie is:

$ ffmpeg -qscale 5 -r 20 -b 9600 -i foo.%d.ppm movie.mp4

I certainly can't guarantee this will actually work for you, as DICOM appears to be an extremely complicated format with many possible vendor extensions. But hopefully it's a starting point!

> Fools ignore complexity. Pragmatists suffer it. Some can avoid it. > Geniuses remove it.

Alan J. Perlis, Eipgrams on Programming, SIGPLAN Notices Vol. 17, No. 9, September 1982, pages 7-13.

I've been meaning to try NoMachine NX for a while. Its promise of fast remote X11 sessions sounded exactly like what I wanted to log into my work desktop remotely (I really like having a remote desktop with saved state you can just pick up from when using remote access). That was pretty much all I knew about the software, so I was a completely blank slate.

The getting started guide is the perfect example of how not to write a getting started guide.

Firstly, Section 1 - "Getting started" - gives me a full history of the product, goes into significant depth about the challenges of forwarding X11 requests, talks about the caching and compression implementation, round-trip latency measurement, the details of two-way proxying system and discusses every other feature of the software.

My eyes glazed over after about the first paragraph. That's all great -- I just want to know what to do!

At this point, I assume that I'm required to run some sort of daemon at the remote end. I download and install the server package (it is explained that the server package requires the client and agent packages as well, fine).

I'm paging down, looking for something to get me started. I'm happy to see Section 7 - "Set up your NX Server environment" (remember, at this point I though I needed some daemon running in the background constantly). It even has some commands commands to type, so I tap away, running nxserver --useradd nxtest --system. My server binary doesn't even seem to recognise these options. I give up, assuming that the server isn't running and nothing will work. The getting started guide has abruptly ended and I have no idea what to do.

As it turns out, it's all completely trivial. Here's the missing "getting started guide".

• Download and install the client, agent and server packages on the remote end. You need to have ssh access to this box.
• Install the client on your end.
• Run /usr/NX/bin/nxclient. It will start a wizard where you input the remote host name.
• The client will, under the hood, ssh to the remote end, open the tunnel it needs, start the server and do all the magic required to make things "just work". A remote desktop will appear.
• That's it!

Additional tips:

• It's easy to tunnel this connection (for example, if you have to bounce through a ssh gateway to your internal network). Do something like /usr/NX/bin/nxssh -o 'Compression=no' -L 2022:remote.host:22 -f -N user@sshgateway.company.com and then connect the client to localhost:2022. You don't want to compress this link, as NX is already doing it.
• The only way I can find to make a new session is to start nxclient with the --wizard command.
• Don't click "Disable encryption of all traffic" if you're tunneling. AFAICT this tries to redirect the client to a non-encrypted port, which obviously won't get through.

Other than the documentation, it really works as promised, making remote X11 usable. One really nice feature is that it is smart about the resolution of the remote desktop, filling up your local screen. Add to that you don't need anything setup but your normal ssh connection, and it's a great remote desktop solution.

As a photo management application, Facebook sucks. But it is something that people actually look at (as opposed to Flickr, which is great, but getting people to log-in or follow special guest pass links is a PITA).

I like to keep all my raw photos locally, using IPTC for comments (which Flickr reads -- I put them in using some custom scripts and the Python bindings of libiptcdata) and geo-tagged in the EXIF data (using my google maps point locator). I figure this way if Flickr goes bust/gets bought by Microsoft all I need to do is re-upload somewhere else.

I was waiting for Flickr to integrate with Facebook in some good way, but I then came across the very useful pyfacebook bindings, which, although being a little light on documentation, is a great way to easily throw my photos into Facebook (it's pending the NEW queue in Debian, see #511279).

My fbupload.py script might be a useful starting point if you want to do the same thing. It batches up photos into lots of 60 (the maximum photos in an album) and automatically creates the albums and uploads the photos, reading the IPTC data for comments. The only problem is that you'll have to sign up for a developer key and start a new application to get a secret key to talk to the API (if you're still reading this, I'm sure you can figure it out!).

FStream is a really neat streaming radio program for the iPhone. Although it supports various WMA streams, I found that it did not successfully work with some of the Australian ABC WMA streaming radio services.

The most reliable method seems to simply use a low-bandwidth MP3 stream over HTTP (24 kbps sounds fine and works great even over Edge). I could find a number of other blogs, etc. with static methods for streaming, but nothing that reliably did on-the-fly conversion of an incoming stream.

My solution is simple Python HTTP server I'm calling stream2mp3. It uses mplayer, lame and a few pipes to take the incoming stream (which is pretty much anything mplayer can handle, which is pretty much anything unencrypted) and spit it out as a low-bandwidth MP3 stream over HTTP.

It seems to reliably handle dropped and closed connections, and clean-up after itself. I'd certainly be interested in any bug fixes or suggestions. I guess the major disadvantages is you need a dedicated server (get yourself a linode!), it only handles one connection at a time, and if you want multiple stations I guess you run multiple instances on different ports.

With this, you can be sitting in traffic on the 101 heading to San Francisco and, with some local radio, it's just like you're sitting in traffic on the M2 in Sydney! Here's a screenshot:

~/bin$ python stream2mp3.py
Creating WAV fifo /tmp/incoming.wav
Creating MP3 fifo /tmp/output.mp3
Serving <mms://media3.abc.net.au/702Sydney> on port XXXX
mplayer running as 8524
lame running as 8525
mobile-XXX-XXX-130-107.mycingular.net - - [23/Dec/2008 18:59:22] "GET / HTTP/1.1" 200 -
[radio plays until I stop it...]
connection lost
cleanup complete, ready

Rob Landley writes:

A common trick years ago was to set up your routing tables and then have PID 1 exit so the kernel paniced, because the paniced kernel would continue to route packets with _no_userspace_running_. Darn hard to hack a system like that.

This is such a ridiculously stupid idea I think it has wrapped all the way around to the point where it just grazes "genius"!

See if you can spot the bug in this code!

#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    union {
        unsigned char raw[8];
        struct {
            char one;
            int two;
            char three;
            char four;
            char five;
        } formatted __attribute__((packed));
    } test;

    printf("one   : %d\n", (int)&test.formatted.one - (int)&test);
    printf("two   : %d\n", (int)&test.formatted.two - (int)&test);
    printf("three : %d\n", (int)&test.formatted.three - (int)&test);
    printf("four  : %d\n", (int)&test.formatted.four - (int)&test);
    printf("five  : %d\n", (int)&test.formatted.five - (int)&test);
    return 0;
}

$ gcc -Wall  -o packing packing.c
$ ./packing
one   : 0
two   : 4
three : 8
four  : 9
five  : 10

Here's the relevant bit from the gcc manual:

For an enum, struct or union type, you may specify attributes either between the enum, struct or union tag and the name of the type, or just past the closing curly brace of the definition. The former syntax is preferred.

By getting the packed attribute in the wrong place (if it's not clear, it should be before formatted), it is applied to the variable rather than the type. The compiler (both gcc and icc do this) has already laid out the structure, so misses the packing directive, and unfortunately doesn't warn (that may be a bug?).

I can tell you from experience this can be hard to track down!