Why symbol visibility is good

ELF has two related concepts for managing symbols in your programs. The first concept is the symbol binding. Global binding means the symbol is visible outside the file being built; local binding is the opposite and keeps the symbol local only (static) and weak is like global, but suggests that the symbol can be overridden.

$ cat syms.c
static int local(void) { }

int global(void) { }

int  __attribute__((weak)) weak(void) { }

$ gcc -o syms -c syms.c

$ readelf --syms ./syms

Symbol table '.symtab' contains 10 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
...
     5: 00000000     8 FUNC    LOCAL  DEFAULT    1 local
     8: 00000008     8 FUNC    GLOBAL DEFAULT    1 global
     9: 00000010     8 FUNC    WEAK   DEFAULT    1 weak
...

This is all well and good, but starts breaking down when you want to load many different modules and keep strict API's (such as, say, dynamic libraries!).

Consider that for two files to share a common function, the function must end up with a global visibility.

$ cat file1.c
void common_but_not_part_of_api(void) { }

$ cat file2.c
extern void common_but_not_part_of_api(void);

void api_function(void) {
     common_but_not_part_of_api();
}

$ gcc -shared -fPIC  -o library file1.c file2.c
$ readelf --syms ./library

Symbol table '.symtab' contains 60 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
...
    53: 00000424    29 FUNC    GLOBAL DEFAULT   11 api_function
    55: 0000041c     5 FUNC    GLOBAL DEFAULT   11 common_but_not_part_of_api
...

In the example above, both the function we want exported (api_function) and the function we don't wish exported (common_but_not_part_of_api) end up with exactly the same attributes. Binding attributes are useful for the linker putting together object files; but aren't a complete solution.

To combat this, ELF provides for visibility attributes. Symbols can be default, protected, hidden or internal. Using these attributes, we can flag extra information for the dynamic loader so it can know which symbols are for public consumption, and which are for internal use only.

The most logical way to use this is to make all symbols by default hidden with -fvisibility=hidden and then "punch holes in the wall" for those symbols you want visible.

$ cat file1.c
void common_but_not_part_of_api(void) { }

$ cat file2.c
extern void common_but_not_part_of_api(void);

void  __attribute__((visibility("default"))) api_function(void) {
      common_but_not_part_of_api();
}

$ gcc -fvisibility=hidden -shared -fPIC  -o library file1.c file2.c
$ readelf --syms ./library

Symbol table '.symtab' contains 60 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
    48: 000003cc     5 FUNC    LOCAL  HIDDEN   11 common_but_not_part_of_api
    54: 000003d4    29 FUNC    GLOBAL DEFAULT  11 api_function

Now the dynamic loader has enough information to distinguish between the two, and can stop any external access to common_but_not_part_of_api easily.

This extra information also has potential for performance improvements. Any time a symbol may be overridden, the compiler must generate a program lookup table (PLT) entry for the function so that the dynamic loader can re-direct the function call. The PLT is a trampoline which gets the correct address of the function being called (from the global offset table, GOT) and bounces the function call to the right place. An example should illustrate:

Bouncing via the PLT

In the first example, there was not enough information to tell if the function would ever be able to be overridden, hence a PLT entry had to be created and the function called through it (disassemble it to see the details!). With correct symbol visibility attributes, there is enough information to know that common_but_not_part_of_api is never to be overridden, hence the PLT (and the associated costs of trampolining) can be avoided.

The internal attribute is even stricter; it says that this function will never be called from outside this module (for example, we might pass the address of common_but_not_part_of_api to anyone). This can lead to even better code, because on many architectures transitioning to another module might involve flipping global pointer registers or other similarly expensive operations.

So that's how symbol binding and visibility attributes can work together to get you the best performance possible from your program!

Poking around in AUXV, part 2

I've written about AUXV previously, focusing on one of its most interesting applications -- its role in helping find linux-gate.so.1.

If you're starting your program, you can get the dynamic loader to echo out the AUXV fields with the environment variable LD_SHOW_AUXV, but if your process has started you'll need to pull the values out of /proc/pid/auxv directly.

This is pretty internal stuff for the dynamic loader and is probably only useful if you're writing a debugger or doing some other low-level tricks (such as debugging!). However, should you need to, here is some sample code which does just that. Hopefully it will save someone else some time!

Kernel Development Course

Here are some slides and examples I used for a kernel course I developed (some time ago now).

The course was aimed at C developers who wanted an introduction to both general UNIX-style user-space and Linux kernel development with a focus on embedded systems issues. The course is aimed at two 8-hour days, and is pretty packed in even then.

The first day is user-space development and kernel building, focusing on things like make, autotools, advanced gcc, getting cross-compilers working, configuring the kernel and building. The second day we get into kernel internals; building up a kernel module to produce some simple proc nodes, take data, crash and debug, etc, look at internals like concurrency and the driver model, and focus on USB quite a bit.

Here is a tarball of the entire thing, including the examples.

Hopefully these can help out anyone tackling the design of such a course.

  • Day 1
    • Linux philosophy, people and processes
    • UNIX build environment
      • Makefiles, Autotools
    • Executable and Linker Format
    • Advanced GCC
    • Linker, Dynamic Linker, Shared Libraries
    • Cross compilation toolchain
      • Building toolchains on Debian
    • Kernel development process
    • Source control, patches, GIT, Quilt
    • Configuring and building a kernel
    • Cross compiling kernel
    • initrd, initramfs, init tools
  • Day 2
    • Kernel source layout
    • Kbuild, Kconfig
    • Syscalls (x86-32)
    • Kernel view of userspace
    • Virtual Memory
    • Getting data in/out kernel
    • Crashing, Oops, backtraces
    • Various tools
      • systemtap, kprobes
      • perfmon, oprofile
      • valgrind
    • Basic constructs
      • Concurrency, locks
      • Interrupts, sleeping
      • Atomic access
      • Memory allocation
    • VFS layer
    • /proc
    • Char and block devices
    • Driver model
      • sysfs, kobjects
      • buses, devices
    • USB Overview
      • Protocol overview
      • Registering drivers
      • uevents
    • Useful development paradigms
    • Unit testing

Spice hacking

Sick of a ridiculous pile of tiny bottles falling everywhere in your cupboard?

Many people will sell you many expensive solutions. However, I think we've hit on something cheap, effective and practical; zip-lock bags in a deep tub, labelled with sticky tags.

  • Easy to find.
  • Easy to get any sized spoon into the bags, or for tipping if you just open a corner.
  • For freshness, you can vaccuum seal with a straw a-la Alton Brown.
  • Buy the cheap spices in bulk and pay significantly less.
  • The more expensive bags with two rows of sealing are a little easier to close, if they come in a good size for your container.
  • Label with the date if you know it.
Spice rack Spice rack Spice rack

How to find the current glibc version

A recent post reminded me of a problem I once had; determine the glibc version and its support for various things.

There's actually a little known but useful confstr call defined for just this sort of thing. Here's a minimal example:

#include <stdio.h>
#include <unistd.h>
#include <alloca.h>
#include <string.h>

int main (void)
{
        size_t n = confstr (_CS_GNU_LIBC_VERSION, NULL, 0);
        if (n > 0)
        {
                char *buf = alloca (n);
                confstr (_CS_GNU_LIBC_VERSION, buf, n);
                printf("%s\n", buf);
        }
        return 0;
}

man confstr has all the details. If you don't need it in your program, you can also just run /lib/libc/so.6 and it will print out it's version info, e.g.

$ /lib/libc.so.6
GNU C Library stable release version 2.7, by Roland McGrath et al.
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.3.1 20080523 (prerelease).
Compiled on a Linux >>2.6.24.4<< system on 2008-06-02.
Available extensions:
    crypt add-on version 2.1 by Michael Glad and others
    GNU Libidn by Simon Josefsson
    Native POSIX Threads Library by Ulrich Drepper et al
    BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

There's also another glibc trick that often comes in useful; the LD_DEBUG environment variable. Start with help and you can get more details from there.

$ LD_DEBUG=help ls
Valid options for the LD_DEBUG environment variable are:

  libs        display library search paths
  reloc       display relocation processing
  files       display progress for input file
  symbols     display symbol table processing
  bindings    display information about symbol binding
  versions    display version dependencies
  all         all previous options combined
  statistics  display relocation statistics
  unused      determined unused DSOs
  help        display this help message and exit

To direct the debugging output into a file instead of standard output
a filename can be specified using the LD_DEBUG_OUTPUT environment variable.

Whoda' thunk it!

I learned a new word today, thunk. At first, given the context, I thought it meant text (as in program code) hunk, but the Hacker's Dictionary suggests it's actually more a term for a closure.

For those not familiar with what a closure is, it's a "thunk" of encapsulated code created by the compiler which is dynamically generated and managed. Stealing the Wikipedia example:

Closures

Nested functions are probably the closest thing to a closure you can get with C (I've talked about nested functions and trampolines before). However, because the stack disappears it's not nearly as useful, I think the term for these is lexical closures, because it's basically a way to share some stack and restrict name-spaces.

Amusingly, the code using the term "thunk" was assembler, so my first search was for thunk gas (as in GNU Assembler). This of course lead to endless articles on the high price of gas — a non-renewable fossil fuel running out and capitalists realising inelastic demand can be used to line their pockets? Whoda' thunk it!

Netgear WG311 shielding

I recently picked up the Netgear WG311 V3 very cheap from Office Depot.

The card seems to work fine with ndiswrapper; there are other guides on getting it working.

The first problem was the reception was, in a word, rubbish. After putting the box back in it's usual home the house wireless was lucky to get a 7/100 signal rating. I found a work-around while the cheap external antenna I ordered is arriving; shield the antenna with foil. This increased signal to between 30-40/100, a considerable boost making it actually useful.

Cheap RF Shielding

The second problem was coming to terms with wpa_supplicant, of which the details often become very confusing very quickly. Here's the Debian 2-second guide for a simple, standard WPA network I was looking for:

  1. Add to /etc/network/interfaces

    iface wlan0 inet dhcp
    # Useful with ifup -v
    #    wpa-debug-level 3
         wpa-conf /etc/wpa_supplicant.conf
    
  2. /etc/wpa_supplicant.conf should look like:

    network={
            ssid="your_ssid"
            psk="your_password"
            key_mgmt=WPA-PSK
            proto=WPA
    }
    

Sending arbitrary keys through xtightvnc

Although it's not particularly clear, a neat feature of xtightvnc is the ability to modify the pop-up menu that appears when you press F8 to send arbitrary keystrokes.

The trick is overriding some of the X11 resources of the file. If you have a look in /etc/X11/app-defaults/Vncviewer (make sure you have a recent package) you can see the default keybindings for the popup menu, which are a good template. Assuming you want to keep them all, you can add your own buttons by starting at button 9 and re-defining the total button count in your ~/.Xresources:

xtightvncviewer*popupButtonCount: 11

xtightvncviewer*popup*button9.label: Alt-F1
xtightvncviewer*popup*button9.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F1) \
                                            SendRFBEvent(keyup, F1) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button10.label: Alt-F2
xtightvncviewer*popup*button10.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F2) \
                                            SendRFBEvent(keyup, F2) \
                                            SendRFBEvent(keyup, Alt_L)

xtightvncviewer*popup*button11.label: Alt-F12
xtightvncviewer*popup*button11.translations: #override\n\
                                            <Btn1Down>,<Btn1Up>: \
                                            SendRFBEvent(keydown,Alt_L) \
                                            SendRFBEvent(keydown,F12) \

Make sure you merge this with xrdb -merge ~/.Xresources if you don't want to bother logging out and in. This way I can easily send Alt-F1, etc. through to the other side; useful for things like switching virtual terminals in VMware workstation running remotely. I imagine if you were insane you could do all sorts of other tricks too!