Finding out if you're a Rackspace instance

Different hosting providers do things slightly differently, so it's sometimes handy to be able to figure out where you are. Rackspace is based on Xen and their provided images should include the xenstore-ls command available. xenstore-ls vm-data will give you a handy provider and even region fields to let you know where you are.

function is_rackspace {
  if [ ! -f /usr/bin/xenstore-ls ]; then
      return 1
  fi

  /usr/bin/xenstore-ls vm-data | grep -q "Rackspace"
}

if is_rackspace; then
  echo "I am on Rackspace"
fi

Other reading about how this works:

ip link up versus status UP

ip link show has an up filter which the man page describes as display running interfaces. The output also shows the state, e.g.

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
  link/ether e8:03:9a:b6:46:b3 brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
  link/ether c4:85:08:2a:6e:3a brd ff:ff:ff:ff:ff:ff

It is not described what the difference is between these two.

The state output is derived from IFLA_OPERSTATE as per this little bit in ip/ipaddress.c in iproute2

if (tb[IFLA_OPERSTATE])
  print_operstate(fp, rta_getattr_u8(tb[IFLA_OPERSTATE]));

as per operstates.txt a status of UP means that the interface is up and can be used to send packets.

In contrast, ip link show up runs a different filter

if (filter.up && !(ifi->ifi_flags&IFF_UP))
  return 0;

Thus only interfaces that are in state IFF_UP are shown. This means it is administratively up, but this does not mean it is available and ready to send packets.

This may make a difference if you're trying to use the output of ip link in any sort of automation where you have to probe the available network-interfaces and know where packets are going to get out or not.

Non-stick tortilla technique

I have never had much success with tortilla making, generally finding the fragile masa would stick to everything and tear easily. However I was listening to the Cook's Illustrated podcast where they mentioned that cooking a pizza on a stone with silicone-paper makes no difference over doing it on the stone directly. For some reason I had just never thought of cooking it with the silicone-paper on; I'd always frustrated myself trying to separate the pressed tortilla in various ways.

So here's my technique:

  1. Mix the masa as on the packet; 2 cups to 2-half cups water
  2. Leave for 20 minutes or so
  3. Use plastic-wrap on the top of the tortilla press
  4. Cut a silicone-paper square for the bottom
  5. Ball the masa and press
  6. Now the plastic-wrap will peel easily off the top of the uncooked tortilla
  7. Take the silcone-paper and put the tortilla down on the pan/hotplate
  8. Give a slight press with the spatula around the edges
  9. After about 20 seconds, the silicone-paper should peel off the tortilla fairly easily.
  10. repeat!

I've tried mixing in suet, oil and copha and none of them are worth the extra calories over just plain masa in my opinion and don't help with sticking anyway. Enjoy!

Gyro Wheel Battery Pack

Several years ago I bought a Gyro Wheel for my daughter, who probably couldn't even walk at the time. The idea seemed really good; a heavy cast-iron wheel is spun up and the angular momentum keeps the new-rider upright while they learn to ride.

I don't suppose it revolutionised bike-riding, as it seems that the company has disappeared. Now my daughter can reach the pedals, when I finally dug it out the other day it was sad to find it wouldn't charge. Googling seems to show that it's a fairly common problem.

After pulling it apart, I found that it was fairly straight-forward to replace the battery-pack. Getting in requires a torx screwdriver but is otherwise straight-forward.

The battery looks like a bit of a custom job, with two-halves of a 9.6V battery-pack split into two. You won't be able to buy anything that directly replaces this, but you should be able to find individual AA-sized NiMH rechargable batteries with welded-tabs that you can use to create a new battery-pack (don't buy batteries without tabs and try to solder directly onto the battery terminals, it won't work). Unwrap the plastic from the existing packs and just follow the polarity; if you carefully de-solder the existing wires you can save yourself a lot of work making new ones. Wrap it up securely and put everything back together.

Below are some photos just to give you an idea of what to expect.

Tenvis IP391W meta-page

Recently I purchased a Tenvis IP391W-HD camera.

I would be unlikely to recommend it. The price is certainly right and the picture quality is quite good. The Android and iPhone apps do work to watch the stream live.

However, the interface is terrible and almost useless without Internet Explorer. There is a RTSP stream (rtsp://admin:password@ip) which VLC can seem to handle, but not mplayer. The recording format (.h264) is not viewable by VLC or mplayer and all I could find is a Windows .exe to convert them to an .avi.

The motion detection gets troubled by the dark. It would really only be useful for something permanently well-lit. It did send me emails via gmail.

I have got it recording to a NFS server, but I don't have a lot of confidence in the reliability of it. I think I have it configured to record in 3600-second blocks (given the interface, it's hard to tell if I've set it up to the network, or to the internal flash, etc), but it seems to intersperse 60 minute recordings with random small recordings. Given the whole idea of a security camera is to record the unexpected, you want a lot of confidence you're actually recording, which you don't get with this. You can see below it recorded 3 hour blocks, then started going a little crazy...

-rw-r--r-- 1 nobody nogroup  69M Mar 11 01:25 0-003035.v264
-rw-r--r-- 1 nobody nogroup  69M Mar 11 02:25 0-013049.v264
-rw-r--r-- 1 nobody nogroup  69M Mar 11 03:26 0-023103.v264
-rw-r--r-- 1 nobody nogroup 5.9M Mar 11 03:31 0-033117.v264
-rw-r--r-- 1 nobody nogroup 1.5M Mar 11 03:40 0-034350.v264
-rw-r--r-- 1 nobody nogroup  17M Mar 11 04:02 0-035259.v264
-rw-r--r-- 1 nobody nogroup 306K Mar 11 04:10 0-041548.v264
-rw-r--r-- 1 nobody nogroup 4.9M Mar 11 04:23 0-042457.v264

There is a support forum, where I found the following files scattered in various posts. From what I can tell, they are the latest as of this writing. I can confirm they work with my IP391W-HD, which the system tells me is GM8126 hardware and came with firmware 1.2.8.3.

  • 1.3.3.3.pk2 - firmware (b56f211a569fb03a37d13b706c660dcb)
  • web.pk2 - a UI update that includes dropbox support. This is really for the model that has pan and tilt, so those buttons don't work. (0e42e42bd6f8034e87dcd443dcc3594d)
  • V264ToAVIen.exe - converts the output to an AVI file that mplayer will play (with some complaints) (9c5a858aa454fed4a0186cf244c0d234)

www.modern.ie offers free limited-time Windows VM's which will work to upload this firmware. Just make sure you use a bridged network in the VM; I'm guessing the firmware ActiveX control tells the camera to TFTP the data from it, which doesn't work via NAT.

Somewhat worryingly, you can telnet to it and get a login prompt (TASTECH login). So it has a built-in backdoor you can't disable.

There have been some efforts to hack the device. leecher@dose.0wnz.at did an excellent job reverse engineering the .pk2 format and writing tenvis_pack.c (no license, I'm generously assuming public domain). I used this to recreate the firmware above with a telnet daemon listening with a shell on port 2525 (no password, just telnet to it)

It's interesting to poke around, but it seems like the whole thing is really driven by a binary called ipc8126

/ # ipc8126 --help
*** TAS-Tech IPCAM/DVS
*** Version: 1.3.3.3
*** Release date: 2013-08-05 15:48:32

In general, I'd say hackability is quite low.

Warning : any of the above might turn your camera into a paperweight. It worked for me, but that's all I can say...

Skipping pages with django.core.paginator

Here's a little snippet for compressing the length of long Django pagination by just showing context around the currently selected page. What I wanted was the first few and last few pages always selectable with some context pages around the currently selected page; e.g.

Example of skip markers in paginator

If that's what you're looking for, some variation on below may be of use. In this approach, you build up a list of pages similar to the paginator object page_range but with only the relevant pages and the skip-markers identified.

from django.core.paginator import Paginator
import unittest

class Pages:

    def __init__(self, objects, count):
        self.pages = Paginator(objects, count)

    def pages_to_show(self, page):
        # pages_wanted stores the pages we want to see, e.g.
        #  - first and second page always
        #  - two pages before selected page
        #  - the selected page
        #  - two pages after selected page
        #  - last two pages always
        #
        # Turning the pages into a set removes duplicates for edge
        # cases where the "context pages" (before and after the
        # selected) overlap with the "always show" pages.
        pages_wanted = set([1,2,
                            page-2, page-1,
                            page,
                            page+1, page+2,
                            self.pages.num_pages-1, self.pages.num_pages])

        # The intersection with the page_range trims off the invalid
        # pages outside the total number of pages we actually have.
        # Note that includes invalid negative and >page_range "context
        # pages" which we added above.
        pages_to_show = set(self.pages.page_range).intersection(pages_wanted)
        pages_to_show = sorted(pages_to_show)

        # skip_pages will keep a list of page numbers from
        # pages_to_show that should have a skip-marker inserted
        # after them.  For flexibility this is done by looking for
        # anywhere in the list that doesn't increment by 1 over the
        # last entry.
        skip_pages = [ x[1] for x in zip(pages_to_show[:-1],
                                         pages_to_show[1:])
                       if (x[1] - x[0] != 1) ]

        # Each page in skip_pages should be follwed by a skip-marker
        # sentinel (e.g. -1).
        for i in skip_pages:
            pages_to_show.insert(pages_to_show.index(i), -1)

        return pages_to_show

class TestPages(unittest.TestCase):

    def runTest(self):

        objects = [x for x in range(0,1000)]
        p = Pages(objects, 10)

        self.assertEqual(p.pages_to_show(0),
                         [1, 2, -1, 99, 100])
        self.assertEqual(p.pages_to_show(1),
                         [1,2,3,-1,99,100])
        self.assertEqual(p.pages_to_show(2),
                         [1,2,3,4,-1,99,100])
        self.assertEqual(p.pages_to_show(3),
                         [1,2,3,4,5,-1,99,100])
        self.assertEqual(p.pages_to_show(4),
                         [1,2,3,4,5,6,-1,99,100])
        self.assertEqual(p.pages_to_show(5),
                         [1,2,3,4,5,6,7,-1,99,100])
        self.assertEqual(p.pages_to_show(6),
                         [1,2,-1,4,5,6,7,8,-1,99,100])
        self.assertEqual(p.pages_to_show(7),
                         [1,2,-1,5,6,7,8,9,-1,99,100])

        self.assertEqual(p.pages_to_show(50),
                         [1,2,-1,48,49,50,51,52,-1,99,100])

        self.assertEqual(p.pages_to_show(93),
                         [1,2,-1,91,92,93,94,95,-1,99,100])
        self.assertEqual(p.pages_to_show(94),
                         [1,2,-1,92,93,94,95,96,-1,99,100])
        self.assertEqual(p.pages_to_show(95),
                         [1,2,-1,93,94,95,96,97,-1,99,100])
        self.assertEqual(p.pages_to_show(96),
                         [1,2,-1,94,95,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(97),
                         [1,2,-1,95,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(98),
                         [1,2,-1,96,97,98,99,100])
        self.assertEqual(p.pages_to_show(99),
                         [1,2,-1,97,98,99,100])
        self.assertEqual(p.pages_to_show(100),
                         [1,2,-1,98,99,100])


if __name__ == '__main__':
    unittest.main()

Then somehow pass through the pages_to_show to your view (below I added it to the paginator object passed) and use a template along the lines of

<ul class="pagination">

{% if pages.has_previous %}
  <li><a href="foo.html?page={{ pages.previous_page_number }}">&laquo;</a></li>
{% else %}
  <li class="disabled"><a href="#">&laquo;</a></li>
{% endif %}

{% for page in pages.pages_to_show %}
  {% if page == -1 %}
  <li class="disabled"><a href="#">&hellip;</a></li>
  {% elif page == pages.number %}
  <li class="active"><a href="#">{{ page_num }}</a></li>
  {% else %}
  <li><a href="foo.html?page={{ page_num }}">{{page_num}}</a>
  {% endif %}
{% endfor %}

{% if pages.has_next %}
  <li><a href="foo.html?page={{ pages.next_page_number }}">&raquo;</a></li>
{% else %}
  <li class="disabled"><a href="#">&raquo;</a></li>
{% endif %}

</ul>

Running cloud images locally

Fedora and Ubuntu both provide compact cloud images that are useful for spinning up small VM's quickly (much quicker than installing from a huge ISO or even net-install).

Both these pages have single-click buttons to simply launch the VM's at Amazon, which is very cool. However, you may have reason to use them "offline" with a local KVM (outside of EC2 or OpenStack) for testing, or on a plane, etc. I didn't find a lot of information on this (although it is all available once you know what you're looking for).

So here's my quick guide for the complete newbie. Firstly, download your image of choice and put it in /var/lib/libvirt/images. Open up virt-manager and you can create your new VM via "import existing image".

You can try booting it, but the problem you'll hit is the images have no default password or way to log-in because, very sensibly, you're expected to inject that. Booting you'll see cloud-init attempting to contact 169.254.169.254 to collect the meta-data to initalize the VM; eventually it will time out and you're left at a login prompt you can't log in at. cloud-init does many things and is very flexible with lots of different plugins to initialise images in various ways. The particular one we're interested in is the No cloud plugin. This will look for data from a locally attached CD drive, which works very easily for this scenario.

Now that you know that's what you're looking for, the instructions are fairly clear; I'll repeat them just for clarity. Firstly you want to create a minimal meta-data file that has a host-name and an instance-id:

$ cat meta-data
instance-id: iid-local01
local-hostname: myhost

Modifying the instance-id will signal to cloud-init that it should run again because something changed. This is also where you can put static IP address data; see the documentation.

Secondly, create a user-data file that has the commands to enable password login; set a password, and inject your ssh public key so you can easily log-in:

$ cat user-data
#cloud-config
password: mypassword
ssh_pwauth: True
chpasswd: { expire: False }

ssh_authorized_keys:
  - ssh-rsa ... foo@foo.com (insert ~/.ssh/id_rsa.pub here)

Note for new players: that #cloud-config isn't an optional comment, it's a directive. If you leave it out, you'll probably see a little something about Unhandled non-multipart userdata starting... and your changes won't apply.

Insert those two into files into an iso:

$ genisoimage -output init.iso -volid cidata -joliet -rock user-data meta-data

Then copy init.iso into /var/lib/libvirtd/images as well, and connect that as a virtual-cd drive to your virtual-machine created from the image.

When you boot you should see some output as it injects your key. It should get an address via DHCP and then you should be able to log-in with fedora or ubuntu users. Then you can start getting more complicated with things like static addresses, package installations, running commands, etc by reading the documentation.

Skipping bash history for command-lines starting with space

I was recently reading about a case where part of the evidence appears to be a deleted bash history-file. From what I gather, the accused says that the removal was a clean-up job to remove inadvertently stored passwords rather than an attempt to hide nefarious activity.

What had managed to pass me by in 15 or so years of using bash but not reading man bash properly is the HISTCONTROL variable. If it is set to ignorespace then commands entered with a leading-space will not be stored in the history. Like all the best discoveries I found this only by accident on a machine where it was turned on.

It's been quite useful in keeping my history pruned. However, not usually for security reasons around hiding passwords — passwords on the command-line have other problems and I'm sure any security person would tell you not to use them just from a defense-in-depth perspective. That said, from a "keeping ctrl-r reversi-search useful" point-of-view it's often helpful; for example I've mostly trained my fingers to <space>rm -rf because I'm sure I'm not the only one who has deleted the wrong thing via a history-recall-combined-with-typing-to-fast scenario.

So, HISTCONTROL=ignoreboth (which also ignores duplicates) is certainly a useful one to slip into your .bashrc. At least it would be one-less thing to explain to the FBI!