1. Boot your system from a Linux system on a USB stick or CD. I used a USB stick with Ubuntu 11.04 created with unetbootin.
2. Mount at least your root and boot partitions, using cryptsetup under a target directory. I used mkdir /mnt/target && cryptsetup luksOpen /dev/sda2 root && mount /dev/mapper/root /mnt/target && mount /dev/sda1 /mnt/target/boot
3. Mount the required system directories : mount --bind /dev /target/dev && mount --bind /proc /mnt/target/proc && mount --bind /dev/pts /mnt/target/dev/pts && mount --bind /sys /mnt/target/sys
4. chroot to your target system, which should map to the disk layout your laptop would have if it had booted normally : chroot /mnt/target
5. Run grub-install to make grub behave itself : grub-install /dev/sda
6. sync and unmount everything, then reboot and see if things worked.

Wed, 18 May 2011

This TERENA Conference has seen many diverse topics covered: the future of networking, the packet v circuit debate, privacy implications in a modern connected world and the future of NRENs themselves. On top of this there's the constant interactions with other attendees - attendees with whom I collaborate with during the year via video-conference, e-mail, IM, Skype and numerous other ways. This conference is one of the few opportunities to sit down in person and discuss progress on projects, where things are going in the future, and ideas too immature or risque to be committed to e-mail yet.

As a result, it's Day 3 of the conference (day 5 in Prague for me) and I'm exhausted. Exhausted but exceptionally impressed at the quality of the presentations, the insight delivered by the speakers and those posing questions and the sheer dedication of the attendees who are tireless in their efforts to improve research and education networking.

posted at: 16:29 | path: /technical | permanent link to this entry

Tue, 17 May 2011

Why do we build ever larger and more complex networks? Is it because computers are cool? Do we just need more bandwidth to view ever more amusing videos on Youtube, ever more grammatically inaccurate lolcats or socialise more without leaving the house through Facebook? Fundamentally, do we run the risk of perpetuating the network purely for the purpose of perpetuating the network?

Jaroslav Koča disagrees. His work relates to analysis of viruses, with a view to developing medicines to prevent viral infections, or prevent them from spreading. To this end he demonstrated how his team's software, with access to large networks and supercomputing resources can test which molecular bonds viruses might make with human cells, and find alternate molecular structures that will bond better. This provides a key building block to developing medicines that prevent viral infection, or at least control them.

He was able to show that hundreds of thousands of alternative compounds and structures, and variations of them, can be tested against a viral signature, allowing what would have taken months, if not years, of research by trial and error to be achieved in three days computing time. International high-capacity research networks providing links between supercompute and storage resources allow Jaroslav and his team to collaborate with teams in other universities and countries and share compute time to achieve quicker and more accurate results.

I'm proud to be part of an NREN delivering services that might some day literally find a cure for the common cold.

Jaroslav's plenary presentation at TNC2011 provided an excellent example of what big networks, big computing, big storage and tight collaboration between educational and research institutions can achieve - and served to remind me why I do my job with HEAnet.

The presentation slides for Jaroslav's talk are available here and the video is here.
posted at: 10:56 | path: /technical | permanent link to this entry

Mon, 16 May 2011

Did you know that the FileSender project allows you to send arbitrary sized files to one or more recipients simply and confidently? Information on development is on the filesender website and filesender blog and it's freely downloadable under the BSD license if youy want to set up your own repository.

The concept is simple - a user logs in and uploads a file, providing one or more e-mail addresses to send the file to. The recipient(s) get a mail providing a website URL from which they can download the file. Federated authentication is supported, as is the concept of invitations such that people without accounts can be requested to send you files.

HEAnet, the Irish NREN (National Education and Research Network) runs the service at filesender.heanet.ie and is available to anyone in the Edugate federation.
posted at: 17:58 | path: /technical | permanent link to this entry

While the TAC (Technical Advisory Council) meeting is a closed meeting, I can say that this year, more than any previous, a huge spread of material was covered. Good discussions on matters related to next generation network design, mobile connectivity, social networking and strategy for cloud computing were all discussed, with excellent feedback and insightful questions from many of the representatives.
Overall, a hugely informative meeting, though one that posed at least as many questions as it answered. Certainly enough material there to keep me busy for the coming year, and I think that goes for the other representatives and NRENs here too.

(edit: Corrected the title to 'TERENA TAC' - apologies for my brain-fart.)
posted at: 11:45 | path: /technical | permanent link to this entry

Sun, 15 May 2011

It's still a day before the TERENA networking conference, but groups of people are meeting throughout the conference venue to discuss various technical issues and progress the state of Educational and Research Networking throughout Europe.
The Clarion Congress Hotel seems good and the TERENA technical crew here already have eduroam wireless networking in place.
You can follow the events of the conference by following #tnc2011 on twitter (and probably some other tags as well.)
posted at: 13:23 | path: /technical | permanent link to this entry

Sun, 20 Feb 2011

Dear Oracle, I would like to register that I use OpenOffice but I really don't want to agree to pages of terms and conditions to register yet another web account on a website that I'm not going to use: yours.

If you want people to register that they use OpenOffice, just let them do it without having to create a useless account on oracle.com. Thanks :-)

posted at: 14:43 | path: /technical | permanent link to this entry

Sun, 09 Jan 2011

For the interminably lazy, this is how I installed a test instance of Ruby on Rails on a virtual machine for testing purposes recently. it may not be the best way of installing things, but it's working for me and I content that this is the shortest set of instructions you'll find on the matter ;-)

1. Build a standard Ubtuntu 10.10 server instance. (I used 512M RAM + 20G drive for a virtual test server on VirtualBox.)
2. aptitude install ruby-full build-essential (note that this installs emacs. Srsly? wtf?!?)
3. aptitude install mysql-server mysql-client libmysql-ruby libmysqlclient-dev (installs MySQL and adaptor packages.)
4. aptitude install rubygems1.9.1 (install ruby gems)
5. ln -s /usr/bin/gem1.9.1 /usr/bin/gem (Yes - the package actually doesn't bother to alias it to 'gem' ... *sigh*)
6. Discover that the above step has failed and realise that the package is so awful it will not remove itself, pretty much no matter what you do.
7. Download the latest gem package from http://rubygems.org/pages/download
8. untar-gz it and run 'ruby setup.rb' in the directory.
9. rm /usr/bin/gem && ln -s /usr/bin/gem1.8 /usr/bin/gem (Unbreak the brokenness of gem1.9.1 and its packaging for Debian/Ubuntu.)
10. gem update --system (Update the gem installer itself.)
11. gem install mysql (install the MySQL adaptor)
12. gem install rails (install rails itself.)
13. Test your rails installation to make sure everything is working:
• rails new myrailsapp -d mysql
• edit myrailsapp/config/database.yml to make sure the MySQL socket location points to the right place. On Ubuntu it should be /var/run/mysqld/socket.sock
• Test your project by running 'rails server' in the project's root dir.
14. Realise that the mysql2 gem is also secretly required. Find this out by running 'bundle install' and it'll put it in (and any other missing bits) automatically.

Thu, 05 Aug 2010

Google Chrome is Google's browser and provides a refreshingly fast interface to (in particular) Javascript heavy websites. It's also a pretty good general browser, but like many others presents dire warnings of doom if you attempt to view a https secured website which doesn't have a recognised certificate. (Or more correctly, doesn't have a certificate signed by an authority recognised by Google.)
Most other browsers provide a method of installing root certificates but Google decided that they would use an already established certificate management system external to the browser, the NSS Security Toolkit. So if you want to view websites secured by free CACert SSL certificates or have the need to browse to sites secured by self-signed SSL certs (like the https management interface for my Linksys WRT54g wireless router running Tomato firmware for example) then you'll want to add some certificates to your NSS database. Fortunately, it's pretty simple to do...
First, let's install the CACert root certificate. Installing this means that Google Chrome will trust the identity of sites signed by CACert issues certificates. This is a good thing and saves clicking the "Proceed anyway" button every time.

1. First, grab the root cert file from the CACert Root Cert page. You should verify that the file is correct and has not been tampered with; md5sums and SHA1 hashes are included, along with the fingerprint signed by the CACert GPG key. I'll leave it as an exercise for the reader as to how authenticity should be verified, but for the moment let's assume we have a good, trusted copy of the CACert Root Certificate (PEM Format) saved as root.crt in your home directory.
2. If you've not already, you should install the NSS tool set - a simple sudo aptitude install libnss3-tools should work nicely. You will be asked for your login password to ensure you have permissions to install software.
3. Install the CACert root key by running certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n "CACert Root Certificate" -i root.crt
4. There is no step 4... you can now go to CACert secured web pages and Chrome will correctly verify that the SSL site cert has been signed by the CACert root certificate.

1. If you've not done so above, you'll need to install the NSS tools, using sudo aptitude install libnss3-tools or similar.
2. Next you'll need to get a copy of the certificate. Particularly with embedded devices, the easiest way to do this is by pulling the key directly from the device by pretending you're a web client. Thanks to tgulacsi78 who suggested echo QUIT | openssl s_client -connect hostname:443 | sed -ne '/BEGIN CERT/,/END CERT/p' - replacing hostname with the name of the host. In my case, I might use echo QUIT | openssl s_client -connect eeyore.han.signal2noise.ie:443 | sed -ne '/BEGIN CERT/,/END CERT/p' to get the cert from eeyore, my wireless router.
3. Copy and paste the lines between 'BEGIN CERT' and 'END CERT' into a file and save it as (say) router.crt
4. Now we import the cert in exactly the same way as previously, using a command line something like certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n "My router web admin certificate" -i router.crt
5. Note that we have to use the C flag (i.e. mark it as a CA / Certificate Authority) due to a bug in NSS. Note also that you should replace the description after -n with something that means something to you, and the argument after the -i should be the file name and path to the file we created in step 3.
6. Test that everything has worked by using Chrome to access the site secured by the self-signed certificate again and notice that this time no warnings are given.

Mon, 28 Jun 2010

Like many people I run Ubuntu on a number of servers with DNS being just one of the services provided. Bind 9 has worked extremely well to date, though I recently came across a conflict between AppArmor (note American spelling) and bind which resulted in slave domains not being replicated correctly. This happened on my systems some time on or after 2009-07-29 (and I'm only getting around to writing up the blog post now... I know...) If you see lines similar to the following in the output of dmesg, or in your system log, you may have the same problem:
audit(1234567890.462:15891): type=1503 operation="inode_create" requested_mask="w::" denied_mask="w::" name="/etc/bind/zones/slave/tmp-bkTe208LbH" pid=123 profile="/usr/sbin/named" namespace="default"
audit(1234567890.460:15892): type=1503 operation="inode_create" requested_mask="w::" denied_mask="w::" name="/etc/bind/zones/slave/tmp-3VUN2uHFUI" pid=123 profile="/usr/sbin/named" namespace="default"

To explain briefly what this means: AppArmor is an extra layer of security which effectively makes sure that system programs are allowed only limited access to the system, even though they may run as root. This helps significantly in the case of a root escalation vulnerability being discovered, since if AppArmor is correctly configured your system is somewhat compartmentalised so damage should be limited. Without this, a root privilege escalation in a daemon would allow full read and write access to the entire system, with predictably bad results. Permissions for various daemons under AppArmor are configurable in files under /etc/apparmor/ and /etc/apparmor.d/ (on an Ubuntu system at least.)

Under Ubuntu it appears that AppArmor is misconfigured for BIND 9, specifically with regard to the default setting that slave zones - where your DNS server is acting as a secondary DNS server for some domain(s) - are stored under /etc/bind9/zones/slave/. From the log lines above it's clear that AppArmor is disallowing writes to this location for BIND, with the result that secondary / slave zones can never be updated. This is a bad thing.

Fortunately, there is an easy solution. Either update your BIND config to write slave zone files somewhere else, or update AppArmor to allow BIND to write to the slave zones directory. I chose the latter, accomplished by adding the following line to /etc/apparmor.d/usr.sbin/named
/etc/bind/zones/slave/** rw,

This allows bind to read and write to the slave directory, enabling it to create the files it needs to store updated zones coming via AXFR transfers from master DNS servers. You can test by running rndc reload and watching the syslog and file timestamps in the slave directory. You should also no longer see log writings with the denied_mask="W::" key-value pair as above.

Thanks to alphageek for the first clue as to what was going on and hopefully this blog entry will provide a more secure fix for the problem.

posted at: 13:31 | path: /technical | permanent link to this entry

Sun, 02 May 2010

Since moving to Ubuntu 9.04 and 9.10 I've had repeated problems with my Dell D430 being able to suspend. One of the things that Apple got right with their laptops was that when you close the lid, the computer goes to sleep and it wakes up again ready for use upon opening the lid again. I used to enjoy similar functionality on my Dell D430 until moving to the more recent versions of Ubuntu - which is a real pity. I always hate to see regression instead of progression, particularly when it comes to open source.

I thought the issues were resolved finally until a large update was issued which my laptop installed on 1st May 2010, and now I'm back to a laptop that I have to shut down to move about again - no suspend for me. This makes the laptop significantly less useful and is pushing me (and others) towards replacing portable hardware with Apple products, since they just work. I can't speak for the world in general, but I and the people I regularly deal with don't have time to deal with intermittently faulty systems and laptops that may or may not lock up when you shut the lid (losing any work that was not saved.)

I'll try and provide a better bug report to Ubuntu, but as a reminder to myself, the following update was what has killed suspend again. Also, if you're a Dell D430 owner running Ubuntu 9.10, DO NOT grab the latest updates!

I have installed 10.04 (Lucid) on a D420 but I can't say I've tested it properly yet. I'll get around to putting it on the D430 shortly - and hopefully will be posting a good news story about how suspend works again and proper regression testing has been instituted such that it doesn't get broken again.

Aptitude 0.4.11.11: log report
Sat, May  1 2010 12:35:10 +0100

IMPORTANT: this log only lists intended actions; actions which fail due to
dpkg problems may not be completed.

Will install 34 packages, and remove 0 packages.
172MB of disk space will be used
===============================================================================
[INSTALL, DEPENDENCIES] linux-headers-2.6.31-21
[INSTALL, DEPENDENCIES] linux-headers-2.6.31-21-generic
[INSTALL] linux-image-2.6.31-21-generic
[UPGRADE] aisleriot 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] compiz 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] compiz-core 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] compiz-gnome 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] compiz-plugins 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] compiz-wrapper 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] glchess 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] glines 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnect 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnibbles 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnobots2 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnome-blackjack 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnome-games 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnome-games-common 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnome-mahjongg 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnome-sudoku 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnometris 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnomine 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnotravex 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gnotski 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] gtali 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] iagno 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] libdecoration0 1:0.8.4-0ubuntu2 -> 1:0.8.4-0ubuntu2.1
[UPGRADE] libpq5 8.4.3-0ubuntu9.10 -> 8.4.3-0ubuntu9.10.1
[UPGRADE] linux-generic 2.6.31.20.33 -> 2.6.31.21.34
[UPGRADE] linux-headers-generic 2.6.31.20.33 -> 2.6.31.21.34
[UPGRADE] linux-image-generic 2.6.31.20.33 -> 2.6.31.21.34
[UPGRADE] linux-libc-dev 2.6.31-20.58 -> 2.6.31-21.59
[UPGRADE] same-gnome 1:2.28.0-0ubuntu1 -> 1:2.28.0-0ubuntu3
[UPGRADE] tzdata 2010h-0ubuntu0.9.10 -> 2010i-0ubuntu0.9.10
[UPGRADE] tzdata-java 2010h-0ubuntu0.9.10 -> 2010i-0ubuntu0.9.10
===============================================================================

Log complete.

Mon, 19 Apr 2010

...is a pile of epic fail (on Linux at least.) I like the idea of full-text searching - but I also like my laptop to not have the CPU at melting temperature all the time and the fans running at full speed as thunderbird burns through CPU indexing what I can only assume are the same messages over and over and over again. I also like to start blog posts with a hearty generalisation, even if the behaviour is something relatively specific to my situation - it grabs the reader's attention and makes sure they read the remainder of the article ;-)

Normal behaviour, and better use of your CPU can be restored by disabling full text search / indexing. You can do this by setting the advanced configuration option mailnews.database.global.indexer.enabled to false, or as this has obviously become a big enough problem for enough people, the Edit | Preferences | General tab now has a checkbox marked Enable Global Search and Indexer. Unchecking this will also turn off this feature.

More usefully, some exceptionally bright spark has made a plugin called GlodaQuilla available. It's experimental but has not yet set fire to my laptop or caused my pets to explode - your experience may be different of course and any use of experimental plugins is completely at your own risk. I did the following and now have a full indexed search of the subset of my mail that interests me, along with a much more responsive machine (and some 500MB of disk space no longer used by the full-text index)

1. Install GlodaQuilla in the usual fashion
2. Restart Thunderbird
3. Disable indexing completely (see above for how to do that)
4. Quit Thunderbird and wait a few moments to make sure it's definitely no longer running
5. Delete the old full-text database taking up space on your hard drive. It's a file called /home/$username/.thunderbird/random.default/global-messages-db.sqlite on my Ubuntu Linux machine. You'll need to find your own global-messages-db.sqlite yourself, but the path above should give you a clue as to where it is.
6. Start Thunderbird and right click on a folder containing mail you do not want indexed. (e.g. my Archive/ folders contain old mail that I don't really need searched or indexed.) Uncheck 'Inherit' and you can now individually enable or disable indexing of the mail folder.
7. Once you have set which folders you do and do not want indexed, remember to re-enable indexing and restart Thunderbird, just to be sure to be sure.
8. Leave your computer for a period of time, if possible, as Thunderbird will need to rebuild indices for those mail folders you left with indexing enabled. This may take some time. Expect performance to be poor during this time. You can track activity by selecting Tools | Activity Manager from the menu.
9. Enjoy a nice, fast, performant full text search of the mails you actually care about searching and indexing.

Sat, 17 Apr 2010

I set up my eeePC 701 as a fileserver in the house over the Easter weekend. It runs Ubuntu and with a little configuration happily shared a few USB disks over the home network with SAMBA and NFS. Most importantly it ran pretty cool and didn't use a lot of power - important for something that was going to be left on all the time.

All worked well until the last couple of days when I noticed some pretty strange messages appearing in the logs. A reboot resulted in GRUB giving me the dreaded Error 17. Further investigation seems to indicate that the SSD has shed its mortal coil. Poo! Cue research to determine whether it's more cost and time efficient to get a replacement SSD or just buy a Mac Mini.

posted at: 22:33 | path: /technical | permanent link to this entry

Thu, 01 Apr 2010

We at HEAnet labs have been testing 3G mobile broadband USB dongles for some time now, and the key weakness identified is the problem that data download is asymmetric. This means that while you can download content at high speed, uploading content is much much slower. This means that 3G dongles are unsuitable for things like uploading large photographs or video information to the interwebs. This severely restricts their usefulness - or at least, it used to.

While the technology of 3G and in particular the USB dongles means you are limited to an asymmetric connection, by changing the polarity of the USB power feed you can alternate between high speed upload and high speed download. This means making up a USB cable yourself, but as John Beale helpfully demonstrates, this is pretty easy. Just (carefully) swap the black and red wires.

This still leaves us with the problem that the connection is asymmetric, but that's easily solved by using iptables and a second USB dongle with the polarity set to normal. Effectively, we want to configure iptables to route the download traffic through USB dongle A (which is wired normally) and route the upload traffic through USB dongle B (wired with the polarity reversed.) Thanks to PaulJ on the ILUG list for providing a basic config for iptables:

ip route add default \
   nexthop via dev weight x \
   nexthop via dev weight y
change x and y so that x/y == bandwidth ISP1 / bandwidth ISP2

Tue, 09 Mar 2010

...disk has corrupted. that would explain its inability to run backups for some time. Thanks Apple for such useful error messages ("Failed!") - helpful. *sigh*
Fortunately it's only my backup that's corrupted, and hopefully a 'Disk Utility' repair will put that back in a working state.

posted at: 23:31 | path: /technical | permanent link to this entry

Wed, 10 Feb 2010

I'm going to try making a small project in Ruby on Rails, so first I need to install the development environment on my laptop. More for my own future reference than anyone else's benefit, here's what I did:

sudo aptitude update (because it's just a good idea)
sudo aptitude install ruby-full build-essential (This installs emacs. wtf?!? Oh well...)
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz (Find the latest RubyGems package from rubyforge.org)
tar xzvf rubygems-1.3.5.tgz
cd rubygems-1.3.5
sudo ruby setup.rb (It appears to hang for a while with no output but eventually works and returns you to a prompt. So far so good...)
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
sudo gem update --system (Should say nothing to update if you've pulled down the latest version)

sudo gem install rails (Install rails)

All done and nothing remaining but to check that MySQL is up and running and start developing actual code.

posted at: 18:19 | path: /technical | permanent link to this entry

copyright © 2005-2011, Gareth Eason