Saturday, 30 March 2013
In the Linaro Enterprise Group, my task for the last several weeks was to work through a huge number of packages looking for assembly code. Why? So that we could identify code that would need porting to work well on AArch64, the new 64-bit execution state coming to the ARM world Real Soon Now.
Working with some Ubuntu and Fedora developers, we generated a list of packages included in each distribution that seemed to contain assembly code of some sort. Then I worked through that list, checking to see:
I've written a full report about what I found in the scan, and I'll be writing some more articles based on it shortly.Tuesday, 19 February 2013
My lovely wife Jo has signed up for a charity event in May this year. To help support the UK Fire Fighters Charity, she has committed to a sponsored driving challenge. If she can raise enough funds in sponsorship, she'll get to drive a fire engine, a tractor and (it's rumoured) maybe even a combine harvester too! I've chipped in some money towards the total already; I'm sure Jo would love it if some other nice people pledged on her JustGiving page too.
Go on, it's for a good cause! :-)Tuesday, 13 November 2012
Ever since I passed the age of 18, I've used my legal right in the UK to vote in every poll I could: local council, British Parliament, European Parliament and even the messed-up referendum on the Alternative Vote that we had last year. In each of those cases, I made a point of considering my available options and I voted accordingly. I went along and marked my ballot paper in person where possible, or I filled in a postal vote for the last Westminster election in 2010 as I was on vacation in Florida when it was held.
Unfortunately, it seems that my diligence in voting makes me unusual amongst the UK population these days. The long-term trend for voter turnout is downwards (as you can see).
However, the latest election that's happening in the UK (well, England and Wales outside London) this week is for the newly-created posts of Police and Crime Commissioners. It's a textbook example of how not to organise an election, as pointed out eloquently by the Electoral Reform Society. This election has been incredibly badly managed and publicised by the Government, and lots of early polls are suggesting a tiny turnout of less than 20%. Despite the political crap about "democracy" being spouted by some Tory ministers, it's clear that very few people want this change in how the police are run, and it has simply been imposed from the top. Too few people care about the results of these elections for them to be valid - most people don't want the police politicised.
After a lot of thought about the issue, I've decided how I'm going to vote this time. For the first time ever, I'm explicitly going to turn up and spoil my ballot paper as a protest - I do not want this crap. I urge other people in England and Wales to consider doing the same.Friday, 28 September 2012
We've been thinking about getting a dog for ages, but it's been difficult to work out the logistics when we've been travelling so much. Well, today's the day!
Pepper is a Parson Russell Terrier, and she's 3 years old. We just collected her from the lovely people at Wood Green Animals Shelter. Her previous owner passed away recently, so she was looking for a new home.
We've just brought her home and things are going great - she's very well behaved (so far!) and is loving a new place with lots of fuss and cuddles.
Back at the beginning of September, Jo and I were invited up to Aberdeen for a weekend for a party to celebrate the 50th wedding anniversary of some friends. Then we decided to stay around for a few days - it was our 1st anniversary just two days later. :-)
We had a great time at the party, and had some awesome weather for the weekend (especially considering the time of year!). We travelled around a bit for the next few days, visiting Crathes Castle, John O' Groats and Culloden amongst other places. We stayed in a hotel overlooking Loch Ness for the evening of our own anniversary, but no sign of Nessie! We headed back via Gretna Green and visited the famous (if tacky!) Blacksmith's there, then headed home. Having done both Land's End and John O' Groats in one year, we've covered the length of Great Britain.
We thoroughly enjoyed ourselves for a few days, especially the lovely roads with very few people on them! Also we got to see some very unusual road signs that made us chuckle...
We'll definitely go back again soon!Monday, 03 September 2012
Continuing on from my third EFI progress report...
I've added some i386 EFI support in debian-cd and built new images. Alongside a new amd64 image, we now have an i386 image and a mixed image which should hopefully boot and work on both 32-bit and 64-bit machines. The i386 support is more experimental at this point, and it's more difficult for me to test due to a lack of physical hardware. If you've got an older 32-bit EFI machine, this might work for you...
Grab the images from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload4/ if you'd like to help test. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I've posted my full set of d-i patches to the debian-boot list for review now, and hopefully we'll be able to get those changes merged in after the Wheezy d-i beta 2 build is done. Full speed ahead for EFI in beta 3!Friday, 24 August 2012
Continuing on from my second EFI progress report...
My second alpha release of a Wheezy netinst CD for amd64 EFI seemed to work OK for me and a few others, but it wasn't 100% good, with (at least) two issues:
In the couple of days since then, I've fixed both of these issues. Yay! Hybrid boot was easier than I expected - I've just added a tiny FAT partition to the image containing the grub EFI bootloader and that seems to work fine. I also found that I made a mistake in the ordering of the El Torito boot records in the last image. BIOS boot seems to be limited to just the first image available on the machines I've tested with, wherease EFI will happily work as a secondary image. I've swapped them around in my code, and things look much better now. So, time for another CD alpha release for testing.
Grab the image from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload3/ if you'd like to help test it. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I've posted my full set of d-i patches to the debian-boot list for review now, and hopefully we'll be able to get those changes merged in after the Wheezy d-i beta 2 build is done. Full speed ahead for EFI in beta 3!Wednesday, 22 August 2012
Continuing on from my first EFI progress report...
My first alpha release of a Wheezy netinst CD for amd64 EFI worked OK for me and a few others, with reasonable feedback. But it was far from complete, with (at least) three outstanding issues:
In the days since then, I've carried on hacking on things. I've not yet sorted out hybrid boot, but I've fixed the other issues. To make graphics work on the grub boot menu was a simple case of loading the right module from grub.cfg ("insmod png"). Getting grub-efi to work took quite a lot more effort, though. I've been looking through Ubuntu's version of debian-installer code and merged a fair amount of their code into my local builds, adding tweaks as necessary. Result? I've got another CD alpha release ready for testing. Yay!
Grab the image from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload2/ if you'd like to help test it. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I'll post more d-i patches to the debian-boot list for review shortly.Sunday, 12 August 2012
As promised last month at DebConf, I've been working on adding EFI support to debian-cd and debian-installer, hopefully to get it up and running. At the moment, this is basic support only for booting and setting up the installed system via UEFI. I'm not worrying about "Secure" (better named "Restricted"?) Boot yet - that comes later, and depends on the basic stuff working first.
So, how is it going? Quite well, really. I've got a
branch that I'm working on locally, and with a relatively small
amount of work there I've got a locally-built CD that boots in EFI
mode, using grub-efi as a boot loader. I've borrowed a script
Next step: the installer. At the moment, generic amd64 and i386 machines are assumed to use need dos-style partitioning, and then either lilo or grub as a bootloader. I've patched and rebuilt some of the d-i packages locally to add a new sub-architecture of "efi" for both amd64 and i386, alongside the existing "generic" and "mac" variants. Using that new sub-architecture, it's possible to add checks for amd/efi and i386/efi in various places to make the necessary changes:
Why elilo? It's the default for current ia64 systems, and for EFI Macs. I tried to get grub-efi working with the grub-installer package, but with little success so far. I'll try again with grub-efi, but for now elilo is much simpler and (importantly!) it works for me.
So, I have a netinst CD built that works for me in local testing in a qemu VM using James Bottomley's OVMF build of Tianocore. It's not pretty, but it works fine, covering all the normal Debian installation steps and giving me a UEFI system at the end of the process. I'm about to start testing on real hardware now, so I think it's worth sharing this netinst image with others too. Grab it from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload1/ . The "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. If you have an EFI system and would like to help us get it supported well in Debian, please download this image and play with it! Feedback to the debian-cd and/or debian-boot lists, please!Monday, 09 July 2012
Great productive couple of days hacking on debian-cd with Joey. I don't do DebCamp often enough - it's great fun...Friday, 06 July 2012
Yay, got to Nicaragua OK last night. It's the first time I've managed to get to DebCamp in quite a while, and I'm making the most of it. I've got slides ready for two of my BoF sessions next week (both ARM-related), and I'm preparing for the other four now. Why did I sign up to run six sessions? Maybe because I'm a bit silly, or maybe because they needed doing and other people hadn't registered them. Maybe both. :-)
BTW: these are all BoF (Birds of a Feather) sessions, not lectures. While I have an interest in each topic, I'm neither wanting nor expecting to just talk at a room full of people - I'm wanting to provoke discussions in each case. If you're interested, please join in those discussions, either physically in Managua or virtually using the live video streaming and IRC back-channels:
See you there...!Monday, 04 June 2012
...should be banned. On the flight back from Hong Kong to London, a collection of about half a dozen small children conspired to cry, shout, wail and shriek nigh-on constantly. For twelve hours. :-(
I think it's incredibly selfish to take kids on planes when they're too young to understand what's going on, or to behave reliably. Children make noise, that's natural and expected. Forcing hundreds of other people around you to deal with that noise in an enclosed space for extended periods with no way to escape is just wrong, in my opinion.
Jo and I flew in to HK a few days early for the Linaro conference, so we got some time to ourselves for doing touristy stuff. That was a good plan - we got to see quite a lot of Kowloon for a couple of days, then we wandered over to Hong Kong island itself for a day. Jo also had more time during the week while I was in the sessions at the conference.
Hong Kong is a busy place, very densely packed at all times of day and night that we could see. The first impression we got on the taxi from the airport on day one was that it was a city full of tower blocks - huge numbers of apartments stacked very closely together for living space. That makes sense, I guess, with the large population in such a small space. There are houses around with more space for people to live, but they are understandably very expensive.
Later on in Kowloon, we got to meet lots of the local street vendors, mostly offering us "Copy Rolex", "Fake handbag" and the like. Even "Sharp trousers" at one point. We decided not to try them out... :-) We saw lots and lots of very expensive shops downtown, crammed into huge shopping centres in the tourist area. As an antidote for all of that, we headed up to the "Golden Computer Arcade" to go browsing around the vast array of tiny technology stalls and shops there. Picked up some nice cheap laptop memory and some SD cards there. Yay!
Across on Hong Kong island itself on Sunday, we found my grandfather's grave then rode around on the trams for a while to get a good view of the city, including a democracy protest that happened to be on that day! We had a quick look around Tin Hau Temple, where we found a very specific set of rules for visitors:
I can only guess that they must have been terrified by gangs of kids racing their RC cars at one point!
We finished the day by heading up to Victoria Peak for the sunset view and some dinner, then back to the hotel. Killing time in a shopping centre waiting for the bus to turn up, we found a truly astonishing little item on sale, a Solar Queen!
Apparently this gift has a solar panel on the handbag to provide power to a little motor in the Queen's hand to make her wave to her subjects. We decided that we'd somehow have to live without such a delight in our lives... :-)
HK was a cool place to visit. It still feels very British in some ways, as you'd expect from an ex-colony (power sockets are still the same as ours, driving on the left, everybody spoke English, etc.) but utterly alien in others (the climate, working public transport *grin*). I look forwards to heading out there again!
Sometimes conferences can be dull and boring, but sometimes they can be just awesome in terms of finding the right people to collaborate with. Linaro Connect in Hong Kong last week was definitely one of the great ones!
I chaired my my usual sessions (armhf status and cross-distro ARM) and we had some lively discussion in both. We're probably just about done with the armhf sessions, as most distros have accepted a hard-float ARMv7 port now and there's not so much specific work left there now that future sessions will be necessary. The cross-distro work for ARM ports is likely to continue into the future, but we're going to be concentrating on bootstrapping work for ARMv8 soon.
Talking v8, there were a lot of meetings discussing the various work topics for this new 64-bit ARM architecture: kernel, toolchains, bootstrapping etc. More to come on that soon!
On top of all this useful discussion and planning, we also found time during the week for some hacking. This was the highlight of the week for me, as I found some expert help to solve my long-standing Ruby on ARM bug (Debian bug #652674, ruby1.9.1: FTBFS on armhf: test suite segfaults). Ulrich Weigand (an IBM toolchain wizard seconded into the Linaro team) sat with me for a couple of hours while we worked through reproducing the problem, only to find that he could not reproduce it! The crash had looked very much like a pthreads locking bug, which was scaring me. After some digging, we worked out where the problem was, and how it was now fixed.
For a long time on ARM, the Linux getcontext/setcontext system calls have never been implemented; apparently nobody really missed them, so they have never been a priority. In Ruby 1.9.x, the implementation of the new "fiber" primitive wanted to use getcontext/setcontext to control stack state etc. in different threads of control. In cases where they are not available or known not to work well, Ruby has fallback code to implement similar functionality. It seems that fallback code is buggy. Maybe it was correct at some point and has bit-rotted due to not being exercised, or maybe it was buggy as written, but it clearly does not work correctly for us now. In the last few months, getcontext/setcontext have finally been implemented for ARM in glibc trunk (by Michael Hope, also in the Linaro toolchain team!) and backported into current Debian and Ubuntu eglibc versions. Re-running configure and rebuilding Ruby against the most recent code in both Sid and Precise fixed the test suite crash we were seeing earlier. Yay! We could also provoke the bug again at will by quickly hacking around the Ruby source to force it to switch back to the fallback code, thereby verifying the fix.
Why does this matter? As we're expecting ARM servers to enter the market soon, web apps written in Ruby on Rails are going to be an important part of the software stack that customers will want to run. Broken fibers and threading would not help here!
It's great to meet up and work with talented folks like this; Linaro Connect is an excellent event for getting stuff done!
I never met my father's father, as he died many years before I was born. He was a soldier in the British Army and was killed in an accident in Burma in the 1950s. I'm told that back then the Army did not repatriate casualties as a matter of course like they do today, so none of the family got to see his funeral or even visit the grave. Like many of the military casualties in the Far East, he was buried in the Colonial Cemetery in Hong Kong. A few photographs were sent to my grandmother of the ceremony and the headstone, but that's all we ever saw.
Fast forward to 2012. I travelled to Hong Kong for a conference (Linaro Connect, more about that later...). I looked up the details of where grandad was laid to rest, and found a major coincidence. He was killed on 27th May 1952, almost exactly 60 years ago! I resolved to go to find him on 27th May 2012 to make the most of this lucky anniversary. Wikipedia told me that in the intervening years the Colonial Cemetery had been renamed to Happy Valley Cemetery, then simply Hong Kong Cemetery, and repurposed from military to civilian use.
Jo and I bought some flowers and headed over to Happy Valley on Hong Kong Island on Sunday afternoon. We found a very large cemetery with a helpful map posted to show the different sections, but nothing to tell us where we should be looking. One of the attendants at the office on site looked at the details we had and shrugged to say "sorry, can't help". Ah well, nobody said this was going to be very easy... What we found after a few minutes was that the graves weren't laid out in any obviously logical fashion. In any given small area, you'd find people buried from roughly the same period, but people for any given period could (and would) be scattered across multiple sections at all corners of the site. Great, time for an exhaustive search then. I started combing the site, checking all the markers I could find that looked anything like the tiny grainy black and white photo we had of grandad's headstone.
Almost two hours later, I eventually found him. Two hours of heavy work: the cemetery is built on the side of quite a steep hill, and the prevailing weather was very hot with 100% humidity. But, I forgot all that as we eventually stumbled across the correct grave in (no exaggeration!) the very last part of the last section of the site, 20a.
I was relieved at this point: as things had taken so long, I had started to worry that maybe the grave had been moved or the headstone damaged and lost. But no, I found the man who had gone off to war leaving my dad and aunt as small children. We took some photos and took note of where we had found the final resting place of Sergeant William Alexander McIntyre of the Royal Signals, a man I never met in life but clearly a very important member of our family.Monday, 09 January 2012
Back in September, I wrote about the machines that I set up to help bootstrap the new armhf port in Debian. Basing on Konstantinos' huge efforts in bringing up the new "architecture" in debian-ports, we started importing armhf into the main Debian archive on the 24th of November. Since then, those builders have been churning away night and day to build the huge collection of software that makes up the Debian archive. The current state can be seen on the armhf buildd status page, and there's a nice graph showing how quickly we've managed to run from 0 to over 90% of the archive here. (Click on the image for a larger version, or visit https://buildd.debian.org/stats/ for other versions. We overtook hurd-i386 quickly and are now ahead of the kfreebsd-* architectures.
We've recently brought 3 more similar build machines online (hildegard, howells and hummel), again sponsored by the nice folks at Linaro but now hosted at the York NeuroImaging Centre at the University of York. This gives us both more build horsepower to keep up with building more different bits of Debian (experimental, updates etc.) and more redundancy in case of problems.
We now have the vast majority of the archive built, and now a number of us are concentrating on fixing the remaining issues: language bootstraps and bugs. Also, on the 7th of January we were just added into testing, the next step on our path for inclusion as a Debian release architecture.
Setting up the machines
A lot of people have been asking me about the physical setup I showed in my last blog about these machines, so here's more details for those who are interested.
Mount the 6 boards into the mini-rack, connect up the Molex power connectors to each board, attach ethernet cables and turn it all on! Each board comes with a micro-SD card containing uboot and an Ubuntu installation. I've configured uboot to boot off the hard drive directly, but leaving configuration available to use the Ubuntu on the micro-SD as a simple rescue system should the need arise.
The Quickstart boards are not ideal physically for two reasons: the lack of SATA power, plus you need to push a power button on each board to boot it - they don't boot automatically the moment power is applied. However, they're quite inexpensive little machines and have done a great job of building the Debian archive so far! The ideal machines for us would also include more RAM at this point. CPU on these is adequate, but the larger C++ packages (yay webkit!) use a huge amount of memory at link time. Linking in swap is not the best thing, performance-wise... :-(
UPDATE 2012-01-12: Ian tells me that the newer Quickstart-R boards apparently have a different power controller; these now boot up straight away without needing you to push a button. That sounds useful.Monday, 05 September 2011
I'm in the middle of setting up new build machines for the armhf port (see the wiki for more details). We'll shortly have six machines set up in the machine room here at ARM in Cambridge:
All of these machines are Freescale i.MX53 Quickstart (aka "loco") development boards. They include a 1GHz i.MX53 CPU (based around the ARM Cortex A8, one of the ARMv7-A family). They have 1GB of RAM and native SATA. They're lovely little machines, measuring just 3 inches square. To mount them usefully in a machine room, I've mounted each board with a 320GB notebook hard drive and the necessary cabling onto a small perspex card as you can see here. Then we can fit 6 such machines and a normal PC-style ATX PSU into a 3U mini-rack. Well, it almost fits - the power supply pokes out a little so we'll need 4U of space when we come to mount it.
The Quickstart boards have been sponsored by Linaro, and ditto my time setting up these machines. Thanks!
As is common with new development boards, these machines are not quite fully supported in Debian yet. The kernels we're using are locally-built, using the sources supplied by Freescale. For now, that means a heavily-patched "2.6.35" kernel but we're expecting to be able to switch to mainline very soon. The .config I'm using is kernel.config, and I've built it natively on harris using
Similarly to the setup for the armel machines, for now I've tweaked things when installing the kernel:
Finally, I've tweaked the uboot config on the machines to use the uImage and uInitrd files that are generated by flash-kernel:
And I've added extra config into uboot to use the pre-installed Ubuntu system on the micro SD card as a fall-back:
Monday, 15 August 2011
Every year I worry that DebConf might not be as good as I hope, or not as good as previous years. Well, I've yet to be let down!
The road trip down to Banja Luka was good fun, even if it took a little longer than planned. We ended up travelling down through Germany on the same Friday as much of the country finished work/school for their summer vacation, so there was a lot of traffic. Meh, we got there in the end.
I made my usual mistake of planning some things to hack on during the DebConf week; by now I should know better... :-) I made a start on one small project, but then got so distracted by so many talks and side meetings with people that it's still waiting. As always, my own personal TODO list picked up huge amounts of extra stuff from those discussions. Catching up with, and socialising with, Debian friends from all over the world was also fun as always!
Lots of highlights of the week for me, both for technical and social reasons:
Banja Luka was a lovely place to visit, and a great host city for DebConf. Looking forwards to Managua now!Monday, 25 July 2011
For several years, Kenshi Muto has been doing some awesome work on updated/backported Debian CDs, producing CDs with support for newer hardware and newer features. I've used them myself in the past to help get awkward machines working, and I know they are a great resource for lots of other Debian users.
We've discussed things in the last few weeks, and agreed that it would be useful to host his images on cdimage.debian.org. The best place to look now is http://cdimage.debian.org/cdimage/unofficial/backports/. These are unofficial images, so please don't report bugs in the Debian BTS for them.Wednesday, 13 July 2011
So, the "Harmony Project" launched their set of contributor agreements and tools last week. Colour me unimpressed...
There's a claim on their website that they are a "community-centered group", but I don't see any list of people and organisations who contributed to this work. That bothers me. Regarding their aim to "assist organisations which use contribution agreements", I don't think that there is anything of value here for the Free Software community at all. Free Software developers don't need contribution agreements, and in my opinion encouraging their use like this is only going to cause further splintering of the community. We've managed for a very long time without them, why start now?
As a developer, I personally don't believe in contribution agreements at all. If I contribute code to a project, it will be under the terms of a good Free Software license or not at all. That's all that's needed. There's a fair body of opinion out there on this - see pieces from Bradley Kuhn, Richard Fontana and Dave Neary for more discussion.
What do you think?