Tuesday, 01 November 2016
So, it's now been twenty years since I became a Debian Developer. I couldn't remember the exact date I signed up, but I decided to do some forensics to find out. First, I can check on the dates on my first Debian system, as I've kept it running as a Debian system ever since!
jack:~$ ls -alt /etc ... -rw-r--r-- 1 root root 6932 Feb 10 1997 pine.conf.old -rw-r--r-- 1 root root 6907 Dec 29 1996 pine.conf.old2 -rw-r--r-- 1 root root 76739 Dec 7 1996 mailcap.old -rw-r--r-- 1 root root 1225 Oct 20 1996 fstab.old jack:~$
I know that I did my first Debian installation in late October 1996, migrating over from my existing Slackware installation with the help of my friend Jon who was already a DD. That took an entire weekend and it was painful, so much so that several times that weekend I very nearly bailed and went back. But, I stuck with it and after a few more days I decided I was happier with Debian than with the broken old Slackware system I'd been using. That last file (fstab.old) is the old fstab file from the Slackware system, backed up just before I made the switch.
I was already a software developer at the time, so of course the first thing I wanted to do once I was happy with Debian was to become a DD and take over the Debian maintenance of mikmod, the module player I was working on at the time. So, I mailed Bruce to ask for an account (there was none of this NM concept back then!) and I think he replied the next day. Unfortunately, I don't have the email in my archives any more due to a disk crash back in the dim and distant past. But I can see that the first PGP key I generated for the sake of joining Debian dates from October 30th 1996 which gives me a date of 31st October 1996 for joining Debian.
Twenty years, wow... Since then, I've done lots in the project. I'm lucky enough to been to 11 DebConfs, hosted all around the world. I'm massively proud to have been voted DPL for two of those twenty years. I've worked on a huge number of different things in Debian, from the audio applications I started with to the installer (yay, how things come back to bite you!), from low-level CD and DVD tools (and making our CD images!) to a wiki engine written in python. I've worked hard to help make the best Operating System on the planet, both for my own sake and the sake of our users.
Debian has been both excellent fun and occasionally a huge cause of stress in my life for the last 20 years, but despite the latter I wouldn't go back and change anything. Why? Through Debian, I've made some great friends: in Cambridge, in the UK, in Europe, on every continent. Thanks to you all, and here's to (hopefully) many years to come!Friday, 09 September 2016
Another year, another OMGWTFBBQ. By my count, we had 49 people (and a dog) in my house and garden at the peak on Saturday evening. It was excellent to see people from all over coming together again, old friends and new. This year we had some weather issues, but due to the delights of gazebo technology most people stayed mostly dry. :-)
Also: thanks to a number of companies near and far who sponsored the important refreshments for the weekend:
As far as I could tell, everybody enjoyed themselves; I know I definitely did!Friday, 02 September 2016
A couple of weekends back, we had an awesome time at Jonathan and Charlene's wedding. I'd have blogged sooner, but I had to wait for this photo of the happy couple with the Debian gang... :-)
Unfortunately, somebody let Charlene hide at the back! Follow the link for more photos...Sunday, 29 November 2015
I'm happy to chip in and help the awesome folks at the Software Freedom Conservancy with funding for their work, standing up for copyleft. It's clear that there are a lot of people who will ignore the terms of Free Software licensing, whether by oversight or deliberately, so the SFC are doing an important job working to defend the Freedoms that lots of us are using every day as developers and users.
It seems that some of SFC's corporate supporters have stopped sponsorship since the beginning of their lawsuit in Germany against VMware for GPL violations. Maybe some folks are happy to support SFC, but not when they really push things like this. Let's hope that we can find many more individual supporters to cover SFC's funding needs instead. This week, there's an anonymous pledge to match donations from new supporters so right now is an even better time to sign up!
Thanks to Paul Wise for posting about this and indirectly prodding me to sign up too.Friday, 25 September 2015
VLANd is a python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.
There's more information in the README file. I've just released v0.4, with a lot of changes included since the last release:
VLANd is Free Software, released under the GPL version 2 (or any later version). For now, grab it from git; tarballs will be coming shortly.Sunday, 02 August 2015
We've just started a new team in Debian for maintaining our UEFI packages together, with git repositories in a shared project on alioth etc. We're just working out the exact details of how we're going to manage things, but for now we've moved the following packages under the team's umbrella:
and in the future we'll clearly end up adding more. We've also started a new IRC channel (#debian-efi) on irc.debian.org aka irc.oftc.net. New members always welcome to help with the work here!
There can be issues with shipping installer images including UEFI. But they're mainly due to crappy UEFI implementations that vendors have shipped. It's fairly well-known that Apple have shipped some really shoddy firmware over the years, and to allow people to install Debian on older Apple x86 machines we've now added the workaround of a non-UEFI 32-bit installer image too. But Apple aren't the only folks shipping systems with horrendously buggy UEFI, and a lot of Linux folks have had to deal with this over the last few years.
I've been talking to a number of other UEFI developers lately, and we've agreed to start a cross-distro resource to help here - a list of known-broken UEFI implementations so that we can share our experiences. The place for this in in the OSDev wiki at http://wiki.osdev.org/Broken_UEFI_implementations. We're going to be adding new information here as we find it. If you've got a particular UEFI horror story on your own broken system, then please either add details there or let me know and I'll try to do it for you.
You might have seen some of the posts I've written in the last few months about adding support in Debian for so-called Mixed-EFI systems like the Intel Bay Trail: a 64-bit processor shipped with a 32-bit EFI implementation.
I've finally seen a public justification from Intel evangelist Brian Richardson as to why these systems are crippled^Wconfigured this way, and it's nice to see our guesses confirmed. The reason is simply cost - like most consumer PCs shipped today, they come with Windows. In terms of system design, it's cheaper to just include the limited memory and storage needed for 32-bit Windows. 64-bit Windows takes a lot more storage in particular. And on modern systems 32-bit Windows can only boot using 32-bit UEFI. Fair enough...
However, Brian goes on to state some more things that are simply out of date, saying that "Linux support for UEFI IA32 is still an unanswered question". Ummm, Brian: we've got working 32-bit x86 UEFI support in our standard Jessie (and newer) installation images already, and they work just fine on CD/DVD or USB stick. We've even gone one stage further than anybody else (thus far!) in adding easy support for running a full 64-bit Linux system on top of those 32-bit UEFI implementations.
I say "thus far" here because all the work here here is Free Software. Other folks added the support in Linux for making a 64-bit kernel work with a 32-bit UEFI; I added code in Linux to expose some of the details to userspace, and code in Grub to work with it. My changes have gone upstream already, so I'd expect to see other distros like Fedora or Ubuntu also using them soon.Friday, 31 July 2015
VLANd is a python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.
There's more information in the README file. I've just released v0.3, with a lot of changes included since the last release:
VLANd is Free Software, released under the GPL version 2 (or any later version). For now, grab it from git; tarballs will be coming shortly.Wednesday, 01 July 2015
Jo and I spent a few days in Sweden and had an awesome time! The main reason for being there was Leif and Maria's wedding way up north in Skellefteå. They cunningly organised their ceremony for the Midsummer weekend, which was an excellent plan - we had a full weekend of partying while we were there. :-)
We had some time to ourselves while we were there, so we wandered about a little and got to see some of the beautiful coastal countryside.
Then on the way home we stopped off in Umeå to visit Mattias Wadenstein (maswan) and his wife Melanie, and he showed me around some of the machines that he's been admining on behalf of Debian. Maybe I'm a sad geek, but I feel quite a bond with one of the machines there, pettersson.debian.org. It's the official CD build machine for Debian, and I've been responsible for thrashing it really hard for the last 5 years or so... :-)
The only downsides from the trip were the massive tiredness (midnight sun is pretty, but notconducive to sleep!) the mosquito bites and the nasty plague^Wcold that we picked up while we were there... Ah well. :-)Thursday, 11 June 2015
I've had some 8GB USB keys made, with the Debian swirl and text. By buying a reasonable number, I've got what I think is a good price for nice high-quality keys (metal body with a solid loop for attaching to a keyring). I'm now selling these for 7 pounds each, and I'm planning on bringing some to DebConf 15 too, where they'll be 10 EUR.
They're selling faster than I expected - if you're interested in buying one (or several!), please let me know. If there's enough demand, I may order more.Tuesday, 19 May 2015
As shipped, Debian Jessie (8.0) did not include kernel support for the USB controller on APM X-Gene based machines like the Mustang. In fact, at the time of writing this that support has not yet gone upstream into the mainline Linux kernel either but patches have been posted by Mark Langsdorf from Red Hat.
This means that installing Debian is more awkward than it could be on these machines. They don't have optical drives fitted normally, so the neat isohybrid CD images that we have made in Debian so far won't work very well at all. Booting via UEFI from a USB stick will work, but then the installer won't be able to read from the USB stick at all and you're stuck. :-( The best way so far for installing Debian is to do a network installation using tftp etc.
Well, until now... :-)
I've patched the Debian Jessie kernel, then re-built the installer and a netinst image to use them. I've put a copy of that image up at http://cdimage.debian.org/cdimage/unofficial/arm64-mustang/ with more instructions on how to use it. I'm just submitting the patch for inclusion into the Jessie stable kernel, hopefully ready to go into the 8.1 point release.Thursday, 23 April 2015
I'm happy with the progress we've made for debian-installer and related packages for the Jessie release. We're going to end up with a release that's better in a number of ways than what we've had before.
1. Big EFI enhancements
I've already blogged a lot about the stuff I've worked on here, so I'll just summarise for now some of the improvements we've got over Wheezy.
2. Openstack images
In collaboration with Thomas Goirand, we now have amd64 Openstack Jessie image builds being produced every week, and there will be an official image made to go with the Jessie release too. See http://cdimage.debian.org/cdimage/openstack/testing/ for the current image.
3. Debian-live images
As of a few weeks ago, we've also added started doing weekly builds of live Debian images for amd64 and i386, using software and configuration from the Live Systems Project. See http://cdimage.debian.org/cdimage/weekly-live-builds/ for the current weekly images. These will be produced in sync with the Jessie release too.
4. New architectures
We've added installation media for the two new architectures added
I'm particularly proud of the arm64 images. With help from Ian
Campbell, Leif Lindholm and Thomas Schmitt I've managed to make
EFI-compatible CD images in an
Hopefully post-Jessie we'll even be able to start providing live images and openstack images for more architectures too.
More help needed yet!
First of all, we're planning to release Jessie as Debian 8 this coming Saturday (25th April). Help with testing the installation and live images as they're produced would be lovely - please join us on the #debian-cd channel on irc.debian.org and we'll co-ordinate there.
Secondly, there's an almost endless variety of machines out there. I've updated information about how Debian installation works on some of the more awkward Mac Mini machines, but we don't yet cover all the bases even there. It would be great to update the information about other machines such as the Macbook range as well - currently a lot of these pages are well out of date and won't be helpful for new users. Please test on machines if you have them, and help improve Debian's documentation here.Wednesday, 08 April 2015
As a follow-up to my post about bootstrapping arm64 in Debian, we've had more hardware given to Debian for us to use in porting and building packages for arm64. Applied Micro sent me an X-Gene development machine to set up and use. Unfortunately, the timing was unlucky and the machine sat on my desk unopened for a few weeks while I was on long holiday in Australia. Once I was back, I connected it up and got it working. Out of the box, a standard Jessie arm64 installation worked using network boot (dhcp and tftp). I ran through d-i as normal and installed a working system, then handed it over to the DSA and buildd folks to get the machine integrated into our systems. Easy! The machine is now up and running as arm-arm-03.debian.org and has been building packages for a few weeks now. You can see the stats here on the buildd.debian.org site.
In terms of installation, I also got the machine to boot using one of our netinst images on a USB stick, but that path didn't get very far. The USB drivers for this hardware have only quite recently gone into the mainline kernel, and haven't been backported to the Debian Jessie kernel yet. I'm hoping to get those included shortly. There's also an option to replace the U-Boot firmware that came with the X-Gene with UEFI instead, which would be much more helpful for a server platform like this. I'll look into doing that upgrade soon too, but probably after the Jessie release is done. I don't want to jinx things just now. *grin*
Thanks to APM for their generous donation here, and particularly to Richard Zenkert for his help in getting this machine shipped to us.Monday, 30 March 2015
One final update on my work for UEFI improvements in Jessie!
All of my improvements have been committed into the various Debian packages involved, and the latest release candidate for Jessie's debian-installer build (RC2) works just as well as my test builds on the Bay Trail system I've been using (Asus X205TA). Job done! :-)
I'm still hoping to maybe get more hardware support for this particular hardware included in Jessie, but I can't promise. The mixed EFI work has also improved things for a lot of Mac users, and I'm planning to write up a more comprehensive list of supported machines in the Debian wiki (for now).
There's now no need to use any of the older test installer images - please switch to RC2 for now. See http://cdimage.debian.org/cdimage/jessie_di_rc2/ for the images. If you want to install a 64-bit system with the 32-bit UEFI support, make sure you use the multi-arch amd64/i386 netinst or DVD. Otherwise, any of the standard i386 images should work for a 32-bit only system.
My kernel patch to add the new /sys file was accepted upstream a while back, and has been in Linus' master branch for some time. It'll be in 4.0 unless something goes horribly wrong, and as it's such a tiny piece of code it's trivial to backport to anything remotely recent too.
I've also just seen that my patch for grub2 to use this new /sys file has been accepted upstream this week. Again, the change is small and self-contained so should be easy to copy across into other trees too.
Mixed EFI systems should now have better support across all distros in the near future, I hope.Friday, 20 March 2015
Jo and I just got back from our massive holiday in Australia. We had an awesome time overall, fitting in lots of stuff in 4 weeks. Time for a quick write-up and some photos!
We flew into Sydney, then straight onto Uluru for the obligatory sunset and sunrise viewings. We didn't climb the Rock, both for sensitivity reasons and (to be more honest!) it looked way too much like hard work in 40-plus degree heat.
Coach over to Alice Springs, where we had a very quick look around before taking the Ghan train down to Adelaide. The train was fun for a day, and we got to see a lot of desert. In Adelaide, we had a look around the city (lovely colonial feel!) and got a couple of evenings in fun comedy shows at the Fringe. Great fun!
On to Tasmania, where we did a quick (3 days) run around the island by car: into Hobart, up the east coast. Stopped in Swansea (a nice version!) for some heavenly Devonshire teas, then on up to Grindelwald near Launceston. Visited Trowunna Wildlife Park to see (and cuddle!) lots of local animals, which was amazing - Jo's favourite day of the holiday. Then on to Queenstown and drive back down to Hobart past some impossibly beautiful views around Cradle Mountain. Tassie's gorgeous - like the best bits of Scotland, Wales and Cornwall but with even fewer people and better weather.
Next, on to Sydney for Harry and Cath's wedding. We stayed up in Chatswood. Not knowing anything about the area beforehand, we were a little surprised to basically find ourselves back in Hong Kong! We spent most of the weekend catching up with friends from the wedding group, and the wedding itself was at Quarantine Station, overlooking the harbour. It couldn't have been a more perfect location / weather / view for our friends' big day! We squeezed in a couple of the open-top bus tours of Sydney on the Sunday, but got caught in the horrendous storm that hit and ended up sheltering downstairs under cover on the bus. I'm told Bondi is lovely, but it all looked grey from the bus. :-P
Down to Melbourne on the train (bit of a wasted day, in hindsight), where we wandered around the city quite a bit. Caught up with an old friend who lives there for a day, and we did a wine tour up the Yarra Valley which was fun too.
Up to Port Douglas, where we headed out to the Reef for my highlight of the holiday: a snorkelling tour with some local marine experts who showed us the local flora and fauna. We also visited a local Aboriginal cultural centre, skyrail and scenic railway around Kuranda village.
Down to Hervey Bay and a 1-day tour of Fraser Island - an amazing place in combination with quite a thrill-ride experience just being driven around on the sand tracks. Finally, down to Brisbane where we wandered around and visited both the Lone Pine Koala Sanctuary (more cuddles!) and the Gold Coast. Then the long flights home. Whew!
We're knackered now. We knew we could't fit everything in, but we're glad we travelled all over and got tastes of almost everything. Now we can work out where we want to spend more time on our future visit(s). We'll definitely want to head over and see Perth and some of WA next time, and definitely more time in Tasmania, Sydney and Adelaide.Friday, 13 February 2015
I've been working on this for too long without really talking about it, so let's fix that now!
VLANd is a simple (hah!) python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.
There's more information in the README file. I've just released v0.2, with a lot of changes included since the last release:
I've demonstrated this code today in Hong Kong at the Linaro Connect event, and now I'm going on vacation for 4 weeks. Australia here I come! :-)Sunday, 08 February 2015
Seen on an ATM in Hong Kong airport.Sunday, 11 January 2015
Time for another update on my work for UEFI improvements in Jessie!
I've spent more time on the integration of 32-bit grub-efi with a 64-bit Debian system, and just published a new test image on pettersson. I've added:
These remove the manual steps that were necessary for a 64-bit installation with the previous build. I've just used this exact image (and a network mirror) to install a fully-functional 64-bit Gnome system on the X205TA, simply by selecting "64-bit install" from the GRUB menu and following prompts. Yay! Visit http://cdimage.debian.org/cdimage/unofficial/efi-development/jessie-upload3/ to download and test the image.
Now, there's no guarantee that the kernel patch I've submitted to the linux-efi folks will be accepted in its current form, and even if it is I'll have to get it and the other code I've written accepted into the various packages and then into Jessie! But for now this image should work just fine for Bay Trail folks I hope!
WARNING: this CD is provided for testing only. Use at your own risk! If you have appropriate (U)EFI hardware, please try this image and let me know how you get on, via the debian-cd and debian-boot mailing lists.
For now, I'm going to pause development here. The core code I'm using to make these images is all in the debian-cd and d-i repos, and I'll push the other patches once I know they'll work with the kernel. But I've got a slew of other things that I need to work on in the next few weeks, in no particular order:
I'm currently not planning to make all of Debian's amd64 images bootable using 32-bit UEFI like this image - I'm happy to leave this as just an option for our multi-arch i386/amd64 images (netinst or DVD only). I think that's a reasonable compromise here, and it's also the easiest thing for me to do with the current debian-cd build system.
Finally, apologies if you've asked me questions about the earlier images in this series and I've not responded yet. Fixing that ASAP!Tuesday, 06 January 2015
I promised to write about this a long time, ooops... :-)
Another ARM port in Debian - yay!
arm64 is officially a release architecture for Jessie, aka Debian version 8. That's taken a lot of manual porting and development effort over the last couple of years, and it's also taken a lot of CPU time - there are ~21,000 source packages in Debian Jessie! As is often the case for a brand new architecture like arm64 (or AArch64, to use ARM's own terminology), hardware can be really difficult to get hold of. In time this will cease to be an issue as hardware becomes more commoditised, but in Debian we really struggled to get hold of equipment for a very long time during the early part of the port.
First bring-up in Debian Ports
To start with, we could use ARM's own AArch64 software models to build the first few packages. This worked, but only very slowly. Then Chen Baozi and the folks running the Tianhe-2 supercomputer project in Guangzhou, China contacted us to offer access to some arm64 hardware, and this is what Wookey used for bootstrapping the new port in the unofficial Debian Ports archive. This has now become the normal way for new architectures to get into Debian. We got most of the archive built in debian-ports this way, and we could then use those results to seed the initial core set of packages in the main Debian archive.
Second bring-up - moving into the main Debian archive
By the time that first Debian bring-up was done, ARM was starting to produce its own "Juno" development boards, and with the help of my boss^4 James McNiven we managed to acquire a couple of those machines for use as official Debian build machines. The existing machines in China were faster, but for various reasons quite difficult to maintain as official Debian machines. So I set up the Junos as buildds just before going to DebConf in August 2014. They ran very well, and (for dev boards!) were very fast and stable. They built a large chunk of the Debian archive, but as the release freeze for Jessie grew close we weren't quite there. There was a small but persistent backlog of un-built packages that were causing us issues, plus the Juno machines are/were not quite suitable as porter boxes for Debian developers all over the world to use for debugging their packages on the new architecture.
More horsepower - Linaro machines
This is where Linaro came to our aid. Linaro's goal is to help improve Free and Open Source Software on ARM, and one of the more recent projects in Linaro is a cluster of servers that are made available for software developers to use to get early access to ARMv8 (arm64) hardware. It's a great way for people who are interested in this new architecture to try things out, port their software or indeed just help with the general porting effort.
As Debian is seen as such an important part of the FLOSS ecosystem, we managed to negotiate dedicated access to three of the machines in that cluster for Debian's use and we set those up in October, shortly before the freeze for Jessie. Andy Doan spent a lot of his time getting these machines going for us, and then I set up two of them as build machines and one as the porter box we were still needing.
With these extra machines available, we quickly caught up with the ever-busy "Needs-Build" queue and we've got sufficient build power now to keep things going for the Jessie release. We were officially added to the list of release architectures at the Cambridge mini-Debconf in November, and all is looking good now!
And in the future?
I've organised the loan of another arm64 machine from AMD for Debian to use for further porting and/or building. We're also expecting that more and more machines will be coming out soon as vendors move on from prototyping to producing real customer equipment. Once that's happened, more kit will be available and everybody will be able to have arm64-powered computers in the server room, on their desk and even inside their laptop! Mine will be running Debian Jessie... :-)
There's been a lot of people involved in the Debian arm64 bootstrapping at various stages, so many that I couldn't possibly credit them all! I'll highlight some, though. :-)
First of all, Wookey's life has revolved around this port for the last few years, tirelessly porting, fixing and hacking out package builds to get us going. We've had loads of help from other teams in Debian, particularly the massive patience of the DSA folks with getting early machines up and running and the prodding of the ftpmaster, buildd and release teams when we've been grinding our way through ever more package builds and dependency loops. We've also had really good support from toolchain folks in Debian and ARM, fixing bugs as we've found them by stressing new code and new machines. We've had a number of other people helping by filing bugs and posting patches to help us get things built and working. And (last but not least!) thanks to all the folks who've helped us beg and borrow the hardware to make the Debian arm64 port a reality.
Rumours of even more ARM ports coming soon are entirely scurrilous... *grin*