Thursday, 23 April 2015
I'm happy with the progress we've made for debian-installer and related packages for the Jessie release. We're going to end up with a release that's better in a number of ways than what we've had before.
1. Big EFI enhancements
I've already blogged a lot about the stuff I've worked on here, so I'll just summarise for now some of the improvements we've got over Wheezy.
2. Openstack images
In collaboration with Thomas Goirand, we now have amd64 Openstack Jessie image builds being produced every week, and there will be an official image made to go with the Jessie release too. See http://cdimage.debian.org/cdimage/openstack/testing/ for the current image.
3. Debian-live images
As of a few weeks ago, we've also added started doing weekly builds of live Debian images for amd64 and i386, using software and configuration from the Live Systems Project. See http://cdimage.debian.org/cdimage/weekly-live-builds/ for the current weekly images. These will be produced in sync with the Jessie release too.
4. New architectures
We've added installation media for the two new architectures added
I'm particularly proud of the arm64 images. With help from Ian
Campbell, Leif Lindholm and Thomas Schmitt I've managed to make
EFI-compatible CD images in an
Hopefully post-Jessie we'll even be able to start providing live images and openstack images for more architectures too.
More help needed yet!
First of all, we're planning to release Jessie as Debian 8 this coming Saturday (25th April). Help with testing the installation and live images as they're produced would be lovely - please join us on the #debian-cd channel on irc.debian.org and we'll co-ordinate there.
Secondly, there's an almost endless variety of machines out there. I've updated information about how Debian installation works on some of the more awkward Mac Mini machines, but we don't yet cover all the bases even there. It would be great to update the information about other machines such as the Macbook range as well - currently a lot of these pages are well out of date and won't be helpful for new users. Please test on machines if you have them, and help improve Debian's documentation here.Monday, 30 March 2015
One final update on my work for UEFI improvements in Jessie!
All of my improvements have been committed into the various Debian packages involved, and the latest release candidate for Jessie's debian-installer build (RC2) works just as well as my test builds on the Bay Trail system I've been using (Asus X205TA). Job done! :-)
I'm still hoping to maybe get more hardware support for this particular hardware included in Jessie, but I can't promise. The mixed EFI work has also improved things for a lot of Mac users, and I'm planning to write up a more comprehensive list of supported machines in the Debian wiki (for now).
There's now no need to use any of the older test installer images - please switch to RC2 for now. See http://cdimage.debian.org/cdimage/jessie_di_rc2/ for the images. If you want to install a 64-bit system with the 32-bit UEFI support, make sure you use the multi-arch amd64/i386 netinst or DVD. Otherwise, any of the standard i386 images should work for a 32-bit only system.
My kernel patch to add the new /sys file was accepted upstream a while back, and has been in Linus' master branch for some time. It'll be in 4.0 unless something goes horribly wrong, and as it's such a tiny piece of code it's trivial to backport to anything remotely recent too.
I've also just seen that my patch for grub2 to use this new /sys file has been accepted upstream this week. Again, the change is small and self-contained so should be easy to copy across into other trees too.
Mixed EFI systems should now have better support across all distros in the near future, I hope.Sunday, 11 January 2015
Time for another update on my work for UEFI improvements in Jessie!
I've spent more time on the integration of 32-bit grub-efi with a 64-bit Debian system, and just published a new test image on pettersson. I've added:
These remove the manual steps that were necessary for a 64-bit installation with the previous build. I've just used this exact image (and a network mirror) to install a fully-functional 64-bit Gnome system on the X205TA, simply by selecting "64-bit install" from the GRUB menu and following prompts. Yay! Visit http://cdimage.debian.org/cdimage/unofficial/efi-development/jessie-upload3/ to download and test the image.
Now, there's no guarantee that the kernel patch I've submitted to the linux-efi folks will be accepted in its current form, and even if it is I'll have to get it and the other code I've written accepted into the various packages and then into Jessie! But for now this image should work just fine for Bay Trail folks I hope!
WARNING: this CD is provided for testing only. Use at your own risk! If you have appropriate (U)EFI hardware, please try this image and let me know how you get on, via the debian-cd and debian-boot mailing lists.
For now, I'm going to pause development here. The core code I'm using to make these images is all in the debian-cd and d-i repos, and I'll push the other patches once I know they'll work with the kernel. But I've got a slew of other things that I need to work on in the next few weeks, in no particular order:
I'm currently not planning to make all of Debian's amd64 images bootable using 32-bit UEFI like this image - I'm happy to leave this as just an option for our multi-arch i386/amd64 images (netinst or DVD only). I think that's a reasonable compromise here, and it's also the easiest thing for me to do with the current debian-cd build system.
Finally, apologies if you've asked me questions about the earlier images in this series and I've not responded yet. Fixing that ASAP!Tuesday, 06 January 2015
Time for another update on my work for UEFI improvements in Jessie!
I now have a mixed 32- and 64-bit UEFI netinst up and running right now, which will boot and install on the Asus X205TA machine I have. Since the last build, I've added 64-bit (amd64) support and added CONFIG_EFI_MIXED in the kernel so that the 64-bit kernel will also work with a 32-bit UEFI firmware. Visit http://cdimage.debian.org/cdimage/unofficial/efi-development/jessie-upload2/ to download and test the image. There are a few other missing pieces yet for a complete solution, but I'm getting there...!
WARNING: this CD is provided for testing only. Use at your own risk! If you have appropriate (U)EFI hardware, please try this image and let me know how you get on, via the debian-cd and debian-boot mailing lists.Friday, 02 January 2015
Time for another update on my work for UEFI improvements in Jessie!
I've got an i386-only UEFI netinst up and running right now, which will boot and install on the Asus X205TA machine I have. I've got the required i2c modules included in this build so that the installer can use the keyboard and trackpad on the machine - useful...! See #772578 for more details about that. Visit http://cdimage.debian.org/cdimage/unofficial/efi-development/jessie-upload1/ to download and test the image. There are a few other missing pieces of hardware support for this machine yet, but the basics are there. See the wiki for more information.
My initial i386 test CD is not yet going to do an amd64 installation for you, but it should let you get going with Debian on these machines! I'm going to continue working on that 32-64 support next.
WARNING: this CD is provided for testing only. Use at your own risk! If you try this on an early Intel-based Mac, it will not work. Otherwise, this should likely work for most folks using 32-bit x86 hardware just like any other Debian Jessie daily netinst build.
If you have appropriate (U)EFI hardware, please try this image and let me know how you get on, via the debian-cd and debian-boot mailing lists.Sunday, 21 December 2014
I spoke about adding support for installing grub-efi into the removable media path (#746662). That went into Debian's grub packages already, but there were a couple of bugs. First of all, the code could end up prompting people about EFI questions even when they didn't have any grub-efi packages installed. Doh! (#773004). Then there was an unexpected bug with case-insensitive file name handling on FAT/VFAT filesystems (#773092). I've posted (and tested!) patches to fix both, hopefully in an upload any day now.
Next, I mentioned getting i386 UEFI support going again. This is a major feature that a lot of people have been asking for. It's also going to involve quite a bit of effort...
Our existing (amd64) UEFI-capable images in Debian use the standard x86 El Torito CD boot method, with two boot images provided. One of these images gives us the traditional isolinux BIOS-boot support. The second option is an alternate El Torito image, including at a 64-bit version of grub-efi. For most machines, this works just fine - the BIOS or UEFI firmware will automatically pick the correct image and everybody's happy. This even works on our multi-arch i386/amd64 CDs and DVDs - isolinux will boot either kernel from the first El Torito image, or the alternate UEFI image is amd64 only.
However, I can now see that there's been a long-standing issue with those multi-arch images, and it's to do with Macs. On the advice of Matthew Garrett, I've borrowed an old 32-bit Intel Mac to help testing, and it's quite instructive in terms of buggy firmware! The firmware on older 32-bit Intel Macs crashes hard when it detects more than one El Torito boot image, and I've now seen this happen myself. I've not had any bug reports about this, so I can only assume that we haven't had many users try that image. As far as I can tell, they've been using the normal i386 images in BIOS boot mode, and then struggling to get bootloaders working afterwards. There are a number of different posts on the net explaining how to do that. That's OK, but...
If I now start adding 32-bit UEFI support to our standard set of i386 images, this will prevent users of old Macs from installing Debian. I could just say "screw it" and decide to not support those users at all, but that's not a very nice thing to do. If we want to continue to support them and add 32-bit UEFI support, I'll have to add another flavour of i386 image, either a "Mac special" or a "32-bit UEFI special". I'm not keen on doing that if I could avoid it, but the two options are mutually exclusive. Given the Mac problem is only on older hardware which (hopefully!) will be dying out, I'll probably pick that one as the special-case CD, and I'll make an extra netinst flavour only for those users to boot off.
So, I've started playing with i386 UEFI stuff in the last couple of weeks too. I almost immediately found some showstopper behaviour bugs in the i386 versions of efivar and efibootmgr (#773412 and #773007), but I've debugged these with our Debian maintainer (Jared Dominguez) and the upstream developer (Peter Jones) and fixes should be in Jessie very soon.
As I mentioned last month, the machines that most people have been requesting support for are the latest Bay Trail-based laptops and tablets. There are using 64-bit Intel Atom CPUs, but crippled with 32-bit UEFI firmware with no BIOS compatibility mode. This makes for some interesting issues. It's probably impossible to get a true answer why these machines are so broken by design, but there are several rumours. As far as I can see, most of these machines seem to ship with a limited version of 32-bit Windows 8.1. 32-bit Windows is smaller than 64-bit Windows, so fits much better in limited on-board storage space. But 32-bit Windows won't boot from 64-bit UEFI, so the firmware needed buggering to match. Ugh!
To support these Bay Trail machines properly, we'll want to add a 32-bit UEFI installation option to our 64-bit images. I can tweak our CDs so that both 32-bit and 64-bit versions of grub-efi are included, and the on-board UEFI will load the right one needed. Then I'll need to make sure that all the 64-bit images also include grub-efi-ia32-bin from now on. With some extra logic, we'll need to remember that these new machines need that package installing instead of grub-efi-amd64-bin. It shouldn't be too hard, but let's see! :-)
So, I've been out and bought one of these machines, an Asus X205TA. Lucas agreed that Debian will reimburse me (thanks!), so I'm not stuck with spending my own money on an otherwise unwanted machine! I can see via Google that none of the mainstream Linux distros support the Bay Trail machines fully yet, so there's not a lot of documentation yet. Initial boot on the new machine was easy using a quick-hack i386 UEFI image on USB, but from there everything went downhill quickly. I'll need to investigate some more, but the laptop's own keyboard and trackpad are not detected by the installer system. Neither is its built-in WiFi. Yay! I had to go and dig out a USB hub to connect the installer image USB key, a keyboard, mouse and a USB hard drive to the machine, as it only has 2 USB ports. I've taken a complete backup of the on-board 32GB flash before I start experimenting, so I can restore the machine back to its virgin state for future testing.
I guess I now have a project to keep me busy over Christmas...!
In other news, we've been continuing work on UEFI support for and within the new arm64 port. My ARM/Linaro colleague Leif Lindholm has been back-porting upstream kernel features and bug fixes to make d-i work, and filing Debian bugs when silly things break on arm64 because people don't think about other architectures (e.g #773311, doh!). As there are more and more people interested in (U)EFI support these days, I've also proposed that we create a new debian-efi mailing list to help focus discussion. See #773327 and follow up there if you think you'd use the list too!
You can help! Same as 2 years ago, I'll need help testing some of these images. For the 32-bit UEFI support, I now have some relevant hardware myself, but testing on other machines too will be very important! I'll start pushing unofficial Jessie EFI test images shortly - watch this space.Thursday, 20 November 2014
So, my work for Wheezy gave us working amd64 UEFI installer images. Yay! Except: there were a few bugs that remained, and also places where we could deal better with some of the more crappy UEFI implementations out there. But, things have improve since then and we should be better for Jessie in quite a few ways.
First of all, Colin and the other Grub developers have continued working hard and quite a lot of the old bugs in this area look to be fixed. I'm hoping we're not going to see so many "UEFI boot gives me a blank black screen" type of problems now.
For those poor unfortunates with Windows 7 on their machines, using BIOS boot despite having UEFI support in their hardware, I've fixed a long-standing bug (#763127) that could leave people with broken systems, unable to dual boot.
We've fixed a silly potential permissions bug in how the EFI System Partition is mounted: (#770033).
Next up, I'm hoping to add a workaround for some of the broken UEFI implementations, by adding support in our Grub packages (and in d-i) for forcing the installation of a copy of grub-efi in the removable media path. See #746662 for more of the details. It's horrid to be doing this, but it's just about the best thing we can do to support people with broken firmware.
Finally, I've been getting lots of requests for adding i386 (32-bit x86) UEFI support in our official images. Back in the Wheezy development cycle, I had test images that worked on i386, but decided not to push that support into the release. There were worries about potentially critical bugs that could be tickled on some hardware, plus there were only very few known i386 UEFI platforms at the time; the risk of damage outweighed the small proportion of users, IMHO. However, I'm now revisiting that decision. The potentially broken machines are now 2 years older, and so less likely to be in use. Also, Intel have released some horrid platform concoction around the Bay Trail CPU: a 64-bit CPU (that really wants a 64-bit kernel), but running a 32-bit UEFI firmware with no BIOS Compatibility Mode. Recent kernels are able to cope with this mess, but at the moment there is no sensible way to install Debian on such a machine. I'm hoping to fix that next (#768461). It's going to be awkward again, needing changes in several places too.
You can help! Same as 2 years ago, I'll need help testing some of these images. Particularly for the 32-bit UEFI support, I currently have no relevant hardware myself. That's not going to make it easy... :-/
I'll start pushing unofficial Jessie EFI test images shortly.Monday, 03 September 2012
Continuing on from my third EFI progress report...
I've added some i386 EFI support in debian-cd and built new images. Alongside a new amd64 image, we now have an i386 image and a mixed image which should hopefully boot and work on both 32-bit and 64-bit machines. The i386 support is more experimental at this point, and it's more difficult for me to test due to a lack of physical hardware. If you've got an older 32-bit EFI machine, this might work for you...
Grab the images from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload4/ if you'd like to help test. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I've posted my full set of d-i patches to the debian-boot list for review now, and hopefully we'll be able to get those changes merged in after the Wheezy d-i beta 2 build is done. Full speed ahead for EFI in beta 3!Friday, 24 August 2012
Continuing on from my second EFI progress report...
My second alpha release of a Wheezy netinst CD for amd64 EFI seemed to work OK for me and a few others, but it wasn't 100% good, with (at least) two issues:
In the couple of days since then, I've fixed both of these issues. Yay! Hybrid boot was easier than I expected - I've just added a tiny FAT partition to the image containing the grub EFI bootloader and that seems to work fine. I also found that I made a mistake in the ordering of the El Torito boot records in the last image. BIOS boot seems to be limited to just the first image available on the machines I've tested with, wherease EFI will happily work as a secondary image. I've swapped them around in my code, and things look much better now. So, time for another CD alpha release for testing.
Grab the image from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload3/ if you'd like to help test it. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I've posted my full set of d-i patches to the debian-boot list for review now, and hopefully we'll be able to get those changes merged in after the Wheezy d-i beta 2 build is done. Full speed ahead for EFI in beta 3!Wednesday, 22 August 2012
Continuing on from my first EFI progress report...
My first alpha release of a Wheezy netinst CD for amd64 EFI worked OK for me and a few others, with reasonable feedback. But it was far from complete, with (at least) three outstanding issues:
In the days since then, I've carried on hacking on things. I've not yet sorted out hybrid boot, but I've fixed the other issues. To make graphics work on the grub boot menu was a simple case of loading the right module from grub.cfg ("insmod png"). Getting grub-efi to work took quite a lot more effort, though. I've been looking through Ubuntu's version of debian-installer code and merged a fair amount of their code into my local builds, adding tweaks as necessary. Result? I've got another CD alpha release ready for testing. Yay!
Grab the image from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload2/ if you'd like to help test it. Please do, and let me know via debian-boot/debian-cd how you get on.
As previously, the "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. My debian-cd changes are still in my branch but are just about ready for merging I think. I'll post more d-i patches to the debian-boot list for review shortly.Sunday, 12 August 2012
As promised last month at DebConf, I've been working on adding EFI support to debian-cd and debian-installer, hopefully to get it up and running. At the moment, this is basic support only for booting and setting up the installed system via UEFI. I'm not worrying about "Secure" (better named "Restricted"?) Boot yet - that comes later, and depends on the basic stuff working first.
So, how is it going? Quite well, really. I've got a
branch that I'm working on locally, and with a relatively small
amount of work there I've got a locally-built CD that boots in EFI
mode, using grub-efi as a boot loader. I've borrowed a script
Next step: the installer. At the moment, generic amd64 and i386 machines are assumed to use need dos-style partitioning, and then either lilo or grub as a bootloader. I've patched and rebuilt some of the d-i packages locally to add a new sub-architecture of "efi" for both amd64 and i386, alongside the existing "generic" and "mac" variants. Using that new sub-architecture, it's possible to add checks for amd/efi and i386/efi in various places to make the necessary changes:
Why elilo? It's the default for current ia64 systems, and for EFI Macs. I tried to get grub-efi working with the grub-installer package, but with little success so far. I'll try again with grub-efi, but for now elilo is much simpler and (importantly!) it works for me.
So, I have a netinst CD built that works for me in local testing in a qemu VM using James Bottomley's OVMF build of Tianocore. It's not pretty, but it works fine, covering all the normal Debian installation steps and giving me a UEFI system at the end of the process. I'm about to start testing on real hardware now, so I think it's worth sharing this netinst image with others too. Grab it from http://cdimage.debian.org/cdimage/unofficial/efi-development/upload1/ . The "bits" subdirectory contains all the tweaked d-i packages I've played with, in both source and binary form. If you have an EFI system and would like to help us get it supported well in Debian, please download this image and play with it! Feedback to the debian-cd and/or debian-boot lists, please!Monday, 25 July 2011
For several years, Kenshi Muto has been doing some awesome work on updated/backported Debian CDs, producing CDs with support for newer hardware and newer features. I've used them myself in the past to help get awkward machines working, and I know they are a great resource for lots of other Debian users.
We've discussed things in the last few weeks, and agreed that it would be useful to host his images on cdimage.debian.org. The best place to look now is http://cdimage.debian.org/cdimage/unofficial/backports/. These are unofficial images, so please don't report bugs in the Debian BTS for them.Friday, 07 January 2011
New feature for Debian CDs
We've had a wishlist bug (#551951) open against debian-cd for some time, asking for the creation of "isohybrid" CD images for i386 and amd64. These are special in that as well as the normal CD-based ISO9660 filesystem they also contain a valid-looking DOS-style partition table. Thus, if you simply copy one of these images raw to a USB stick a normal PC BIOS will boot the image directly. This would be a neat feature, making it much easier for people to use standard Debian installer images on their USB sticks without having to follow a lot more instructions.
Support for this feature was added in syslinux/isolinux quite a
while back (thanks to H Peter Anvin and the other syslinux
contributors), but this depended on post-processing images with
It's been a delight working with Thomas and George - they're enthusiastic, helpful and friendly. Thanks, guys!
What does this mean for end users?
Instead of having to specially prepare USB sticks for the
installer, you can now simply use
# dd if=debian-testing-i386-netinst.iso of=/dev/sdXThe USB stick should now boot directly, showing exactly the same bootup screen as if you'd written the image directly to a CD. Yay!
I'm expecting to be using this code from now on for new releases, i.e. d-i beta and RC releases and for Squeeze itself when we get there. I'm not planning on backporting this code into the Lenny branch of debian-cd as it's a lot of work and I don't want to destabilise things there.
We need to clear up some of the maintenance around the code as it stands; I'm planning on moving libjte inside my existing jigit package, as that's a sensible place to put it. That will come after the Squeeze release, though.
We need to finalise boot support for other architectures in xorriso; I added support for them into mkisofs/genisoimage a long time ago and we've already moved some of that into xorriso.
Thomas and George have also been working hard to give me faster/better size estimation code, useful for making debian-cd run faster.
Finally: I'd like to add support into xorriso for creating the nasty HFS hybrid images that are needed for booting Macs. The code that does this in cdrkit is probably some of the worst that I've ever worked with, and I'd like to get away from it. If only Apple hadn't stupidly built their proprietary platform around this shit and had used open standards instead. :-(
Once we have these bits in xorriso, I'll be able to move production of all of the CDs across from genisoimage to xorriso. Very nearly there for being able to drop the last remnants of Schily-ware from the Debian archive...Wednesday, 15 December 2010
We've just posted a news item about the removal of non-free firmware from Debian's Linux kernel packages. Kudos to the kernel team and others for the work involved to make this happen!
While we would be much happier if these binary blobs were not necessary at all, it's clear that some of our users may depend on them to support the hardware they are using. So, as mentioned in that news item, we've started making some unofficial CD images available that include the non-free firmware packages that we can redistribute. If you need one of these CDs, daily builds are to be found at http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/ . Starting with the next debian-installer release (hopefully RC1, coming soon!), we'll be generating fixed versions of these images to match each release too.
Be aware that these are only netinstall images, equivalent to the normal free netinst images that we normally produce but with the firmware files added. If you want to do an installation entirely from a set of CDs/DVDs/BDs then you will need to burn one of these in addition to that set, then feed the rest of the set when prompted to by the installer.Friday, 20 August 2010
Thanks to the sterling work of aurel32, debian-cd now supports creating CDs for the kfreebsd-amd64 and kfreebsd-i386 ports. For now, only the daily "sid" builds will be available - see http://cdimage.debian.org/cdimage/daily-builds/sid_d-i/arch-latest/. Once there's been another alpha/beta release of d-i then we'll get squeeze daily builds and weekly builds too.Wednesday, 16 April 2008
Forgot to mention, but some people seem to have spotted anyway. The last (ever!) official set of Sarge CDs and DVDs is available. I'd strongly advise against most people using them, as Etch is a much better bet and Lenny isn't too far away. But for those people that might want them, you can find them in the "archive" area: http://cdimage.debian.org/cdimage/archive/3.1_r8/Friday, 21 April 2006
In the expected place shortly.