About
Steve's blog,
The Words of the Sledge
steve@einval.com

Subscribe
Subscribe to the RSS feed.

Links

Friends

Firmware vote result - the people have spoken!

It's time for another update on Debian's firmware GR. I wrote about the problem back in April and about the vote itself a few days back.

Voting closed last night and we have a result! This is unofficial so far - the official result will follow shortly when the Project Secretary sends a signed mail to confirm it. But that's normally just a formality at this point.

A Result!

The headline result is: Choice 5 / Proposal E won: Change SC for non-free firmware in installer, one installer. I'm happy with this - it's the option that I voted highest, after all. More importantly, however, it's a clear choice of direction, as I was hoping for. Of the 366 Debian Developers who voted, 289 of them voted this option above NOTA and 63 below, so it also meets the 3:1 super-majority requirement for amending a Debian Foundation Document (Constitution section 4.1.3).

So, what happens next?

We have quite a few things to do now, ideally before the freeze for Debian 12 (bookworm), due January 2023. This list of work items is almost definitely not complete, and Cyril and I are aiming to get together this week and do more detailed planning for the d-i pieces. Off the top of my head I can think of the following:

• Update the SC with the new text, update the website.
• Check/add support for the non-free-firmware section in various places:
• d-i build
• debian-cd
• debmirror (?)
• ftpsync (?)
• Any others?
• Uploads of firmware packages targeting non-free-firmware.
• Extra d-i code to inform users about what firmware blobs have been loaded and the matching non-free-firmware packages. Plus information about the hardware involved. Maybe a new d-i module / udeb for this? Exact details here still TBD. Probably the biggest individual piece of work here.
• Tweaks to add the non-free-firmware section in the apt-setup module if desired/needed.
• An extra boot option (a debconf variable) to disable loading extra firmware automatically, then exposed as an extra option through the isolinux and GRUB menus. d-i "expert mode" can also be used to tweak this behaviour later, except (obviously) for things like audio firmware that must be loaded early if they're going to be useful at all.
• Update the image build scripts to include the n-f-f packages, only build one type of image. I'll do my best to keep config and support around too for images without n-f-f included, to make it easier to still build those for people who still want them.
• Matching updates to docs, website, wiki etc.
• ...

If you think I've missed anything here, please let me and Cyril know - the best place would be the mailing list (debian-boot@lists.debian.org). If you'd like to help implement any of these changes, that would be lovely too!

15:15 :: # :: /debian/issues :: 0 comments

Firmware again - updates, how I'm voting and why!

Updates

Back in April I wrote about issues with how we handle firmware in Debian, and I also spoke about it at DebConf in July. Since then, we've started the General Resolution process - this led to a lot of discussion on the the debian-vote mailing list and we're now into the second week of the voting phase.

The discussion has caught the interest of a few news sites along the way:

• Debian to vote on its firmware path in Linux Weekly News
• Debian Begins A General Resolution To Decide What To Do With Non-Free Firmware in Phoronix
• Debian set to start vote on including non-free packages in install media in ITWire
• Why Debian Might Include Non-Free Firmware in Future Releases in MakeUseOf

My vote

I've also had several people ask me how I'm voting myself, as I started this GR in the first place. I'm happy to oblige! Here's my vote, sorted into preference order:

  [1] Choice 5: Change SC for non-free firmware in installer, one installer
  [2] Choice 1: Only one installer, including non-free firmware
  [3] Choice 6: Change SC for non-free firmware in installer, keep both installers
  [4] Choice 2: Recommend installer containing non-free firmware
  [5] Choice 3: Allow presenting non-free installers alongside the free one
  [6] Choice 7: None Of The Above
  [7] Choice 4: Installer with non-free software is not part of Debian

Why have I voted this way?

Fundamentally, my motivation for starting this vote was to ask the project for clear positive direction on a sensible way forward with non-free firmware support. Thus, I've voted all of the options that do that above NOTA. On those terms, I don't like Choice 4 here - IMHO it leaves us in the same unclear situation as before.

I'd be happy for us to update the Social Contract for clarity, and I know some people would be much more comfortable if we do that explicitly here. Choice 1 was my initial personal preference as we started the GR, but since then I've been convinced that also updating the SC would be a good idea, hence Choice 5.

I'd also rather have a single image / set of images produced, for the two reasons I've outlined before. It's less work for our images team to build and test all the options. But, much more importantly: I believe it's less likely to confuse new users.

I appreciate that not everybody agrees with me here, and this is part of the reason why we're voting!

Other Debian people have also blogged about their voting choices (Gunnar Wolf and Ian Jackson so far), and I thank them for sharing their reasoning too.

For the avoidance of doubt: my goal for this vote was simply to get a clear direction on how to proceed here. Although I proposed Choice 1 (Only one installer, including non-free firmware), I also seconded several of the other ballot options. Of course I will accept the will of the project when the result is announced - I'm not going to do anything silly like throw a tantrum or quit the project over this!

Finally

If you're a DD and you haven't voted already, please do so - this is an important choice for the Debian project.

18:46 :: # :: /debian/issues :: 0 comments

Firmware - what are we going to do about it?

TL;DR: firmware support in Debian sucks, and we need to change this. See the "My preference, and rationale" Section below.

In my opinion, the way we deal with (non-free) firmware in Debian is a mess, and this is hurting many of our users daily. For a long time we've been pretending that supporting and including (non-free) firmware on Debian systems is not necessary. We don't want to have to provide (non-free) firmware to our users, and in an ideal world we wouldn't need to. However, it's very clearly no longer a sensible path when trying to support lots of common current hardware.

Background - why has (non-free) firmware become an issue?

Firmware is the low-level software that's designed to make hardware devices work. Firmware is tightly coupled to the hardware, exposing its features, providing higher-level functionality and interfaces for other software to use. For a variety of reasons, it's typically not Free Software.

For Debian's purposes, we typically separate firmware from software by considering where the code executes (does it run on a separate processor? Is it visible to the host OS?) but it can be difficult to define a single reliable dividing line here. Consider the Intel/AMD CPU microcode packages, or the U-Boot firmware packages as examples.

In times past, all necessary firmware would normally be included directly in devices / expansion cards by their vendors. Over time, however, it has become more and more attractive (and therefore more common) for device manufacturers to not include complete firmware on all devices. Instead, some devices just embed a very simple set of firmware that allows for upload of a more complete firmware "blob" into memory. Device drivers are then expected to provide that blob during device initialisation.

There are a couple of key drivers for this change:

• Cost: it's typically cheaper to fit smaller flash memory (or no flash at all) onto a device. The cost difference may seem small in many cases, but reducing the bill of materials (BOM) even by a few cents can make a substantial difference to the economics of a product. For most vendors, they will have to implement device drivers anyway and it's not difficult to include firmware in that driver.

• Flexibility: it's much easier to change the behaviour of a device by simply changing to a different blob. This can potentially cover lots of different use cases:

  • separating deadlines for hardware and software in manufacturing (drivers and firmware can be written and shipped later);
  • bug fixes and security updates (e.g. CPU microcode changes);
  • changing configuration of a device for different users or products (e.g. potentially different firmware to enable different frequencies on a radio product);
  • changing fundamental device operation (e.g. switching between RAID and JBOD functionality on a disk controller).

Due to these reasons, more and more devices in a typical computer now need firmware to be uploaded at runtime for them to function correctly. This has grown:

• Going back 10 years or so, most computers only needed firmware uploads to make WiFi hardware work.

• A growing number of wired network adapters now demand firmware uploads. Some will work in a limited way but depend on extra firmware to allow advanced features like TCP segmentation offload (TSO). Others will refuse to work at all without a firmware upload.

• More and more graphics adapters now also want firmware uploads to provide any non-basic functions. A simple basic (S)VGA-compatible framebuffer is not enough for most users these days; modern desktops expect 3D acceleration, and a lot of current hardware will not provide that without extra firmware.

• Current generations of common Intel-based laptops also need firmware uploads to make audio work (see the firmware-sof-signed package).

At the beginning of this timeline, a typical Debian user would be able to use almost all of their computer's hardware without needing any firmware blobs. It might have been inconvenient to not be able to use the WiFi, but most laptops had wired ethernet anyway. The WiFi could always be enabled and configured after installation.

Today, a user with a new laptop from most vendors will struggle to use it at all with our firmware-free Debian installation media. Modern laptops normally don't come with wired ethernet now. There won't be any usable graphics on the laptop's screen. A visually-impaired user won't get any audio prompts. These experiences are not acceptable, by any measure. There are new computers still available for purchase today which don't need firmware to be uploaded, but they are growing less and less common.

Current state of firmware in Debian

For clarity: obviously not all devices need extra firmware uploading like this. There are many devices that depend on firmware for operation, but we never have to think about them in normal circumstances. The code is not likely to be Free Software, but it's not something that we in Debian must spend our time on as we're not distributing that code ourselves. Our problems come when our user needs extra firmware to make their computer work, and they need/expect us to provide it.

We have a small set of Free firmware binaries included in Debian main, and these are included on our installation and live media. This is great - we all love Free Software and this works.

However, there are many more firmware binaries that are not Free. If we are legally able to redistribute those binaries, we package them up and include them in the non-free section of the archive. As Free Software developers, we don't like providing or supporting non-free software for our users, but we acknowledge that it's sometimes a necessary thing for them. This tension is acknowledged in the Debian Free Software Guidelines.

This tension extends to our installation and live media. As non-free is officially not considered part of Debian, our official media cannot include anything from non-free. This has been a deliberate policy for many years. Instead, we have for some time been building a limited parallel set of "unofficial non-free" images which include non-free firmware. These non-free images are produced by the same software that we use for the official images, and by the same team.

There are a number of issues here that make developers and users unhappy:

1. Building, testing and publishing two sets of images takes more effort.
2. We don't really want to be providing non-free images at all, from a philosophy point of view. So we mainly promote and advertise the preferred official free images. That can be a cause of confusion for users. We do link to the non-free images in various places, but they're not so easy to find.
3. Using non-free installation media will cause more installations to use non-free software by default. That's not a great story for us, and we may end up with more of our users using non-free software and believing that it's all part of Debian.
4. A number of users and developers complain that we're wasting their time by publishing official images that are just not useful for a lot (a majority?) of users.

We should do better than this.

Options

The status quo is a mess, and I believe we can and should do things differently.

I see several possible options that the images team can choose from here. However, several of these options could undermine the principles of Debian. We don't want to make fundamental changes like that without the clear backing of the wider project. That's why I'm writing this...

1. Keep the existing setup. It's horrible, but maybe it's the best we can do? (I hope not!)

2. We could just stop providing the non-free unofficial images altogether. That's not really a promising route to follow - we'd be making it even harder for users to install our software. While ideologically pure, it's not going to advance the cause of Free Software.

3. We could stop pretending that the non-free images are unofficial, and maybe move them alongside the normal free images so they're published together. This would make them easier to find for people that need them, but is likely to cause users to question why we still make any images without firmware if they're otherwise identical.

4. The images team technically could simply include non-free into the official images, and add firmware packages to the input lists for those images. However, that would still leave us with problem 3 from above (non-free generally enabled on most installations).

5. We could split out the non-free firmware packages into a new non-free-firmware component in the archive, and allow a specific exception only to allow inclusion of those packages on our official media. We would then generate only one set of official media, including those non-free firmware packages.

   (We've already seen various suggestions in recent years to split up the non-free component of the archive like this, for example into non-free-firmware, non-free-doc, non-free-drivers, etc. Disagreement (bike-shedding?) about the split caused us to not make any progress on this. I believe this project should be picked up and completed. We don't have to make a perfect solution here immediately, just something that works well enough for our needs today. We can always tweak and improve the setup incrementally if that's needed.)

These are the most likely possible options, in my opinion. If you have a better suggestion, please let us know!

I'd like to take this set of options to a GR, and do it soon. I want to get a clear decision from the wider Debian project as to how to organise firmware and installation images. If we do end up changing how we do things, I want a clear mandate from the project to do that.

My preference, and rationale

Mainly, I want to see how the project as a whole feels here - this is a big issue that we're overdue solving.

What would I choose to do? My personal preference would be to go with option 5: split the non-free firmware into a special new component and include that on official media.

Does that make me a sellout? I don't think so. I've been passionately supporting and developing Free Software for more than half my life. My philosophy here has not changed. However, this is a complex and nuanced situation. I firmly believe that sharing software freedom with our users comes with a responsibility to also make our software useful. If users can't easily install and use Debian, that helps nobody.

By splitting things out here, we would enable users to install and use Debian on their hardware, without promoting/pushing higher-level non-free software in general. I think that's a reasonable compromise. This is simply a change to recognise that hardware requirements have moved on over the years.

Further work

If we do go with the changes in option 5, there are other things we could do here for better control of and information about non-free firmware:

1. Along with adding non-free firmware onto media, when the installer (or live image) runs, we should make it clear exactly which firmware packages have been used/installed to support detected hardware. We could link to docs about each, and maybe also to projects working on Free re-implementations.

2. Add an option at boot to explicitly disable the use of the non-free firmware packages, so that users can choose to avoid them.

Acknowledgements

Thanks to people who reviewed earlier versions of this document and/or made suggestions for improvement, in particular:

• Cyril Brulebois
• Matthew Garrett
• David Leggett
• Martin Michlmayr
• Andy Simpkins
• Neil Williams

01:24 :: # :: /debian/issues :: 44 comments

Project Harmony?

So, the "Harmony Project" launched their set of contributor agreements and tools last week. Colour me unimpressed...

There's a claim on their website that they are a "community-centered group", but I don't see any list of people and organisations who contributed to this work. That bothers me. Regarding their aim to "assist organisations which use contribution agreements", I don't think that there is anything of value here for the Free Software community at all. Free Software developers don't need contribution agreements, and in my opinion encouraging their use like this is only going to cause further splintering of the community. We've managed for a very long time without them, why start now?

As a developer, I personally don't believe in contribution agreements at all. If I contribute code to a project, it will be under the terms of a good Free Software license or not at all. That's all that's needed. There's a fair body of opinion out there on this - see pieces from Bradley Kuhn, Richard Fontana and Dave Neary for more discussion.

What do you think?

16:57 :: # :: /debian/issues :: 1 comment

vmware eats modifier keys

As much for my own reference as anything else...

I quite often use vmplayer on my work desktop, and occasionally on my laptop. Other options might work better for other people, but the company have already settled on using vmware on their Linux machines to give users access to Outbreak, Office etc. I try to avoid them as much as possible, but about the only effective option for calendaring at work is Outbreak. *spit*. And on my laptop, I just have a vmplayer setup containing a trivial Windows XP installation for things like Nokia firmware updates. It's more hassle than I can be arsed with to move to something else - the only thing worse than using Windows is installing it, in my experience.

In most respects, vmplayer is working OK for me. But: it has one intermittent and really annoying bug. Every now and again, as I switch away from the vmplayer session to another desktop using Ctrl-Alt, then Ctrl-Alt-Cursor, vmplayer eats the state of the X keyboard modifier keys. This makes things interesting after that point: my normal use of emacs, firefox, xterms, mutt (etc.) is fairly heavily dependent on being able to use the Shift and Ctrl keys.

This looks like it has been documented as a bug in a number of places over the past few months/years (1, 2, 3, 4, etc.) but there doesn't seem to be a fix coming. However, what I have found out is that there is a workaround: setxkbmap will reset the modifier key state when things have broken. If only I'd been able to find that earlier - I've ended up restarting X or rebooting in the past... :-(

19:01 :: # :: /debian/issues :: 1 comment

Why I Became a Debian Developer

I've been doing Debian for 10 years, as of this month. I reckon that's a good excuse for some ponderings...

For a long time in college in Cambridge, I was a Slackware user and supporter. It was my first distribution, installed in May 1994. I sent patches to Pat for bugs that I'd found, and I helped several of my friends in college maintain their machines running Slackware too. But after a couple of years of that, I grew tired of spending more time maintaining the OS on my machine than actually using it; I had even gone through the trials of the transition to ELF by hand. Some friendly pushing by my friend Jon Rabone (at the time also a DD) was finally enough, and one weekend in October 1996 we sat down together and started to install Debian on my PC.

That first installation was a nightmare! It was a major struggle, and more than one time that weekend I threatened to go back to the comfortable security of Slackware. The installer was awful, and needed lots of hand-holding. Eventually, however, we got there. hammer.chu.cam.ac.uk became a Debian machine, and most of it worked. By the end of that Sunday, I was convinced to stay with it.

Next, I decided that I wanted to contribute. The NM process in 1996 was quite simple - I mailed Bruce Perens with my PGP public key and asked him for an account. The package I wanted to maintain was mikmod. At the time I was one of the upstream developers, and I wanted to make sure it worked well in Debian. Unfortunately, I had already been beaten to it - Joey Hess had already packaged it in the few days since I first started work. Some things never change... :-) I mailed Joey and took over the package, then I started looking for other things to help with. I had long been annoyed that my Slackware patches had never met with any response, so Debian seemed ideal for me - a place where I could make a difference directly.

Over the following years, I took on, worked on and passed on lots of packages. At one point, I was maintaining lots of audio programs. At another I worked on lots of archiving and compression programs. Then I moved onto the debian-cd team. Debian was excellent fun: I got to use a great, stable operating system and I got to work on it and help make it better!

It hasn't all been plain sailing - there have been plenty of times when I've become disillusioned with things. There have been times when I've spent an entire night or weekend hacking on things and felt unhappy that either I hadn't made any progress or (even worse) my effort wasn't appreciated. There have been times when I've gone without sleep to get a package fixed or a CD build done, because I felt it mattered.

To offset that, there have been great moments: going along to Linux Expos and meeting users of my packages who wanted to say thanks and buy me beer; travelling around the world, meeting up with DDs and chewing the fat; playing Mao in the middle of the night at Debconf; most especially the feeling of achievement when my^Wour work is done and we manage to finish a release.

Oh, and that first installation? It's still around, almost ten years later. It has moved onto different hardware more times than I can remember, but I've continued upgrading it regularly ever since. It was once my only computer, but now it's one of many. hammer became sledge, then jack. It's now the machine that holds my Debian mirror and serves files to the rest of my home network. That's what I call upgradeability! It's a story that I tell people at Expos, and every year it gets more impressive... :-)

00:57 :: # :: /debian/issues :: 0 comments

Using votes to help detect MIA Debian developers?

Raphaël Hertzog writes about the Condorcet voting system that we use in Debian. He then goes on to consider whether we could use the system to help us elsewhere - could we track MIA developers by checking their voting records? This strikes a chord with another idea I've heard recently: make voting in the DPL election mandatory for active DDs.

I expect that even suggesting such a thing may appal some of us - forcing volunteer DDs to do anything is not likely to be popular, and I admit I have misgivings about it myself on that front. However, I can also see the attraction of the idea. The DPL election is a very(!) well publicised event that happens regularly every year, and is something that most DDs are likely to care about. Asking the project secretary after the polls close for a list of DDs that have not voted might be a good way of picking up on those who have stopped paying attention to Debian issues.

Comments? :-)

21:02 :: # :: /debian/issues :: 4 comments

Sarge dual-layer DVD builds, again

The dual-layer DVDs are looking less likely at this point - sarge is just too big. The maximum size available on a DL DVD is 8.5 billion bytes, but from a quick test run this morning I've got the following figures:

alpha: Image size 9100232704 bytes
arm:   Image size 8295522304 bytes
i386:  Image size 9308045312 bytes
ia64:  Image size 9641955328 bytes
m68k:  Image size 9787232256 bytes
src:   Image size 9703890944 bytes
...

We're only just going to fit onto 2 normal single-layer DVDs for i386 at this point, and some arches may yet take 3! I'm investigating sizes without the contrib section now, to see how that looks.

To be able to fit onto a DL DVD, we may have to trim some things...

14:00 :: # :: /debian/issues :: 0 comments

I never thought I'd see the day...

...when a kook would stand for election as Debian Project Leader. We've had joke candidates in the past, sure, but even in those cases I'd have been happy enough to see one of them elected. This is just about the first time I've had strong enough views about a candidate that I'd vote RON above them. I hope that enough of the Debian electorate are aware of Walther's views that he could never be elected, but it'll be interesting to see how many votes he actually gets. Even his own nomination mail contains one of his usual controversial sigs...!

00:36 :: # :: /debian/issues :: 0 comments

Do we want sarge to fit on a single DVD?

I've been talking to one of the UK Debian CD/DVD sellers about the sarge release. Woody sold well as a DVD set - binaries on one CD and source on another. Sarge, of course, is a lot bigger. I've been looking into the issues in creating DVDs. What would be really nice would be selling sarge on dual-layer DVDs in the same way. It would be slightly more expensive to produce, but end users installing sarge would have an easy time - boot a single disc and just hit go.

However, we have a problem. DVD sizes are complex:

• DVD5 is a single-sided, single-layer disc that holds 4.7 billion bytes (NOT 4.7GB)
• DVD10 is a double-sided, single-layer disc that holds 9.4 billion bytes (i.e. just two DVD5 discs glued back-to-back)
• DVD9 is a single-sided, double-layer disc that holds 8.5 billion bytes, 4.25 billion bytes per layer
• DVD18 is a double-sided, double-layer disc that holds 17 billion bytes (again, two DVD9s back-to-back)

(DVD18 is apparently very much a deprecated format. While there are no problems with drives reading them, manufacturing is a pig and there are reliability problems with the discs.)

With the above in mind, I'm hoping that we can fit sarge on a single DVD9 for binaries. Double-sided DVD10 discs are not ideal for installation; users will have to flip discs while installing. BUT: we have a problem. Fitting sarge onto a DVD9 is going to be awkward. The latest test images I've created don't fit for several of the architectures:

alpha/alpha-1 8471242752 bytes
arm/arm-1 7612305408 bytes
hppa/hppa-1 8148635648 bytes
i386/i386-1 8735232000 bytes
ia64/ia64-1 9016291328 bytes
m68k/m68k-1 8810422272 bytes
mips/mips-1 7869827072 bytes
mipsel/mipsel-1 7720431616 bytes
powerpc/powerpc-1 10134548480 bytes
powerpc/powerpc-2 355729408 bytes
s390/s390-1 8079792128 bytes
sparc/sparc-1 8143155200 bytes
src/src-1 9058893824 bytes

I'm not sure where to go from here. Powerpc is HUGE!; i386 doesn't fit either. Those are the 2 architectures I'm really targeting with this to start with. Maybe it's time to start excluding things from the DVD build. We've done that before when doing CD releases, but it's not something to do lightly.

23:14 :: # :: /debian/issues :: 4 comments

Large Packages and Sources files, revisited

I wrote a while ago about the problem I saw with large Packages and Sources files, and AJ jumped on what I'd written. I've been away for a while since and not had a chance to respond. Basically, I don't have a problem with what AJ's suggesting - I hadn't seen the pdiff idea before so thanks for the pointer. I prefer the timestamped and sorted Packages file personally (compared to a potentially large number of extra diff files on the mirror), but I don't have a great attachment to the idea - there's more than one way to do this and so long as something is done then I'll be happy.

One thing that's not clear from AJ's description of pdiffs is how a client should work out which diffs it needs. Timestamps could be unreliable unless the client and ftpmaster agree on times. But Daniel mentioned earlier that simply MD5ing the client's existing Packages file and having it ask for diff.<MD5> from the server should do the job. Fine, I see how that works.

So, next thing to do is have a look at the archive scripts to make this work...

18:10 :: # :: /debian/issues :: 1 comment

Bandwidth problems with large Packages and Sources files

I've heard several complaints that modem/slow connection users struggle to keep up - even tracking security updates can take a while. When there is a security update, the client machine will have to download the entire Packages file each time. This problem will get worse as time goes on and the number of packages grows. And for people tracking testing or unstable over a modem (such people do exist!), it already takes ages for them to just sync Packages files, let alone actually downloading and installing the new packages they want.

How do other people do this?

Microsoft have a central pool of servers for windows updates which keeps a database of updates. Each client machine connects to an update server and checks for any updates that have not yet been installed on the client. This works for Microsoft, but they have to maintain this huge central server pool which will be hammered solidly, constantly from the millions of client machines scattered across the world.

We could do something similar too. We'd need to write a new application/server/cgi/something to run on security.debian.org and modify apt and friends to talk to that program. This could be done, but it's reinventing the wheel. We ask people not to mirror the security site, but it happens anyway; people would not be able to use the mirrors for this service unless those mirrors have the same program installed. That's a problem.

If we want the mirrors to be able to work, we need to push out the information in a standard form (files/directories) that will propagate easily to mirrors via existing channels: HTTP/FTP/rsync/whatever. We need to keep some state over time so that client machines can compare timestamps on the information they already have and then only retrieve the changes to get them up to current state. If the client does not have any state, or if its idea of state is too old, then this should be quickly recognised and the client should download all current state; we don't want to slow these users down any more.

Various people have discussed ways to do this in the past. Suggestions have included providing periodic (e.g. daily) diffs of the Packages files that clients can download. These have never really taken off.

There is a much simpler solution to the problem, found after some discussion at the UKUUG Conference 2004. Apt and dpkg already cope with Packages/Sources stanzas containing extra fields that they do not understand; they simply ignore the extra fields. Equally, they do not care about the sort order of the files.

My proposed way to solve the problem is:

• add a Timestamp: field to each entry (a simple time_t would be easy) for the date/time that that entry was first added to the file
• sort the entries in the Packages and Sources files on that timestamp, with the most recent first

This way, clients can simply download new versions of these files and stop once a timestamp is older than the most recent timestamp of the last version they downloaded. If they do not have an older version or their old version is ancient, they will just end up downloading the entire file this time. The client program doing the download can then merge the old file version with the new information. It would even be possible to create a normal standard-format Packages/Sources file if that is still wanted.

One issue that this does not cope with is removed packages and sources. There is an easy way to do that too: add a new small stanza for a binary/source package with a new Removed-time: field. When the client sees this stanza, it will know to remove older information about that package/source from its merged output.

Creating the new Packages/Sources files should (I hope) be easy; in the main archive, Katie already uses a database backend when processing packages so dumping timestamps should take little effort.

Comments? I'm sure I must have missed something here, but I can't see any holes...

Thanks to Phil, Wookey and Martin for ideas and discussion.

11:17 :: # :: /debian/issues :: 3 comments

Priceless...

Andrew Suffield in debian-legal today:

"Right, there's at least two or three of you running around and trying to undermine the project. Cut it out. This idiotic attempt to create discord is not productive; it's somewhere between trolling and deliberate sabotage."

Sigh...

10:48 :: # :: /debian/issues :: 0 comments

"I think we have a troll infestation"

*giggle* I shouldn't let Suffield wind me up, but this is just hilarious.

I'm sometimes not sure what to think about Suffield - does he actually believe what he writes...?

12:07 :: # :: /debian/issues :: 2 comments

Problem with debian-legal?

There seems to have been a falling-out between the contributors to debian-legal and the rest of the project recently. Why? It's a common opinion that Debian people are, by definition, not normal; Bruce Perens complained that trying to lead Debian was like herding cats. We have a large concentration of intelligent, opinionated people in the project, so it's quite amazing at times that we don't have more disagreements. Having developers maintain their own packages without so much discussion most of the time must help here...

So, why the spat? Debian-legal has attracted a hard core of posters who have worked hard in researching and critiquing a huge variety of different software licenses, ascertaining their merits and (mainly, most importantly) whether or not they can be classified as free under the terms of the DFSG. Unfortunately, the group has become more and more hostile to outsiders (including other developers) trying to join in the discussions. Things have now deteriorated to the point where other developers just don't get involved any more.

The latest discussion about the MPL is a prime example of this. The latest opinion seems to be that the MPL is now non-free, despite it clearly being treated as a Free license in the past. Nothing has changed in the MPL or the DFSG in the intervening period, but suddenly it's now non-free. Some of the points raised against it seem entirely arbitrary (choice of legal venue, patent license limits) and don't seem to be justified by the rules of the DFSG. That's my opinion, anyway. But I've got better things to do than get dragged into a flame war on debian-legal over a license. People have tried to honestly discuss license issues recently and have been flamed by supercilious, holier-than-thou "armchair lawyers" on a regular basis.

That's why people are giving up on debian-legal...

14:44 :: # :: /debian/issues :: 0 comments