About
Steve's blog,
The Words of the Sledge
steve@einval.com

Subscribe
Subscribe to the RSS feed.

Links

Friends

Linaro VLANd v0.4

VLANd is a python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.

There's more information in the README file. I've just released v0.4, with a lot of changes included since the last release:

• Large numbers of bugfixes and code cleanups
• Code changes for integration with LAVA:
  • Added db.find_lowest_unused_vlan_tag()
  • create_vlan() with a tag of -1 will find and allocate the first unused tag automatically
• Add port numbers as well as names to the ports database, to give human-recognisable references. See README.port-numbering for more details.
• Add tracking of trunks, the inter-switch connections, needed for visualisation diagrams.
• Add a simple http-based visualisation feature:
  • Generate network diagrams on-demand based on the information in the VLANd database, colour-coded to show port configuration
  • Generate a simple website to reference those diagrams.
• Allow more ports to be seen on Catalyst switches
• Add a systemd service file for vland

VLANd is Free Software, released under the GPL version 2 (or any later version). For now, grab it from git; tarballs will be coming shortly.

01:44 :: # :: /linaro :: 0 comments

Linaro VLANd v0.3

VLANd is a python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.

There's more information in the README file. I've just released v0.3, with a lot of changes included since the last release:

• Massive numbers of bugfixes and code cleanups
• Added two new switch drivers:
  • TP-Link TL-SG2XXX family (TPLinkTLSG2XXX)
  • Netgear XSM family (NetgearXSM)
• Added "debug" option to all the switch drivers to log all interactions
• Added internal caching of port modes within the driver core for a large speed-up in normal use
• Bug fix to handling of trunk ports in the CiscoCatalyst driver, improving VLAN interop with other switches
• Huge changes to the test lab, now using 5 switches and 10 hosts
• Big improvements to the test suite:
  • Match the new test lab layout
  • Move more of the core test code into the test-common utility library
  • Massively improved the check-networks test runner for the test hosts
  • Added parsing of the UP/DOWN results in test-common to give a simple PASS/FAIL result for each test
  • Added more tests
• All logging now in UTC

VLANd is Free Software, released under the GPL version 2 (or any later version). For now, grab it from git; tarballs will be coming shortly.

17:04 :: # :: /linaro :: 0 comments

Linaro VLANd v0.2

I've been working on this for too long without really talking about it, so let's fix that now!

VLANd is a simple (hah!) python program intended to make it easy to manage port-based VLAN setups across multiple switches in a network. It is designed to be vendor-agnostic, with a clean pluggable driver API to allow for a wide range of different switches to be controlled together.

There's more information in the README file. I've just released v0.2, with a lot of changes included since the last release:

• Massive numbers of bugfixes and code cleanups
• Improve how we talk to the Cisco switches - disable paging on long output
• Switch from "print" to "logging.foo" for messages, and add logfile support
• Improved test suite coverage, and added core test scripts for the lab environment

I've demonstrated this code today in Hong Kong at the Linaro Connect event, and now I'm going on vacation for 4 weeks. Australia here I come! :-)

07:53 :: # :: /linaro :: 2 comments

Yet another reason to not use Windows on your embedded devices...

Seen on an ATM in Hong Kong airport.

16:23 :: # :: /linaro :: 0 comments

Bootstrapping arm64 in Debian

I promised to write about this a long time, ooops... :-)

Another ARM port in Debian - yay!

arm64 is officially a release architecture for Jessie, aka Debian version 8. That's taken a lot of manual porting and development effort over the last couple of years, and it's also taken a lot of CPU time - there are ~21,000 source packages in Debian Jessie! As is often the case for a brand new architecture like arm64 (or AArch64, to use ARM's own terminology), hardware can be really difficult to get hold of. In time this will cease to be an issue as hardware becomes more commoditised, but in Debian we really struggled to get hold of equipment for a very long time during the early part of the port.

First bring-up in Debian Ports

To start with, we could use ARM's own AArch64 software models to build the first few packages. This worked, but only very slowly. Then Chen Baozi and the folks running the Tianhe-2 supercomputer project in Guangzhou, China contacted us to offer access to some arm64 hardware, and this is what Wookey used for bootstrapping the new port in the unofficial Debian Ports archive. This has now become the normal way for new architectures to get into Debian. We got most of the archive built in debian-ports this way, and we could then use those results to seed the initial core set of packages in the main Debian archive.

Second bring-up - moving into the main Debian archive

By the time that first Debian bring-up was done, ARM was starting to produce its own "Juno" development boards, and with the help of my boss^4 James McNiven we managed to acquire a couple of those machines for use as official Debian build machines. The existing machines in China were faster, but for various reasons quite difficult to maintain as official Debian machines. So I set up the Junos as buildds just before going to DebConf in August 2014. They ran very well, and (for dev boards!) were very fast and stable. They built a large chunk of the Debian archive, but as the release freeze for Jessie grew close we weren't quite there. There was a small but persistent backlog of un-built packages that were causing us issues, plus the Juno machines are/were not quite suitable as porter boxes for Debian developers all over the world to use for debugging their packages on the new architecture.

More horsepower - Linaro machines

This is where Linaro came to our aid. Linaro's goal is to help improve Free and Open Source Software on ARM, and one of the more recent projects in Linaro is a cluster of servers that are made available for software developers to use to get early access to ARMv8 (arm64) hardware. It's a great way for people who are interested in this new architecture to try things out, port their software or indeed just help with the general porting effort.

As Debian is seen as such an important part of the FLOSS ecosystem, we managed to negotiate dedicated access to three of the machines in that cluster for Debian's use and we set those up in October, shortly before the freeze for Jessie. Andy Doan spent a lot of his time getting these machines going for us, and then I set up two of them as build machines and one as the porter box we were still needing.

With these extra machines available, we quickly caught up with the ever-busy "Needs-Build" queue and we've got sufficient build power now to keep things going for the Jessie release. We were officially added to the list of release architectures at the Cambridge mini-Debconf in November, and all is looking good now!

And in the future?

I've organised the loan of another arm64 machine from AMD for Debian to use for further porting and/or building. We're also expecting that more and more machines will be coming out soon as vendors move on from prototyping to producing real customer equipment. Once that's happened, more kit will be available and everybody will be able to have arm64-powered computers in the server room, on their desk and even inside their laptop! Mine will be running Debian Jessie... :-)

Thanks!

There's been a lot of people involved in the Debian arm64 bootstrapping at various stages, so many that I couldn't possibly credit them all! I'll highlight some, though. :-)

First of all, Wookey's life has revolved around this port for the last few years, tirelessly porting, fixing and hacking out package builds to get us going. We've had loads of help from other teams in Debian, particularly the massive patience of the DSA folks with getting early machines up and running and the prodding of the ftpmaster, buildd and release teams when we've been grinding our way through ever more package builds and dependency loops. We've also had really good support from toolchain folks in Debian and ARM, fixing bugs as we've found them by stressing new code and new machines. We've had a number of other people helping by filing bugs and posting patches to help us get things built and working. And (last but not least!) thanks to all the folks who've helped us beg and borrow the hardware to make the Debian arm64 port a reality.

Rumours of even more ARM ports coming soon are entirely scurrilous... *grin*

18:03 :: # :: /linaro :: 0 comments

Successful Summer of Code in Linaro

It's past time I wrote about how Linaro's students fared in this year's Google Summer of Code. You might remember me posting earlier in the year when we welcomed our students. We started with 3 student projects at the beginning of the summer. One of the students unfortunately didn't work out, but the other two were hugely successful.

Gaurav Minocha was a graduate student at the University of British Columbia, Vancouver, Canada. He worked on Linux Flattened Device Tree Self-checking, mentored by Grant Likely from Linaro's Office of the CTO. Gaurav achieved all of his project's goals, and he was invited to Linaro's recent Linaro Connect USAConnect conference in California to meet people and and talk about his project. He and Grant presented a session on their work; it was filmed, and video is online. Grant said he was very happy with Gaurav's "strong, solid performance" during the project.

Varad Gautam was a student at Birla Institute of Technology and Science, Pilani, India. He succeeded in porting UEFI to the BeagleBone Black. Leif Lindholm from the Linaro Enterprise Group was his mentor for the summer. At the end of the summer, Varad delivered a UEFI port ready for booting Linux and his code was included in Linaro's September UEFI release. Leif said that he was "very pleased with Varad's self sufficiency and ability to pick up an entirely new software project very quickly". We were hoping to invite Gaurad to Connect in California also, but travel document delays got in the way. With luck we'll see him at the next Connect in Hong Kong in February 2015.

Well done, guys! It was great to work with these young developers for the summer, and we wish them lots more success in their future endeavours.

Google have also just confirmed that they will be running the Summer of Code program again in 2015. I'm hoping that Linaro will be accepted again next year as a mentoring organisation. I'll post more about that early next year.

18:06 :: # :: /linaro :: 0 comments

Linaro welcomes GSOC 2014 students

After several weeks of review and discussion, the application and selection period for the 2014 Google Summer of Code is over. 4,420 students proposed a total of 6,313 projects for this summer. From those, 1,307 students have been accepted (more details), and Linaro is one of the 190 Open Source projects that will be working with students this year.

In our first year as a GSOC mentoring organisation, we received 17 applications and Google allocated us 3 slots for student projects. It was quite a challenge to pick just 3 projects from the excellent field, and it's a shame that the limited number of slots meant we had no choice but to disappoint some people. Thanks to all those who applied!

I'm delighted to announce our 3 chosen interns for 2014:

• Gaurav Minocha is a graduate student at the University of British Columbia, Vancouver, Canada. His project is Linux Flattened Device Tree Self-checking, mentored by Grant Likely from Linaro's Office of the CTO.
• Ricardo de Freitas Gesuatto is a student at Federal University of São Carlos (UFSCar), Brazil. He will be working on a project entitled "Lightweight IP Stack on top of OpenDataPlane", mentored by Maxim Uvarov from the Linaro Networking Group.
• Varad Gautam is a student at Birla Institute of Technology and Science, Pilani, India. He will be Porting UEFI to Low-Cost Embedded Platform (BeagleBoneBlack). Leif Lindholm from the Linaro Enterprise Group will be mentoring.

Please join me in welcoming these three new engineers to the Linaro team!

We have a GSOC wiki ready for our students to use at

https://gsoc.linaro.org/

and hopefully they will start adding content there soon about themselves and their projects (hint!). In the meantime, we have more information about our original proposals and the GSOC program in the main Linaro wiki.

Starting today, the next phase of the program is the so-called "bonding period". Students are encouraged to get to know people within Linaro (especially their mentors!) and prepare to start work on their projects, whatever is needed. The official start of the work period for GSOC is May 19th, and it runs through to August 18th. We will give updates on progress through the summer, and we're hoping to talk about our results at the next Linaro Connect in September.

Good luck, folks!

12:33 :: # :: /linaro :: 0 comments

Scanning for assembly code in Free Software packages

In the Linaro Enterprise Group, my task for the last several weeks was to work through a huge number of packages looking for assembly code. Why? So that we could identify code that would need porting to work well on AArch64, the new 64-bit execution state coming to the ARM world Real Soon Now.

Working with some Ubuntu and Fedora developers, we generated a list of packages included in each distribution that seemed to contain assembly code of some sort. Then I worked through that list, checking to see:

1. if there was actually any assembly there;
2. if so, what it was for, and
3. whether it was actually used

I've written a full report about what I found in the scan, and I'll be writing some more articles based on it shortly.

03:08 :: # :: /linaro :: 2 comments

Linaro Connect in Hong Kong, 2012: Ruby bug squashed!

Sometimes conferences can be dull and boring, but sometimes they can be just awesome in terms of finding the right people to collaborate with. Linaro Connect in Hong Kong last week was definitely one of the great ones!

I chaired my my usual sessions (armhf status and cross-distro ARM) and we had some lively discussion in both. We're probably just about done with the armhf sessions, as most distros have accepted a hard-float ARMv7 port now and there's not so much specific work left there now that future sessions will be necessary. The cross-distro work for ARM ports is likely to continue into the future, but we're going to be concentrating on bootstrapping work for ARMv8 soon.

Talking v8, there were a lot of meetings discussing the various work topics for this new 64-bit ARM architecture: kernel, toolchains, bootstrapping etc. More to come on that soon!

On top of all this useful discussion and planning, we also found time during the week for some hacking. This was the highlight of the week for me, as I found some expert help to solve my long-standing Ruby on ARM bug (Debian bug #652674, ruby1.9.1: FTBFS on armhf: test suite segfaults). Ulrich Weigand (an IBM toolchain wizard seconded into the Linaro team) sat with me for a couple of hours while we worked through reproducing the problem, only to find that he could not reproduce it! The crash had looked very much like a pthreads locking bug, which was scaring me. After some digging, we worked out where the problem was, and how it was now fixed.

For a long time on ARM, the Linux getcontext/setcontext system calls have never been implemented; apparently nobody really missed them, so they have never been a priority. In Ruby 1.9.x, the implementation of the new "fiber" primitive wanted to use getcontext/setcontext to control stack state etc. in different threads of control. In cases where they are not available or known not to work well, Ruby has fallback code to implement similar functionality. It seems that fallback code is buggy. Maybe it was correct at some point and has bit-rotted due to not being exercised, or maybe it was buggy as written, but it clearly does not work correctly for us now. In the last few months, getcontext/setcontext have finally been implemented for ARM in glibc trunk (by Michael Hope, also in the Linaro toolchain team!) and backported into current Debian and Ubuntu eglibc versions. Re-running configure and rebuilding Ruby against the most recent code in both Sid and Precise fixed the test suite crash we were seeing earlier. Yay! We could also provoke the bug again at will by quickly hacking around the Ruby source to force it to switch back to the fallback code, thereby verifying the fix.

Why does this matter? As we're expecting ARM servers to enter the market soon, web apps written in Ruby on Rails are going to be an important part of the software stack that customers will want to run. Broken fibers and threading would not help here!

It's great to meet up and work with talented folks like this; Linaro Connect is an excellent event for getting stuff done!

20:22 :: # :: /linaro :: 0 comments