Club Linux Coffs Harbour

The DDL code paths for Drizzle are increasingly different from MySQL. For example, the embedded_innodb StorageEngine CREATE TABLE code path is completely different than what it would have to be for MySQL. This is because of a number of reasons, the primary one being that Drizzle uses a protobuf message to describe the table format instead of several data structures and a FRM file.

We are pretty close to having the table protobuf message format being final (there’s a few bits left to clean up, but expect them done Real Soon Now (TM)). You can see the definition (which is pretty simple to follow) in drizzled/message/table.proto. Also check out my series of blog posts on the table message (more posts coming, I promise!).

Drizzle allows either your StorageEngine or the Drizzle kernel to take care of storage of table metadata. You tell the Drizzle kernel that your engine will take care of metadata itself by specifying HTON_HAS_DATA_DICTIONARY to the StorageEngine constructor. If you don’t specify HTON_HAS_DATA_DICTIONARY, the Drizzle kernel stores the serialized Table protobuf message in a “table_name.dfe” file in a directory named after the database. If you have specified that you have a data dictionary, you’ll also have to implement some other methods in your StorageEngine. We’ll cover these in a later post.

If you ever dealt with creating a table in MySQL, you may recognize this method:

virtual int create(const char *name, TABLE *form, HA_CREATE_INFO *info)=0;

This is not how we do things in Drizzle. We now have this function in StorageEngine that you have to implement:

int doCreateTable(Session* session, const char *path,                   Table& table_obj,                  drizzled::message::Table& table_message)

The existence of the Table parameter is largely historic and at some point will go away. In the Embedded InnoDB engine, we don’t use the Table parameter at all. Shortly we’ll also get rid of the path parameter, instead having the table schema in the Table message and helper functions to construct path names.

Methods name “doFoo” (such as doCreateTable) mean that there is a method named foo() (such as createTable()) in the base class. It does some base work (such as making sure the table_message is filled out and handling any errors) while the “real” work is done by your StorageEngine in the doCreateTable() method.

The Embedded InnoDB engine goes through the table message and constructs a data structure for the Embedded InnoDB library to create a table. The ARCHIVE storage engine is much simpler, and it pretty much just creates the header of the ARZ file, mostly ignoring the format of the table. The best bet is to look at the code from one of these engines, depending on what type of engine you’re working on. This code, along with the table message definition should be more than enough

Typical beach activity, Santa Cruz

Originally uploaded by Nooks

I would have liked to catch the lighthouse to the left of the frame in this shot but it didn't happen and I live the wave activity too much to use another image instead.

Ooops. Emergency woblomo post coz I forgot. Here’s the link to the other day’s screencast that apparently didn’t make it through aggregators. Hohum.

I’ve been an advocate for free software ever since I started using it in 1999 (or was it 1998?). Some who know me would probably say that I’m an anti-Microsoft, pro Free software zealot. Maybe that’s true.

The way I see it, there are three camps. Those who think that open source software is good and so choose to use it; those who think that all software should be free (libre) and that proprietary software is evil; and, those who hate Microsoft and will use anything but.

I want to be in the first camp.

However, it’s true that I also want to convert people to free software. I do. Sorry about that. I want to get people off Windows and Mac and whatever else they are running and onto a free system like Linux. I want others to join me, because I think it’s the better way. Who am I though, to suggest what’s best for someone else?

I don’t like the fact that our Government is a Microsoft shop and spends half a billion dollars on proprietary software licenses a year. Likewise the fact that they create all their data in closed proprietary data formats which I have to try and make available years down the road (as a part of my day job). I wish that people could use an operating system of their choice at work, instead of being forced to use Windows.

Most importantly, I wish that people had choice when it comes to buying a new machine and generally I wish that Microsoft didn’t have their monopoly (can anyone argue that it’s actually been good for the industry as a whole? I think it’s a stretch to say that we wouldn’t have cheap hardware without Microsoft).

Recently, Internode (my much beloved ISP) dumped free software and instead rolled out Microsoft Exchange for mail, calendaring, etc. CEO Simon Hackett outlined his reasons for the migration and it’s a compelling argument.

We’ve been trying to get a working, open source based, cross platform calendaring system happening at Internode for years and years.

And we’ve never managed it.

There’s always something that doesn’t work right, or a need to run a plethora of ‘conduit’ tools between multiple calendar systems. Somehow, no matter how hard you try, that seems to turn into a dogs breakfast, and involves some horrid compromises on at least one of your operating systems in terms of user access to the resulting information.

OK, so that does read a lot like a Microsoft PR statement (I wonder what discount they received for printing that), but if Internode can’t get it right what hope does the rest of the world have?

I remember a prominent free software developer (someone whom I respect greatly) saying to me something along the lines of (and I’m paraphrasing here):

Just because something is open source, doesn’t make it better. Use the best tool for the job, even if it’s proprietary and closed source.

Is that true? If so, then why do I persist with various free software projects which are inferior to their proprietary counterparts?

I guess that it really comes down to your definition of “best” tool for the job. If your set of criteria for best tool does not exclude costly licensing fees, then a closed source solution be the best for the job. On the other hand, if access to the code is important to you, then it doesn’t matter how “great” a closed tool is, it’s never going to be the best tool for the job. So, this is why for some people, Apple iPod might be the best tool for the job, but for others it doesn’t even rank (like me, who has all their music in FLAC).

So you see for Internode, the best tool for the job was one that offered seamless syncing with their iPhones.

C’est la vie.

In writing this I am reminded of a friend who would laugh at me while I spent half an hour on the terminal trying to connect to a WPA encrypted wireless network. His Mac would just work straight away (like what we have now with NetworkManager). For me, a closed operating system wasn’t the best tool for the job, so I persisted with countless hours of command line work to associate with an access point.

I know that this boils down to a world view and I don’t consider myself a free software Zionist, but maybe I have been – or maybe I am. If so, maybe I shouldn’t be.

It’s too tiring to keep on fighting for the adoption of free software and maybe we just shouldn’t be in the first place. I’m reminded of that quote from Linus:

The thing is, at least to me personally, Microsoft just isn’t relevant to what I do. That might sound strange, since they are clearly the dominant player in the market that Linux is in, but the thing is: I’m not in the ”market.” I’m interested in Linux because of the technology, and Linux wasn’t started as any kind of rebellion against the ”evil Microsoft empire.” Quite the reverse, in fact: from a technology angle, Microsoft really has been one of the least interesting companies. So I’ve never seen it as a ”Linus versus Bill” thing. I just can’t see myself in the position of the nemesis, since I just don’t care enough. To be a nemesis, you have to actively try to destroy something, don’t you? Really, I’m not out to destroy Microsoft. That will just be a completely unintentional side effect.

I like that. He’s just getting on with the job of making great open source software because he believes it’s a better way. He doesn’t care what the rest of the world does (at least, that’s what I read from it). That’s what I want. I just don’t want to care any more.

So what’s the point of this post? Well I feel that more and more my own attitude is changing from one of championing free software regardless of circumstance, to a more apathetic position something along the lines of, “I use Linux, you use whatever you want. If you want Microsoft Windows and Office, then go use that.”

If you think that .NET is the future of the Linux desktop, then go right ahead and use it.

Wanna store all your data in proprietary formats? Feel free.

My brother loves Linux and free software now (it’s taken about 10 years of constant barraging to get him to use it) but his wife simply “must” use Outlook. So he still runs Outlook for all their mail. Weird, but OK.

From now on, I won’t push Linux and free software (I’ll probably re-do MakeTheMove to reflect this, someday). This has all been the result of a long time frustration bubbling under the surface, but more recently the culmination of hundreds of events, conversations and epiphanies I’ve had and experienced over the last 6 months or so. I’m done. I’ll simply use what I want to use and leave everyone else to their own devices. If they want to also use free software, as always I’ll be happy to help.

Free Software is Principled

I recall, several years back now, being in some sort of forum somewhere arguing over the implementation of anti-circumvention legislation in Australia.  I recall Rusty Russell talking about ghostscript’s [?] handling of pdf documents at the time and how it respected restrictions settings in the pdf documents.  That is, despite being able to ignore them, ghostscript’s authors decided to respect them.   In practice that would mean that most ghostscript users would also respect those settings.

By way of contrast, today, looking for information about pdf to text conversion tools I came across closed source software whose primary purpose is apparently to remove restrictions from pdf files.  In my experience free software is typically more principled than its closed source counterparts – perhaps stupidly so.

In my day job I do Linux embedded work and as people in the embedded world know, Linux is a pretty commonly used embedded OS. Today I was evaluating a new board and found it had an Intel graphics chip that was not properly detected by Ubuntu 9.10. The ever trusty lspci said this:

00:02.0 VGA compatible controller: Intel Corporation System Controller Hub (SCH Poulsbo) Graphics Controller (rev 07)

We all know that Intel employs a bunch of well known Xorg developers, so this shouldn't be a problem, right?

Unfortunately, it is a problem. Intel's offering for this chipset is the Intel® Embedded Graphics Drivers web page where they offer a 124 megabyte download (registration required). After registration you get to choose which driver pack you want and which OS you are downloading it for. Ubuntu was not on the list and neither was Debian. I chose Fedora 10 (released in 2008) as that was the most recent one.

Now, you can image my surprise when the driver download for Fedora Linux contained just four files:

Archive: /tmp/IEGD_10_3_GOLD.zip testing: UsersGuide_10_3_1525.pdf OK testing: IEGD_10_3_GOLD_1525.exe OK testing: IEGD_SU_10_3_GOLD.pdf OK testing: RELNOTES_10_3_1525.txt OK No errors detected in compressed data of /tmp/IEGD_10_3_GOLD.zip.

Yep, thats right, the driver download for Fedora Linux contains two PDF files, a text file and an executable installer for Windows.

Being the curious (and paranoid) type I decided to explore this further, by running the installer under WINE in a chroot. After the installer you get left with several metric craploads of Java Jar files, and another windows executable iegd-ced.exe that supposedly configures this nightmare. I ran it (again, under WINE in a chroot) but it didn't seem to do anything sensible or worthwhile so I looked around amongst the other installed files and found IEGD_10_3_Linux.tgz.

Inside that tarball there are a bunch of Xorg library binaries (for several different versions of Xorg), a large chunk of source code that gets compiled into the Linux kernel and even better yet, a couple of Microsoft Visual Studio project files. WTF?

Unbe-fscking-lievable. Needless to say, I will avoid any hardware which uses this chipset and any other hardware that requires binary only kernel blobs packaged this badly. Doing so makes my life easier.

The people at Intel who thought this was a good idea must have their own personal mother-lode of stupid.

In my day job I do Linux embedded work and as people in the embedded world know, Linux is a pretty commonly used embedded OS. Today I was evaluating a new board and found it had an Intel graphics chip that was not properly detected by Ubuntu 9.10. The ever trusty lspci said this:

00:02.0 VGA compatible controller: Intel Corporation System Controller Hub (SCH Poulsbo) Graphics Controller (rev 07)

We all know that Intel employs a bunch of well known Xorg developers, so this shouldn't be a problem, right?

Unfortunately, it is a problem. Intel's offering for this chipset is the Intel® Embedded Graphics Drivers web page where they offer a 124 megabyte download (registration required). After registration you get to choose which driver pack you want and which OS you are downloading it for. Ubuntu was not on the list and neither was Debian. I chose Fedora 10 (released in 2008) as that was the most recent one.

Now, you can image my surprise when the driver download for Fedora Linux contained just four files:

Archive: /tmp/IEGD_10_3_GOLD.zip testing: UsersGuide_10_3_1525.pdf OK testing: IEGD_10_3_GOLD_1525.exe OK testing: IEGD_SU_10_3_GOLD.pdf OK testing: RELNOTES_10_3_1525.txt OK No errors detected in compressed data of /tmp/IEGD_10_3_GOLD.zip.

Yep, thats right, the driver download for Fedora Linux contains two PDF files, a text file and an executable installer for Windows.

Being the curious (and paranoid) type I decided to explore this further, by running the installer under WINE in a chroot. After the installer you get left with several metric craploads of Java Jar files, and another windows executable iegd-ced.exe that supposedly configures this nightmare. I ran it (again, under WINE in a chroot) but it didn't seem to do anything sensible or worthwhile so I looked around amongst the other installed files and found IEGD_10_3_Linux.tgz.

Inside that tarball there are a bunch of Xorg library binaries (for several different versions of Xorg), a large chunk of source code that gets compiled into the Linux kernel and even better yet, a couple of Microsoft Visual Studio project files. WTF?

Unbe-fscking-lievable. Needless to say, I will avoid any hardware which uses this chipset and any other hardware that requires binary only kernel blobs packaged this badly. Doing so makes my life easier.

The people at Intel who thought this was a good idea must have their own personal mother-lode of stupid.

A couple of months ago (December 1st for those playing along at home) it marked five years to the day that I started at MySQL AB (now Sun, now Oracle). A good part of me is really surprised it was for that long and other parts surprised it wasn’t longer. Through MySQL and Sun, I met some pretty amazing people, worked with some really smart ones and formed really solid and awesome friendships. Of course, not everything was perfect (sometimes not even close), but we did have some fun.

Up until November 2008 (that’s 3 years and 11 months for those playing at home) I worked on MySQL Cluster. Still love the product and love how much better we’re making Drizzle so it’ll be the best SQL interface to NDB :)

The ideas behind Drizzle had been talked about for a while… and with my experience with internals of the MySQL server, I thought that some change and dramatic improvement was sorely needed.

Then, in 2008, Brian created a tree. I was soon sending in patches at nights, we announced to the whole world at OSCON and it captured a lot of attention.

Since November 2008 I’ve been working on Drizzle full time. It was absolutely awesome that I had the opportunity to spend all my days hacking on Drizzle – both directly with fantastic people and for fantastic people.

But… the Sun set… which was exciting and sad at the same time.

Never to fear! There were plenty of places wanting Drizzle hackers (and MySQL hackers). For me, it came down to this: “real artists ship”. While there were other places where I would no doubt be happy and work on something really cool, the only way I could end up working out where I should really be was: what is the best way to have Drizzle make a stable release that we’d see be suitable for deployment? So, Where Am I Now?

Rackspace.

Where I’ll again be spending all my time hacking Drizzle.

Time to learn Javascript properly.

The Onion: Nation Shudders At Large Block Of Uninterrupted Text

‘…”Why won’t it just tell me what it’s about?” said Boston resident Charlyne Thomson, who was bombarded with the overwhelming mass of black text late Monday afternoon. “There are no bullet points, no highlighted parts. I’ve looked everywhere—there’s nothing here but words.”‘

Story here

Hi all, Time for the March meeting announcement (it's next Tuesday)... The usual details: When: 6:30pm-8:30pm on Tuesday, 16th March, 2010 Where: Room G05/G06 Gerard Innovation Centre Prince Alfred College Capper Street, Kent Town SA Cost: FREE Who: Anyone and everyone. No pre-registration necessary. Presentation: Mark Pulford will be presenting an overview of IPv6 - The Next Generation Internet Protocol. The talk will cover: * Advantages * Differences with IPv4 * Using IPv6 Mark is a Network Engineer at Internode and recently built their IPv6 Broadband Trial: http://ipv6.internode.on.net/access/adsl/ Getting to the Venue: Prince Alfred College faces Dequetteville Terrace, but access to Room G05/G06 is only available from Capper Street; there is no access to these rooms from any other entrances in the College. On Capper Street, Room G05/G06 is closer to The Parade West than to Dequetteville Terrace. Parking is available on-street or maybe off-street (adjacent to the Sports Centre). Look for the two storey stone fronted building with the sign "Gerard Innovation Centre". Enter through the double doors, first room on the right hand side of corridor (ground floor). The external door automatically locks at night. Should it not be propped open please be ready to knock loudly. There are buses that go along Dequetteville Terrace, with a stop adjacent to Capper Street. The 143, 145, and 146 buses leave from North Terrace, and the 141 and 142 buses leave from Currie Street and Grenfell street. If you are in the city, you could also catch the 99C bus to East Terrace, then take a 10-15 minute walk to the College. Pizza: After the meeting, people generally go out for pizza at San Giorgios (cnr. Frome Street and Rundle Street in the city). For more information: Email: organisers@linuxsa.org.au Web Page: http://www.linuxsa.org.au/ Mailing List: linuxsa@linuxsa.org.au IRC: #linuxsa on irc.freenode.net

Recently I’ve been pondering about the role of Linux and free software and whether we should be concentrating on making proprietary stuff work seamlessly with it (or just concentrating on making great free software for us).

I’ve come to the conclusion that we should be making everything “just work.”

Most things are proprietary, my video card is proprietary, my BIOS is proprietary. Linux already works with all these proprietary things, and we should make sure that everything else does too.

Users should be able to seamlessly sync their hardware with Linux, like an iPhone. They should be able to install any Windows application they want to. They should be able to purchase any piece of hardware and have it just work plug ‘n play style.

If users want to use proprietary data formats like H.264 and MP3, they should be able to.

Linux does already do a great job at a lot of these things and we should continue to support as much as we can.

Consumers will buy what they want to buy – things that are nice and shiny and inevitably proprietary. If they can’t use these on Linux, then they will continue to use other operating systems. This vendor lock-in model is not going away any time soon, so unless they work with Linux, these people are tied down elsewhere.

I’m not saying that we should be out there fighting Microsoft for market share, we should just be scratching itches. If it’s an itch to get an iPhone to sync, then it should be scratched and everyone benefits. If the barrier stopping someone from moving to Linux as their primary operating system is because their iPhone won’t sync, then that should be changed.

Sure, Microsoft and Apple are out to destroy us, but who cares? We should just ignore them and keep plodding along making great software for us and scratching itches.

My current job involves working extensively with Omniture products. The company has an annoying habit of secrecy, with documentation only available on request for many aspects of their products.

They also attempt to obfuscate their JavaScript, despite the fact that a determined viewer should be able to work it out eventually. I'm told this is so that people aren't tempted to play with the code. The obvious methods of deobfuscation are pretty tedious, and because Omniture don't use standard (minify et al) methods of obfuscation so it seems a little more difficult. Fortunately I lucked onto a better approach.

You'll need Firebug, and if you don't have that already you should anyway. Go to a page that already has the Omniture "plugin" (function) you want. Open the Firebug console and run alert(s.functionNameYouWant) and run it. You'll be shown a nicely-formatted anonymous function, which will be much easier to read than the line noise you'll see in the actual s_code.js file.

In my case I'm after Cross-Visit Participation, and that's used on the Omniture site itself (though an older version than the latest available from Omniture which has a very useful additional feature).

Contact me

Attendees of LCA2009 may be pleased to know that there is at least *some* hope for the Tassie Devil.



There is a completely separate monoculture that has thus far not been affected by the facial tumor. Whether it will be enough is the $44,000 question.

Santa Cruz Bongo

Originally uploaded by Nooks

One thing I like about several of the Santa Cruz pictures is that they have a couple of layers to them, such as this one, though I do seem to recall that background colors were less muted than they are here.

On the other hand, what's this drum doing sitting on the beach anyway? No-one seemed to be tending it.

BlueTree Search recently re-launched their site with the help of Stripe Creative, switching from plain HTML to WordPress as a CMS. Along the way, they decided to bring their job listing in-house.

Initially, this client asked us to add a job posting section to the web site we developed for them a few years ago. Our awesome programmer dude (Scotty at randomproductions.net) suggested that for the same cost of adding a custom-written application, we could re-create the site in WordPress and find a customizable job posting plug-in. This allowed us to offer the client some serious added value (easier updates and adding new functionality through plug-ins) at no additional cost. The transfer to WordPress was transparent to the end users — we kept the site’s original design — and adding/customizing Job Manager was a breeze.

— Brian Robboy, Stripe Creative

With a little bit of editing of the Individual Job Template, they were able to easily highlight pertinent information, without distracting from the content of the listed job.

And a final bonus comment from Brian:

I love working with people like you who actually CARE about the applications they’re writing…we’d never get this level of dedication, customer service and communication with a commercial product.

Are you doing creative things with Job Manager? Have some time to tell me how and why you’re using it? Drop me a line, you could be the next Job Manager Highlight!

When I was sticking to Xubuntu 8.04, one of the things that saved my life was PPAs. A PPA is a Personal Package Archive, hosted on Launchpad. They let me install the latest versions of OpenOffice and Firefox even though the versions that came with my distro were a few versions behind. They are also useful for a ton of things that for whatever reason don’t make it into the official Ubuntu repositories. Once you have added a PPA to your system, installing and update software from it is as seamless as from the official repositories, and the apt-get style of software management is possibly my favourite thing about Ubuntu/Linux.

However pre-9.10, adding a PPA was kinda fiddly – you had to edit certain files, add special keys, blah blah blah. Now in 9.10 (karmic koala) it is super easy. For example, here is how I install the Arduino and Scratch IDEs from their PPAs:

sudo add-apt-repository ppa:scratch/ppa

sudo add-apt-repository ppa:arduino-ubuntu-team/ppa

sudo apt-get update

sudo apt-get install arduino

sudo apt-get install scratch

I love it!

In an interesting blog on patents, copying and litigation former Sun CEO Jonathan Schwartz discloses that Bill Gates and Steve Balmer tried to put the frighteners on Sun over OpenOffice.org to try and protect their office application monopoly. Their attack went like this:

“Microsoft owns the office productivity market, and our patents read all over OpenOffice.” [...] “We’re happy to get you under license.”

Of course (as ever) they do not identify any patents, as that would let us fix any problems (if there are actually any), they would much rather weave their usual web of FUD on the matter than come clean. Jonathan’s response turned the issue on them on a different tact:

“We’ve looked at .NET, and you’re trampling all over a huge number of Java patents. So what will you pay us for every copy of Windows?”

That killed that angle of attack off..

This item originally posted here:



Microsoft Tried to get Patent Royalties for OpenOffice.org from Sun

A new lens got added to the collection today, a Canon 24-105mm f/4 L IS. I'm just a wee bit excited :-)

James Bradley responds on Slattery CAL Article

A little while ago Luke Slattery wrote an article on the amounts returned by CAL to authors.  The Australian has published a response from James Bradley, ‘one of three author directors on the board of the Copyright Agency Limited’.   Luke Slattery has also added an editorial comment (describing criticism of his article as being at times “histrionic”) at the end.   There does not seem to be a place for Creative Commons material in the Bradley article.