Now this is impressive. It’s like TinEye + DropBox + Googles own similar images functionality.

So I played around for a bit, and it turns out it’s even smarter than that! I dragged an image of me playing a gig into the search box, and it identified my bass!

It recognised that:

1. I was playing an instrument
2. It is a bass
3. It is a Musicman Stingray

Google scares me sometimes.

1. New Site Admin bar – a new admin bar has been added to the logged in live view, enabling quick editing of posts / pages
2. Post formats – now able to tag post format and subsequently display individual post formats eg. gallery post, aside, features
3. Multi-taxonimy queries – essentially an improved searching mechanism, more power to developers
4. Network Admin User – superseeds multi-site WordPress “Super Admin” user
5. In-site post linking – adds the facility to search and add links for internal blog posts / pages – big UI win!

This update certainly moves WordPress on from a simple blogging engine to a fully fledged CMS. Go download the latest update NOW! In honour of the release here’s Django at his best:

Click here to view the embedded video.

RDFa actually became a mandatory requirement for all central government consultations published after 1st January 2010, but in the past year we have seen very little use of this new wealth of freely available data. There were rumours that Directgov planned to use it to import consultations, but I can’t find any evidence of this on their site.

We were sad that we’d gone to all the trouble* to incorporate RDFa into our clients’ consultations, and nobody was making use of it, so we decided to do something with it ourselves. We’ve made an Aggregator that can collect together consultations from any website as long as it publishes its consultations in an RSS feed and includes the appropriate RDFa markup. It also publishes its own RSS feed so that the aggregated consultations can be fed into a further tier of applications.

We’d be really interested to hear of other apps that are making use of RSS and RDFa in relation to consultation data. Surely we’re not the only ones?

*actually it wasn’t much trouble at all thanks to the helpful guidelines from the COI.

Zeoserver will continue to ‘function’ normally even without a Data.fs and regardless of zeoclient restarts.

This happened to us recently. Now I’m sure that this is not something that you will come across very frequently, if at all. However, if you do have this happen to you, the consequences could be disastrous.

So how did you find that out and what triggered it?

Here’s a hypothetical situation for you – let’s imagine that you are developing on a site and you have made some changes to some of the products that you use in that site. It comes time to try installing these new products, so you make a backup of your Data.fs (sensible, well done!) and then you re-buildout, and restart your zeoclient instances one by one, to avoid downtime.

You reinstall your products and add some data into the site, looking out for problems. UH-OH! Something unexpected has happened and you need to roll back!

You roll-back your buildout, replace your Data.fs with that backup you cleverly made earlier, and restart your zeoclient instances. JOY! Your site is back to how it was before, nothing is broken and everything is totally fine. Phew! That was close.

So you carry on working, data gets entered into your site and life goes on. Now at some point in the future, you hit a situation which requires you to restart your ENTIRE site. Zeoclients and Zeoserver. You do so, and when you restart your instances, suddenly you have been flung back in time! All the information you entered into your site since that rollback has disappeared! WTF?!

There were no clues in the system that this was going to happen:

1. Your transaction log / undo history were all running fine and recording what had happened
2. You weren’t getting any errors whenever you saved data to the system
3. You zeoclients were restarting fine

Maybe. Maybe, if you were running backups using repozo you might have noticed that the diffs were not as you might expect, but that system works, it’s just there in case of emergency right? Well all that time, your repozo has been backing up the restored backup of the database, but your zeoserver has been storing everything to a patch of hard drive which was being held by the now non-existent Data.fs that you replaced when rolling back.

What can I do?

Nothing. Unless you get very lucky, scraping your hard drive, you wont recover that information. You’ll have to put it all in again.

The Lesson?

If you ever have to replace your Data.fs, restart your zeoserver as soon as you do so.

Here’s a picture we drew:

A diagram of a disappearing Data.fs

Pictures below…

Big Paper: an open data essential

Best seats, best view #1

Very South By Southwest

Arduino. Proper wires and things.

Best seats, best view #2

Processing Your Freedom data – matrix style

Generating Haiku from Your Freedom data…

Colston Hall – good location for this

Pencil – my favourite tool

Thinking up ideas

Start the day

This morning we had a room full of multi-talented front-end developers, back-end developers, designers, journalists, bloggers, tweeters and publishers, all ready to share their ideas and get making…..

So why have a Hack Day?

“Because it’s fun to make stuff, good to work with the council and talk, tweet and blog about it”. Mr Andy P.

Hack day can explore opening up data the council has and doing more with it, Hack Day is about using data in ways that can benefit people in the city. Bristol City Council.

There has been no shortage of great ideas so far today around different data sets and the creative ways to utilize and represent information.

Teams have been formed and are currently scattered around Colston Hall to work on and develop ideas and digital tools.

Some of the main ideas so far include:

Doing more with the ‘Your Freedom’ Dialogue data the government has now released
Ideas include making the data simpler to understand, running an analysis to discover the most discussed ideas and even condensing these down into poetry.

Mapping out the creative hot spots in Bristol

Looking at Bristol City Council environmental data on shopping trolleys being dumped in rivers and water quality data

Or even a rather unique idea which combines the two, such as creating a mobile game involving getting a shopping trolley home without it going into the river (including info on water quality of course!).

Using Bristol bus data and developing a web app that draws upon live Bristol Bus information

This would allow users to discover where the nearest bus is and when it’s due. Ideas have even been discussed about creating a ‘hyper-local’ bus route newspaper.

We will re-gather at 5 to review the demo’s and prototypes of all the ideas being discussed. Exciting stuff!

We’re here in the Colston Hall

all day, so feel free to drop by and see what’s going on.
If not, then we’re tweeting under #bristolhackday, and will be blogging more too…

Unlike the the Go Programming Language that Google released a while ago, Sawzall is a DSL tailored for fairly specific kind of jobs – processing large amount of log data in a map-reduce style to get statistical summary data.

Working on the Viral Ad Network, I do quite a lot of this kind of work, so when I first came across the Sawzall research paper a few years ago I was quite interested.

I’ve seen a few opinions about Sawzall from ex-googlers online, and the opinions at completely different ends of the spectrum. Now I’ve had a chance to play around with the language myself, here are my initial thoughts:

Firstly, the language itself isn’t exactly pretty (I’m comparing it to Python here), but as far as I’m concerned it’s actually quite good compared to languages such as R, or GLSL.

Sawzall is a remarkably compact language for some kind of jobs though – I quickly ported a simple hadoop streaming map-reduce step from Python to Sawzall, and (to my surprise) what was a 55 line Python program just for the mapper became a 22 line Sawzall program! (admittedly that’s because half of the python code deals with reading key,value pairs in and emitting the values – which are fairly much built-in to Sawzall.

One thing that did surprise me was that having ported my code to Sawzall, my application was slower than the original Python version. (That’s surprising because Sawzall is supposed to generate native code). I’m assuming this is something I could optimize away, and would become negligible for more complex processing for each row.

Of course the Sawzall compiler / runtime isn’t really designed to be run by itself – it’s designed to be embedded into a larger application as part of a distributed map-reduce flow – I haven’t had time to try integrating it into a larger application yet, but I’m guessing time will tell.

While I’m generally pretty old-school when it comes to grammar, my gripe is not really with the new-fangled use of ‘creative’ as a noun (“I’m a Creative at XYZ Media Agency”).  My gripe is that people use it as a job title, on its own, with no qualification or explanation.

“I am a general purpose Creative.  I am a one-stop shop for all your creativity needs.”

There are a great many jobs (and not all of them in the arts) where creativity goes without saying.  My understanding of creativity is about making something original (or at least original as far as the creator is concerned; I don’t suppose it has to be entirely original in the grand scheme of things as I think that’s the definition of ‘genius’).  But anyway, there needs to be some kind of spark, a moment of “Ah-ha! That’s it!”, and a fair bit of autonomy over what you do and how you do it.

I daresay everyone at Rubber could list a fair few creative aspects to their job.  In fact, I can’t imagine any job being much fun unless it included a large chunk of creative license.

So how come the noun “Creative” has come to describe only a subset of such jobs?  By this I mean people in the visual arts and media.  Graphic designers, film directors and game designers are clearly creative (you’d hope), but they aren’t the very definition of the word.

Surely the point of a job title is to differentiate what you do from what other people do. You wouldn’t call yourself “a Competent” or “a Conscientious” – these are just parts of being good at your job.

If you call yourself “a Creative”, you might be implying that the rest of us aren’t capable of being creative, that our work doesn’t require it.  Or perhaps you’re saying that you provide some kind of latent, abstract creativity, but without any specific skill (be it painting, drawing, writing code or closing a deal) that would let you actually express it.  Either way, that single word seems like a pretty unhelpful title to me.

Sure, I was presented with a tantalising list of episodes, but there weren’t any play buttons – so I couldn’t make any use of Channel 4′s (very cool) site.

Suddenly it hit me, what if they were drawing the play buttons with javascript. Javascript is single-threaded, so a single script mis-behaving can stop the whole of the rest of the page from working.

Sure enough, looking at the site carefully it turned out they are including Facebook connect. The Facebook servers were responding incredibly slowly, but failing to respond with a server error that would let the rest of the site run.

So in short, the whole of channel 4′s 4oD site was useless to me (and, I assume to a large number of other people) because of Facebook.

Now I don’t know if Channel 4 have any kind of SLA with Facebook (I’m assuming not), but I’d be interested to know if there was a risk analysis done when they added Facebook Connect to the site.

After one users stumbled upon a serious XSS vulnerability (a type of bug which allows attacker to take control of everything your web browser does on a specific site), things could have got really serious.

A well-planned attack by a well-informed programmer could have run crazy – bombarding a huge number of web users with mallware and other attacks in a very short time, and without YouTube noticing for a significant time if done subtly enough.

Luckly, the news spread via 4Chan – a site who’s average user can hardly be described as well-informed. Within a few minutes the site was over-run with scrolling-marquees and redirects to shock sites – just the kind of thing that’s not going to stay under YouTube’s radar for long.

According to a Google spokesperson, it took under an hour for a temporary fix to be applied, and under two hours for a full fix to be deployed across the whole of youtube.

So thank you 4Chan – your practical jokes have saved a huge number of users from serious attacks. That’s the power of full-disclosure at work.

(Of course you could have just emailed them the vulnerability and saved all the people that did get attacked – YouTube have always responded very quickly when I’ve contacted them with serious issues)