Author Archive

Google Image Search searches by image!

Tom Blockley - June 29th, 2011

So I went on to Google just now to look for an image, and I discovered that they now let you drag and drop images into the image search bar!

Now this is impressive. It’s like TinEye + DropBox + Googles own similar images functionality.

So I played around for a bit, and it turns out it’s even smarter than that! I dragged an image of me playing a gig into the search box, and it identified my bass!

Picture 11 Google Image Search searches by image!
It recognised that:

  1. I was playing an instrument
  2. It is a bass
  3. It is a Musicman Stingray

Google scares me sometimes.

The Case of The Phantom Data.fs

Tom Blockley - January 27th, 2011

It’s rare that I’m moved to write a blog post about something that happens to me while I’m at work. This is mostly because I assume everybody in the world who uses Plone has had the same problems before. Once in a while though you come across something that it seems like you really should be sharing with the world.

Zeoserver will continue to ‘function’ normally even without a Data.fs and regardless of zeoclient restarts.

This happened to us recently. Now I’m sure that this is not something that you will come across very frequently, if at all. However, if you do have this happen to you, the consequences could be disastrous.

So how did you find that out and what triggered it?

Here’s a hypothetical situation for you – let’s imagine that you are developing on a site and you have made some changes to some of the products that you use in that site. It comes time to try installing these new products, so you make a backup of your Data.fs (sensible, well done!) and then you re-buildout, and restart your zeoclient instances one by one, to avoid downtime.

You reinstall your products and add some data into the site, looking out for problems. UH-OH! Something unexpected has happened and you need to roll back!

You roll-back your buildout, replace your Data.fs with that backup you cleverly made earlier, and restart your zeoclient instances. JOY! Your site is back to how it was before, nothing is broken and everything is totally fine. Phew! That was close.

So you carry on working, data gets entered into your site and life goes on. Now at some point in the future, you hit a situation which requires you to restart your ENTIRE site. Zeoclients and Zeoserver. You do so, and when you restart your instances, suddenly you have been flung back in time! All the information you entered into your site since that rollback has disappeared! WTF?!

There were no clues in the system that this was going to happen:

  1. Your transaction log / undo history were all running fine and recording what had happened
  2. You weren’t getting any errors whenever you saved data to the system
  3. You zeoclients were restarting fine

Maybe. Maybe, if you were running backups using repozo you might have noticed that the diffs were not as you might expect, but that system works, it’s just there in case of emergency right? Well all that time, your repozo has been backing up the restored backup of the database, but your zeoserver has been storing everything to a patch of hard drive which was being held by the now non-existent Data.fs that you replaced when rolling back.

What can I do?

Nothing. Unless you get very lucky, scraping your hard drive, you wont recover that information. You’ll have to put it all in again.

The Lesson?

If you ever have to replace your Data.fs, restart your zeoserver as soon as you do so.

Here’s a picture we drew:

2011 01 27 18.40.22 The Case of The Phantom Data.fs

A diagram of a disappearing Data.fs


buildout.zc nested packages zcml quirk

Tom Blockley - September 29th, 2009

I came across this problem a couple of months ago, then promptly forgot about it until it happened to me again this morning.

We have a pretty good understanding of buildouts. If you don’t, you might want to read this great article: Understanding Buildouts

In a buildout config you will usually declare a set of eggs and zcml which you want to import, something like this:


eggs =
my.product
my.otherproduct
zcml =
my.product

Sometimes though, you may have a multilevel package, for example a product that provides optional management tools for an existing product. Instinctively you might register your management package after your main product, like this:


eggs =
my.product
my.product.management
my.otherproduct
zcml =
my.product
my.product.management

If you do that, your product will build out absolutely fine, but when you come to start an instance you will see an error something like this:


ZopeXMLConfigurationError: File
"/(...)/etc/package-includes/002-my.product-configure.zcml", line 1.0-1.64
IOError: [Errno 2] No such file or directory:
'/(...)/my.product.management/src/my/product/configure.zcml'

You’ll think, “this is a bit weird, there’s not supposed to be a configure.zcml there anyway – it’s in my/product/management where it’s supposed to be… isn’t it?”

Then you’ll probably spend a while searching the internet for hints, maybe adding a configure.zcml to that folder just in case and generally scratching your head. You might even try declaring the zcml includes the other way round, like this:


zcml =
my.product.management
my.product

This wont work either. In the end, when none of it works, you might stumble across the answer. Apparently you have to declare the egg includes the other way round instead:


eggs =
my.product.management
my.product

Re-run your buildout and your instance will start with no errors. Why? I don’t know, but if anyone has the answer, I’d love to know it!