After setting up a test hadoop map-reduce cluster for our viral seeding service here at rubber towers earlier this year, I needed to come up with a simple task to run to get my head around the map-reduce programming model.
Stumbling upon this great blog post, I decided to run a task to render a Buddhabrot in glorious detail (6000X4000 pixels).
These images are released under a creative commons non-commercial share-alike licence (click the image for a low-resolution desktop background)

BuddhaBrot by Team Rubber / Tim Wintle is licensed under a Creative Commons Attribution-Non-Commercial-Share Alike 2.0 UK: England & Wales License.
Here is a close-up of the “head” of the brot (the part on the left) in it’s original resolution:

You can download the image scaled down to a desktop background, the high resolution image (not on the site to preserve bandwidth) is roughly 40Mb as a png.
For those interested, the intermediate data (passed between the map and reduce phases) was roughly 250 Gb of raw data. The mapper and reducer were both written in Python, with a single final reduce done off-cluster using PIL (the Python Imaging Library).
UPDATE
You can now download a torrent of the entire, full resolution image.
Tags: code, Developer, fun, map reduce, python













How many nodes in the cluster? How long did it take?
Why not provide a torrent of the full resolution image?
[...] – bookmarked by 5 members originally found by zainab91 on 2008-09-17 Fractals and lots of Machines http://www.teamrubber.com/blog/?p=111 – bookmarked by 5 members originally found by Neldorf on [...]
I’m on a mandelbrot image harvesting spree. Willing to ssh the png to my work server? I tried the torrent but it’s dead. Purely for personal appreciation.