Wahanda TechBlog

When we first started Wahanda we made the decision to have an all Mac OSX development environment. This has served us pretty well over the last year or 2 and the Mac is a great platform. For those like myself who only gets involved in PHP development with a single windows VM for SQL Server it’s still working great.

However our primary site platform now spans a fair few Java Webapps. Running 2 tomcats + VM’s for IE6, IE8 etc. has got to the point where the standard test-amend-compile-test cycle can be in the order of minutes. This was understandably causing a lot of complaints!

So, we all sat down and had a long discussion about what to do. I would love to be in a position to buy everyone dual quad-core Mac Pros but we really don’t have that sort of money at the moment. So with general agreement we decided to move to a more generic hardware platform running Debian Linux (We did debate using Ubuntu but as the servers are all Debian it was decided to go for consistency).

Chris went away and played with some options and ended up with this spec:

• Asus P5QL Pro Motherboard
• Intel Core2 Quad @ 2.66Ghz
• 2 x Crucial 4Gb DDR2 RAM
• GeForce 9600GT Dual DVI Video
• 2 x 250Gb Seagate Barracuda 7200rpm HDD
• Various cheepy cheep ATX cases

This is at least a doubling of the previous developer MacBooks. 2 more cores, twice the RAM and RAIDED drives rather than the rather pathetic 5400rpm notebook drives from before.

The following diagram is just the 2 tomcats starting up. That capacity really is needed!

This one is building the entire site stack. Doesn’t suck up quite as much as an init of the environment but the spread of load across the 4 cores is great to see.

Overall we’ve reduced the test-amend-compile-test back to only a few seconds. An enormous boost to productivity in the team. The downside was that we lost everyone who made the switch for 2 straight days why they got everything set up.

The marketing / design / copy team are also very happy with their pass-me-down MacBooks.

What a month! On Monday we launched our take on collective buying - Wahanda MobDeal.

This is an area of eRetail which is really picking up momentum in these cash-strapped times so we were obviously hoping we would get a good response. As usual our supply team have pulled together some amazing offers and combined with the immediacy of the 24-hour availability period it’s so far proved very compelling.

The premise is fairly simple, and in that respect is a take on a fairly old idea: Collective Purchasing Power. We negotiate an amazingly good deal with a supplier which is only available if we can deliver a minimum number of orders. It’s then put front and center on the site for just 24 hours. So everyone knows what the deal is for that today we send the details via email early in the morning (well early for us techies).

Combined with giving the user the ability to easily spread the word (Facebook, Twitter etc.) it creates a buzz that is truly astonishing.

Getting it built

So what have been the challenges with this from a Technical perspective?

In many ways this was quite a straightforward project. The goal was to best combine features we already have on the site, rather than re-invent any wheels. This made the project far more about the detail than a big engineering effort. We chose the christmas iteration to implement some new designs (more on that to follow) which also created a lot of attention to the little things.

Thanks to the use of electronic fulfillment (a printable voucher) we were able to cut down the booking process to a single page. We then had to implement differing handling based on whether we have reached the required minimum and build lots of different email templates for each variation. Chris did a great job adding the dynamic elements that make the product feel so vibrant. Countdown timers, dynamically updating sales counters and the like. We haven’t made the hourglass actually functional yet but we will.

ZQueue

A pretty crucial part of this process is the ability to only authorise payments up until the point the deal is on. If we don’t hit the minimum then no one pays anything. Up to this point we’ve always authed and completed in a single step so this proved to be quite a big change. Fortunately we were already on the way there with some of core technologies.

We’ve been slowly changing our architecture to make a lot more use of asynchronous message queues. Usually associated with enterprise class systems messaging based systems are a great, simple of way of clearly separating out parts of the architecture, and hence making the development split easier. So the lead developer on the main process was able to simply concern himself with the workflow and not the resulting offline actions like sending emails and processing payments.

Our queuing system was built in-house (it’s really very simple) and is totally persistent. All it consists of are a bunch of stored procedures in SQL Server which deal with doling out items to process or adding items to defined queue. There are some additional maintenance jobs that run to ensure nothing ever gets missed, as well as ensuring the indexes on the queue are usable.

In addition we have some features which break the FIFO rules, allowing prioritization of items and automatic failure mgt. (with re-queuing) if a downstream process is unable to deal with an item successfully. This makes it a great way to see what’s going on at any one time but watching the flow of data across the queue.

Anyway, I hope you all enjoy MobDeals. There’s lots more very cool stuff in the pipeline so watch this space!

Well we are working through the weekend very early in the fresh year so a little to and fro footie with an exercise ball is totally called for.

Weekend Footie Fun!

Another year done. Most of the team here are winding down in preparation for the holiday break. (Customer services and tech operations will still be around). The last few months have been rather crazy here and I haven’t been as good as I could with keeping the tech blog up to date. We’ve launched so many feature in the last 4 months that I haven’t written about including the Supplier Extranet, Questions and Answers, Widgets and more.

My list of resolutions for January will include to be more communicative.

Meanwhile in honour of International Backup Awareness Day here are some random musings on backup strategy…

Site Backup

Over the weekend I was going back and reviewing our backup strategy for the site as a result of this post from Jeff Atwood. For those that don’t know he lost his blog www.coding-horror.com last week as a result of a drive failure on the host server. The regular backup was backing up to the same drive on the host which was a bit of a FAIL. I’m sure Jeff is beating himself up quite nicely to that’s enough of the poking.

Backup is one of the few things that has the potential to keep me up at night. It suffers form being very uninteresting yet absolutely vital to the company. It’s a bit of a no-win in this respect as if everything is fine then no one notices. If it goes wrong (and by wrong I mean really really really wrong) you will probably lose your job.

Anyway, I had a look and found a couple of holes (I always seem to find some) but decided that overall I’m happy. The key to our production strategy (there is a separate model for office stuff) is to keep all the data we actually care about in one place, the database server. We currently have 9 databases storing everything from emails to financials to images. Having everything in the database means all I have to worry about is making sure that the database server is backed up.

My experience with hardware over the last 15 years has taught me is to trust the adage: DATA IN ONLY PLACE DOESN’T REALLY EXIST. This has lead me to be a little paranoid over potential data-loss so as the following diagram shows we have data in lots of places.

Production Db Backup Flow

• Data spread across RAID 5 Array.
• Nightly raw backup to separate HDD (kept for 2 weeks).
• Nightly compressed backup (from raw) to external HDD (kept for 6 months).
• Weekly transfer of backup to office (with restore to dev/uat to verify).
• Physical burn to DVD for long-term storage.

This of course raises the problem of having sensitive data in lots of places to be managed but that’s another story.

So if you haven’t done it recently use this window before your holiday to BACK UP YOUR DATA!