Evaluation, statistics and OR

The difference between audit and evaluation

At Plantagenet I am currently involved in a large number of evaluations - mainly through other consultancy companies.  The reason - apart from the fact that I am good at it - is that a large chunk of European Funding comes to an end in June 2008, and funding and delivery agencies are having to make sure they have evaluated how they have spent the money.  I don't know if it is the same across Europe, but in the UK they are keen to tick all the boxes and demonstrate they have done what is required.

These final evaluations are interesting as they show how the project or programme developed and how well the aims and vision of the work was delivered.  Ideally they are the moment to sit back and reflect that the lives of so many people have been affected for the better in some way.  In addition they can identify what worked best in delivery terms so that next time there is a purchasing round, people know what to look for.

I prefer this in many ways to the formative and interim evaluations where you are following a project as it runs, identifying what is going well and trying to get things that are not going so well sorted out.

Where things get a bit scary is where determining the difference between a final evaluation and an audit.  From the audits that have been done to me, it seems that you are looking for the things that don't add up, the things that weren't valid and need to be repaid, tracking the missing paperwork, identifying the gaps.  My brother used to do some of this in his role as a loss adjuster, and said it could be fascinating when you knew there was a problem, but dull if done for too long.

With an evaluation, I like to be able to follow the process that a project or programme has taken - follow the story if you like.  I also like to be able to delve into the figures on inputs and outputs to put a numeric story to it - and this is where my skills are mainly focused and is what gets close to auditing I suspect.  But for me the bit that makes evaluation a nice type of project, is the chance to go out and speak to delivery agents and similar and hear the stories of where things have worked.  I can't give examples here - mostly these things are very close to the project's aims and very personal - but they do give you a warm and fuzzy feeling when you are writing up the final report.

Do auditors feel warm and fuzzy when they have found a mistake or when they can find no issues with a project?  I guess that depends on the auditor.

Happiness versus marketing and the media

Today's e-mailed Expat Telegraph brought me a story with the headline "We're wealthier than in 1987, but no happier", and said that between 1973 and 2006, people's satisfaction levels have hovered around an average of 86%.  Wow - that tells me a lot.

While ONS who came up with the stats behind this headline talk about an Easterlin Paradox, in which the relationship between income and happiness declines after a certain level of wealth is reached, isn't there a lot more to it?

There is the old adage that "Money can't buy you happiness", although I guess it makes being miserable a lot more comfortable.

The media seem to rule our lives nowadays - or perhaps I am just getting older - and they very rarely seem to focus in on good news stories (apart from Carla Bruni's legs perhaps).  If a positive statistic is mentioned, who ever came up with it is obviously lying; if someone is good, we find a way to shoot them down; if it is a nice sunny day, it is down to global warming; if food has taste, it will give us cancer; if I enjoy a glass of wine, I am obviously a lush heading for every ill on the planet.  Against all this doom and gloom, you have to be a bit special to keep happy.

Then there is advertising, blatant or covert, which is designed to make us feel less than happy with our lot.  There are many models, but I will go back to Simon and March who have a cycle of purchasing that works for most things - you are dissatisfied with the status quo, so you do something to improve it (buy a consultancy project was the option in my research, but it could be buy a new car), once you have gone through that process, you are happier for a while, however a parallel process is going on where you also find that your aspirations have been raised as well, so your satisfaction with the new status quo is lessen (that car isn't quite as fab as the Jones') and round the cycle you go again.

That new car with better fuel consumption, that bigger house with the extra bedroom that you will never use and the garage you won't put your car in, that new TV or computer or MP3 player that is out of date within a week - you are always being told that something better has arrived.  If you don't listen to the marketing chaps, then they take away what you like (Windows 98 for so many people) and you have to find an alternative.

So in many ways, I am not surprised that we aren't all happier than in 1987, when all we really had to worry about was the awfulness of the mullet haircut and what would the new Micheal Jackson video be like.

But am I 86% happy?  If I were materialistic, I'd be surprised if I were that happy, but I don't much care about how new my car is, how many functions my mobile phone has that I can't use, so I hope the marketing has failed on me!