Evaluation, statistics and OR

Unemployment Measures

Today in France, the domestic headlines are dominated by what appears to be good news - the unemployment rate in February fell to the lowest level seen in 24 years, 8.4%.

You would think that joy would be unconfined and that M. Borloo who has overseen this steady fall for the past 3 years would be being feted as a national hero.  Sadly, as we are in the run up to the Presidential elections in France, and M. Borloo is firmly in the Sarkozy camp, 11 other candidates and interest groups, not to mention the Unions are all kicking up.  Their argument is that the figures do not represent reality.

Now the agreed standard method for calculating unemployment across Europe is by survey methods - hence the UK having two main measures - the Government preferred based on the Labour Force Survey, and the Claimant Count, which is based on numbers signing on.  The difference is that the survey method includes those who are actively looking for work, but are not entitled to benefits or who have not signed up for benefits.

All surveys include some level of error, and potentially include some bias as well.  The Labour Force Survey has been criticised for accessing those that are easy to get to, and missing out others such as the homeless, or those who travel a lot for work, but at a national level, the error rate introduced in this way is small and not significant.

The argument in France currently, is that while unemployment (counted as those actively seeking work of any type, including temping or part time) is going down, and the check figure of employment is also going up, the quality of the jobs is not what it was.  It is argued that many are doing jobs that earn less than the minimum wage or are not full time (35 hours per week only in France).  In particular, the CGT - the most militant of the unions it seems - say that there are not enough jobs being created, apart from in Construction and my personal favourite, "the nebulous area of personal services".

France is not really ready for the 21st Century at times - in the developed world, job creation will be in services and particularly the personal services, as manufacturing can be done more cheaply elsewhere.  However the strangely Seventies figure of Bernard Thibault at the head of the CGT will not go down without a fight.  That is one of the biggest problems facing France today.

Ken Bowen

This morning, when checking my e-mails, I received a note from the OR Society letting members know that Ken Bowen had died - I can't speak for the rest of the Society, but for me the OR world is now diminished.

Ken was my external supervisor for my PhD at Hatfield Polytechnic between 1986 and 1989 and he was an inspiration for my working life.  I am proud to have a copy of his PhD on my bookshelves and to have worked with him twice.

His knowledge was wide ranging and profound.  Obviously he was an amazing mathematician, had experience of defence issues, teaching methods and all aspects of OR, but in addition he played cricket until quite recently, was a humanist, and took a lively interest in art and architecture.

He taught me to look for models and heuristic rules in the world around me and to keep thinking in all situations.  One example was when he drove round the M25 from Guildford to Hatfield on one of his regular visits - it was a windy day, and once he arrived I took him to the Senior Common Room for coffee.  There he described to me how he had developed a model for identifying when he was going to face the worst gusts of wind and to be prepared for them, based on topology and lorry numbers.  It wasn't sophisticated, but the lesson was clear in terms of the way one should look at the world around.

Another instance was when I went round to see him at Royal Holloway and Bedford New College at Egham.  We spent the morning discussing how my research was progressing and how I could best gather soft intelligence in a structured manner.  Then at lunch time he took me to the mansion, so I could see the red brick version of Chambord, as he knew my love of France and Europe, and finally to see the hoard of fine art that the College had, including pictures by the "other James Holland".  This was not my father but the Victorian landscape painter, as I had once told Ken that I had not seen any of this other Holland's work.

As a teacher and mentor, in my view he was second to none.  I am sure that other former students and colleagues will feel that a wise and cheerful guide and support has been taken away, and that OR may not be quite as rich in the future as it has been in the past.

Recognising the importance of learning languages

What fabulous news to come out earlier this week - modern foreign languages are to be compulsory in primary schools in England by 2010!

As I have said before in my entry on "I want to communicate!", the power of learning languages and being able to communicate with others who do not share your mother tongue is immense. 

I was e-mailed privately after that blog by a friend who lives in France.  Her deaf niece from England was visiting them, and the children went out to play with French neighbours, and the niece came back speaking French, despite her communication difficulties.  My friend couldn't reiterate strongly enough the importance of the desire to communicate and the pleasure that comes from making that breakthrough in another language.

In my village in France, I have joined the walking group that meets on a Sunday for a 7 to 10 kilometre hike in the local area.  At first the group leader and the other walkers were a bit wary of speaking to me, and either did so very loudly (that endearing thought that the louder you speak, the more likely you are to be understood), or in very simple phrases or words (Look! Deer! Mud!).  Now they have got used to me, and realise that I speak French, and with some I hold quite intricate conversations and jokes, while others realise that I am not a swamp monster, just because I am not French.  It makes for a wonderful feeling of community, and I feel much more part of the village.

While there is a generation of young people in English schools that have been let down but being allowed to drop or never learn languages, future generations will be much better prepared.  Like I said at the start - fabulous news!

The importance of Evaluation

Keeping an eye, as I do, on public sector tenders that are being put out to competition among consultants such as me, it seems there is a major push on evaluation currently.  This should not be a surprise, as for the public and not for profit sectors, evaluation should be a key tool for a number of reasons:

• ensuring the project is doing what was intended
• monitoring what inputs are going into the project and what outputs are being generated by those inputs
• determining the impact of those outputs
• demonstrating the sound deployment of scarce public resource for the good of the target community

In the major public funding rounds, and I can talk from experience of ESF, ERDF, SRB and NRF, from the very moment of winning these funds, there is a clear expectation that the application of the funds will be monitored and evaluated.  (Apologies for the acronyms - the first two are funds from Europe, SRB or the Single Regeneration Budget is mostly complete now, and NRF is the Neighbourhood Renewal Fund).  It is also true that the best of the funding recipients build the evaluation process into their projects - allocating funding up front, being clear on what inputs, outputs and impacts will look like, and setting up internal or external processes and resources to ensuring the evaluation can happen.

For a consultant like me, it is great to work with such organisations, as they are clear in what is required, and with the evaluation included in early stages of the project, there are no nasty surprises at the end, when it comes to justifying spend.  It is also more cost effective to the public purse, in my view, with evaluator and project group working together through the process to ensure data is collected in a timely fashion, that something can be done if the project is going off track, and there is no panic at the end to prove the benefits.

The evaluations that, if I am honest, make consultants a chunk of money, are those that are tagged on at the end of the project in a panic, once the funds have been dished out, and key players have moved on to the next big thing.  The ones where a project manager is reading through their notes of the programme, and gets that awful sinking feeling that there was a final stage to go through - sounds familiar?

Such add-on evaluations, in my view, are never as robust or insightful as evaluations that are part of the programme, but should be able to show the success or otherwise of the funding.  Finding the required data and outputs can be challenging and time consuming, with long programmes, the initial ambitions and aims may be difficult to unearth as players and practises move on, and there is always the excuse that the essential information can no longer be found due to filing errors.

Best practise is to build in evaluation from the start - save time, money, resource and worry by doing that; or be prepared for a painful process at the end of the project!

Learning does change lives!

Usually my posts are triggered by something I have read on an electronic bulletin or on the BBC website or in a newspaper - this week is different.  I am still buzzing from interviewing people who have taken part in Skills for Life learning, and I have to share what I heard.

I was visiting a company that had been using a number of business improvement tools to ensure they were competitive in their field.  One of the things they had done was to look at levels of literacy and numeracy among ALL staff, to ensure there was a minimum standard of level 2, which they hoped would lead to improvements in quality and prevent any health and safety issues.

Using the Skills for Life model, all staff had undertaken tests (unless they could produce relevant certificates), and for those who showed a learning need, training was provided on site.

I had the immense privilege of interviewing people who had recently achieved their level 2s, and talking to them about how they had felt about the learning and what it meant to them to have achieved.  They had a superb tutor - they couldn't praise her enough, for her support, for her attitude towards them, for her professionalism.

They had also changed their lives - increased confidence, new doors opening to them, passionate about their work and their employer, and with a thirst for learning more.

As I say, it was a marvellous experience to talk to them and hear their successes and their hopes for the future.  My worry is that while I know they will received further excellent employer supported training, the learning they want to do for themselves may not live up to their expectations.  It could be the price of the learning, the availability of what they want to do, the attitude of the tutor or the reactions of their fellow learners - but I dread that something takes the shine off their new found passion for knowledge.

Who can afford learning?

My previous posting raised a comment about those who return to higher level learning as opposed to those who could be perceived to follow the learning route to avoid starting a career - perhaps an overly jaundiced view!!

However there is a point here about how does an individual make the choice to invest in learning?  The investment is not just money, particularly for those more advanced in years, who have a job, a family and are perhaps involved in community activities as well.  Surveys show that a barrier to learning for individuals is that they don't have the time - and it is easy to look at this as just another excuse.  Make the time might be a reaction.

I'll use an unnamed example who my husband worked with a number of years ago.  His educational attainment wasn't over Level 2, but he was in work.  His wife wasn't as they had four children to look after.  He worked a full shift every day, overtime whenever there was any, Saturdays whenever the employers required it - all to ensure he could pay his mortgage and feed and clothe himself, his wife and their four children.

In this case, how would X afford the time for learning, let alone the money?  He might in future get a slightly better paid job if he had level 3 qualifications, but the imperative of "now" for his immediate needs outweighs that potential future income stream.

Selling learning to X in those circumstances requires a whole different understanding of learning, and a new method of delivery.

Adult learning must be the main story

On the 9th February, the Learning and Skills Council (LSC) issued a press release about some recent research on adult learning ("One in three adults go back to learning").  The findings of the YouGov poll in December showed that 30% of adults wished they had achieved better qualifications when they were younger and that those that didn't work hard at school were more likely to be unemployed now.  It also showed that 35% of adults had returned to learning to gain more qualifications, and if those qualifications were achieved, this has had a positive effect on their careers.

All of this intelligence is good stuff and provides useful pointers for how to sell the benefits of learning to the vast numbers of adults who do not recognise that they have done any formal or informal learning since leaving school.  Sadly this is not the way it is being used.

The supporting quotations from the LSC and ministers went on to highlight the importance of staying in learning for as long as you can when you are young, and the wide and attractive range of learning opportunities and funding that are available, primarily to those aged under 25.

Both the LSC and the Department for Education and Skills (DfES) are obsessed with young people and their qualifications, yet the Leitch report on skills - endorsed by both organisations - points out that the majority of the workforce for 2020 has already left compulsory education. 

It has also been a mantra of the skills and vocational education sector for at least 10 years now, that there is no longer a job for life, and that individuals will change career up to 5 times - entailing gaining new skills and retraining along the way.

So why - given the excellent news that 35% of adults have chosen to go back to learning in later life, and that it has benefited them - must the policy leaders push this message with young people?  Could it be that if adults picked up the message and decided to act upon it, the system would not be able to support them?  Or are there more rewards from Treasury for involving young people in learning?

To meet the targets set out by Leitch for 2020, everyone needs to be involved in learning, not just selected segments of the population.  Getting this message out to everyone and helping everyone access learning must be the objective for DfES and the LSC.

Who's afraid of Bird 'Flu?

If government statistics are to be believed, 11 million adults in Britain lack numeracy skills.  What does this have to do with bird flu?

The scientists and experts say that the latest outbreak of bird flu in Suffolk poses no immediate risk to public health, that it is exceedingly unlikely that any human will get sick as a result of the outbreak on that one turkey farm, and that globally the numbers of human cases are very, very small indeed.  They also say that the public are safe to continue eating poultry.

How are those with poor numeracy skills going to evaluate statements like exceedingly unlikely or very, very small?  Winning the Lottery is also exceedingly unlikely (14 million to 1 or similar odds), but millions still make that bet regularly, because "it could be you!" 

I feel pretty confident in predicting that there will be a fall in the sales of prepared poultry products, because although you cannot catch bird flu from this source, people still think you can.  I also believe that if people read the fairly out of date information on the BBC website, there may well be a large number of abandoned cats in East Anglia, and there will be concern about travelling in the East Anglia region.  Again, people are not sufficiently familiar with numbers to understand the scale of the risk.

Sadly, there will be less concern over the numbers cited about the Suffolk farm itself - 160,000 birds in 22 sheds, on average between 7,000 and 7,500 birds per shed.  Nor is there likely to be much concern over the statements that on modern poultry farms, feeding is done mechanically and involvement of farm workers with poultry is minimal. 

To me, the remote chances of anyone getting bird flu from this outbreak are too small to even consider.  The real issue this story has highlighted to me are the numbers involved in factory farming, and the lack of understanding about how our food is produced.

Graduate Skills Shortages

Very early this morning a story sneaked onto the BBC Education website, to the effect that while demand for graduates is increasing, 40% of employers aiming to recruit graduates were facing skills shortages.

Now I am used to sifting through employer surveys, looking at reasons why vacancies are hard to fill, and the standard responses for generalist surveys are always that potential recruits lack technical skills or that they lack suitable experience.  The former is usually a starting point for looking at skills shortage vacancies, the latter generally brings a response of how can people gain experience if you won't give them a chance?  Indeed sometimes when the experience issue is probed, what an employer is looking for is so specific to the company that you want to take recruiting managers out and give them a good talking to!!

This survey is rather different, in that it is looking at people being recruited into their first post graduation job, so you would expect those recruiting to be less worried about lacking experience.  The survey results also highlight that candidates are normally academically proficient, which takes care of the technical skills as well.  What they lack is soft skills.

To a certain extent, I would expect a new graduate to be a bit raw on soft skills, despite the key skills curriculum elements that may be included in school or college education, before higher education.  I think I would expect my potential graduate to lack team working skills to an extent (unless anything could be demonstrated from extra curricular activities like team sports, scouts, voluntary work and so forth).  I might also expect that they would not be the world's greatest communicators, unless they had done a communications degree.

The thing that brought me up short was that the soft skills were considered to include verbal and numerical reasoning.  Now to me, verbal and numerical reasoning are essential concepts of "graduateness" - no one should be able to pass a graduate level course without verbal and numerical reasoning (OK - some courses will be heavier on one than the other).

I was involved with one business degree course, where the university was considering making quantitative methods optional - and I was appalled, and said so very firmly.  Perhaps this is a first sign that numerical reasoning is no longer considered important in business, but to me, that is the scariest thought I have come across in a long time. 

Think about it - where are numbers not important in business?  Cost control?  Retail activity?  Wages?  Tax?  Production efficiency?  Activity monitoring?  I wouldn't be interested in a graduate that lacked numerical reasoning skills - would you?

Mumpreneurs and women in business

The Government has a Women's Enterprise Taskforce, with the Australian Glenda Stone as co-chair.  In an interview with the Daily Telegraph, Ms Stone has said she wants to take women's enterprise from mainly cottage industry status to something that creates wealth and jobs, and makes a significant contribution to the economy.

Ms Stone has a mantra of "profit before passion", while it appears she finds the desire to do something emotionally and psychologically rewarding unhelpful unless it can also turn an exceptionally healthy profit.  She wants to see business support for women targeted at those who see their business as more than just a hobby.

This article got to me in a big way, and annoyed me.  Don't get me wrong - I am and have always been a fervent feminist, but one of the things that gave feminism a bad name was women bashing other women, and trying to be ballsier than the men they felt in competition with.  Here is a woman saying that in order to be a women entrepreneur, as opposed to a woman dabbler, you have to suppress those ambitions and aims that come from your nurturing and supporting side.  I'm pretty sure that is not the case.

While it is true that a substantial number of women start a small business as a way of getting back into working life following a career break to have and raise children, there are also a lot of women who go into business who do not have children, or who have not taken a career break.  Is money really the driver for every entrepreneur - male or female?  No.  Go back to Maslow's hierarchy of needs!!

Everyone needs to feel safe, warm, with adequate nutrition - these are basic rights and needs, and unless they are satisfied, they will dominate the activity of the individual.  But after that, why do we strive for more, to do better?  This is where things like quality of life and self-actualisation and self-worth come into play. 

For a number of years I was a public servant - I put in the hours and the miles to do what I thought would have a positive impact on the wellbeing of my fellow citizens.  I became a woman entrepreneur when I could see, with regret, that actually what I was doing was not making any difference, that I could feel little pride in what I was doing, that the long daily commute was doing nothing for my health or my personal life, and that most of the time away from work, I was too drained to make a positive contribution to my family or my community.

While I make a profit, I don't think I fit Ms Stone's entrepreneurial ideal, in that my way of working and what I do is designed to give me a feeling of pride, of a job well done, while at the same time, having the work/life balance right.  In addition, I do all that without the excuse of being a Mum. 

The other thing that worries me about all this is that it focuses on businesses owned by women - once you get out of the sole trader/company with single director model, there is a high likelihood that the business will be owned by a mixture of men and women.  Will the introduction of one man into the ownership or management mix of a company make that business more likely to succeed, be profitable, aim for growth, achieve growth?  I am not sure it will, but that one man will make the business invisible to the women's enterprise taskforce.