Skills, statistics and OR

Better Statistics education for all

The Royal Statistical Society have recently launched a report alongside The Actuarial Profession looking at the quality of teaching of statistics in schools and the role that statistics should have in eduation.  This is so important I am plugging it and their campaign Getstats shamelessly.

I don't know about others who use statistics in their job, but I am fed up with the knowing semi-humorous comments by the likes of Terry Wogan about "another government survey - ho ho" or "lies, damned lies and statistics".  That phrase was more a warning for professionals not to abuse the power that they had over numbers.

Similarly every latest health scare in the press that tells you not to do something that contradicts the last health advice because the likelihood of dying from something has doubled from 1 in 100,000 to 2 in 100,000 really gets on my nerves.  My health statistic?  The likelihood of you dying at some stage is 100% - some may say that there was a chap that didn't do it, and therefore it is less than 100% but I say he had help from another Source and that doesn't count.

So support a worthy cause, and take some time each day to be come a bit more numbers savvy, because otherwise you will be duped at every turn and will only have yourself to blame.

 

Evaluation - what is the purpose?

As budgets in the public sector are constrained by Government policy, one area that seems to be being squeezed is evaluation.  Recently a discussion with a former client over the progress with their new project provoked the following exchange:

- I gather the new project is proving interesting?

- O yes, very much so, but challenging too.

- How will you be evaluating it?

- We don't need to do that, as the requirement for evaluation by the funders has been dropped.

If evaluation was only to prove to funding agencies that work is being done to the schedule and cost that is set out in the contract, then it would be called audit.  There is a real purpose to evaluation, or rather a number of purposes.

Most importantly, formative evaluation, undertaken during the life of the project or programme is there to help those delivering the project ensure that things are going to plan, that there are no perverse effects to beneficiaries or to the market place, and that the systems they employ to manage the project are adequate and not overly onerous.  Lessons can be learnt and changes implemented to ensure that the full final benefits of the project are achieved.

Summative evaluation, at its most basic, provides evidence to all that the project achieved all (and hopefully more) that it set out to at the start, demonstrating the quality of the project manager or organisation in being able to deliver successfully and to time.  Better evaluation will also set out the impact of the project on its beneficiaries and to the wider community or economy to provide evidence for future planning and policy development.  It also should provide an opportunity for a full 360 review of how the project worked to inform future work patterns and personal development plans.

Evaluation is not a nice to have add on to a project, it is a key component of a project, without which wheels will be reinvented, or perhaps TECs and Business Links - who knows?

 

Forecasting and Planning for Extreme Events

Pity the planners and forecasters - their names are mud across Northern Europe as snow brings transport, delivery services and health services to their knees.  Why?  Because most of the time, their forecasts and plans save the public purse and private corporations huge sums of money, by not having stocks of materials, staff and plant on hand but not doing anything.  However extreme events show where the gaps are to cover the worst eventualities, and that is what we are facing at the moment.

When I did temperature demand forecasting for British Gas a long time ago, we planned for peak days - a 1 in 20 cold winter had to be accounted for with ease and on long term planning we looked at a 1 in 50 cold winter to ensure capacity, and a 1 in 50 cold day within the year for short term storage capacity (gas holders they used to be called).  We used weather data (daily maximum and minimum temperatures and daily wind speeds) going back to the mid-1920s to model the climate of northern England, and our reference really bad winters were 1928/29, 1946/47, 1962/63 and 1981/82, with 1946 and 1962 the worst we had to play with.  Some weather monitors are calling 2010/11 a 1 in 100 cold winter already, and we aren't half way through yet, so planning is difficult as it would appear to be more extreme than existing records.

Then think about the resources - in order to cope with a 1in 20 event, on a normal day you will have resources sitting in stock (in the case of things) or that can be diverted to other uses (people, but only if that is in their contract and competence), or just sitting around depreciating (plant that can be used for one thing only - snow plough attachments spring to mind for some reason).  Those who watch the money don't like that, and will look at ways to avoid having stocks of stuff that might not be needed (excess salt), people (one of the biggest costs to an organisation) and plant (surely we can borrow someone's snow plough?) sitting idle.

When that 1 in 50 event happens, there is no-one to take your call however urgent, the salt runs out after two weeks and no-one will lend you a snow plough as they are using them themselves or can't get it to you in time.  The result is a knock on to services to the public in general or to customers in particular.  When I worked at British Gas, we were not long out of being a public monopoly and still ran the business from well head to burner, so it was our pride and our obligation (on pain of dismissal) to ensure that there was enough gas in the system to meet all domestic and non-interruptible needs.  As the monopoly went (and after my time I hasten to add), domestic customers did get cut off and in events that were not as extreme as a 1 in 20 winter.  It was no longer an integrated system and cost savings had to be made.

Then add in the human effect - despite warnings to not travel unless absolutely essential, Parisiens decided that Christmas Shopping, visits to beauticians and hairdressers were essential and got stuck in their cars and had to be rescued.  You can warn, but people won't listen because it won't affect them.  Similarly, the most extreme weather event I dealt with at British Gas was in 1991 when we had a foot of snow, but it wasn't that cold.  Gas usage rocketed because of the psychological effect of snow.  If we could have found the records for lying snow in the weather archives going back to the mid-1920s, I am sure a 0/1 variable for snow would have improved our peak day forecasts. 

Nature is chaotic, and we can't plan for every eventuality (Haiti and the Boxing Day Tsunami are good examples), and if we plan for the very worst, there will be resources that are rarely if ever used.  The irony is that this cold snap has come at the start of a major UK austerity drive in public spending - to save money public sector staff will go, but who will drive the gritters and ploughs in future and who will pay for the salt/grit mix?  I'm not sure tax and council tax payers will want to find the extra cash somehow.

 

Welcome to my world, Mr Cameron!

"We was robbed" seemed to be the general opinion on the lack of success of the bid to host the World Cup 2018.  There were also comments about the unfairness of producing the bid with the best marks but not getting the job, and how the voting process should be changed, and the relative waste of spending money on producing materials as requested, when Britain was out of the process after the first bidding round.  It was all wrong and frightfully unfair!

You would think a politician would understand the vagaries of a voting system, but I doubt our Prime Minister, Prince William or David Beckham have had much experience in the last three years of public tendering processes.  This is why I say "welcome to my world!"

There are very few projects nowadays that are offered to external organisations (builders, consultants, advertising agencies to name a few) without some sort of tendering process, and while not of the scale of a World Cup bid, actually for a micro-enterprise like Plantagenet Consulting, the tendering process is a major expense.  First there is finding out about the tendering opportunity - BIS kindly provides the Supply 2 Gov website, but at about £250 per year subscription for anything other than the most basic view, this is not cheap (other portals are available I hasten to add). 

Relevant opportunities are either one off contracts or framework agreements.  For a business the size of Plantagenet Consulting, you can forget framework agreements - you can never get beyond the tickbox element for the various policies you have to have, as the fact that you have fewer than 5 employees, so you don't need them, is not a valid excuse in general.

So you are looking at an invitation to tender for a single contract.  If you are lucky, the potential client does not have a tendering portal such as Bravosolutions (again other portals are available) to battle against, and does not want a tender in hard copy (don't get me started on the challenges of the Mail system and courriers), but will accept an e-mail document. 

In order to produce a good tender, that meets the requirements of the potential client and represents you well, I estimate the minimum time required is 2 days, and the maximum viable time is two weeks, depending on the potential value of the project.  For a micro business that is between 2 and 10 days, and for an individual consultant, allowing for admin and such things we reckon there are 15 billable days in a month, but for a bigger project, you can see that every extra day is an opportunity cost of whatever your day rate happens to be.

If you are incredibly lucky, you get the project and away you go.  If you are "lucky" you are shortlisted, and get to go to all the additional expense of attending a beauty parade with a presentation of how you and yours would best meet the customer's needs.  You are never recompensed for attending these parades, even if it becomes clear that you are only making up numbers.  Sometimes your pitch wins, sometimes it doesn't.  And in at least 50% of cases, your excellent tender will be rejected (and realistically it will often be a higher per cent of cases), and sometimes you will get useful feedback and sometimes you won't.  If you do, it must be valued and pondered over.

Such is the world of the consultant, occasional highs leading to valuable work and rewarding new relationships, but so often counting the not inconsiderable cost of being one of those who didn't win.  Often as well, you feel that actually you did a good job, and deserved a better hearing, but as with the World Cup bid, it just didn't happen.

Mr Cameron, if you have read this, consider the amount of wasted effort and money in ALL unsuccessful bids around the country in a single year.  Your bid's failure was a bit more public than most, but you expect public suppliers to go through that self same process and beauty pagent, and invest proportionate costs.  In your failure, you found a peer group to relate to!

Evidence based policy?

Those of us in the consultancy game, particularly around evaluation, like to think that when we write conclusions and recommendations, someone listens to them and does something about them.

I have experienced the glow from time to time of having my findings reflected back to me - most recently on a wonderful project called Business Brokers in North Staffordshire.  There, with a colleague and working under the Inspira Consulting banner, we did an extensive mid-term evaluation of the programme to help them meet their contracted targets and also help them think about the future beyond their funding contract and how they could ensure the sustainability of the service.  In the last three months, we have gone back and done the final evaluation of the funded programme, looking at contract performance and also at how our recommendations were followed up.  They had done extremely well in achieving their contracted outputs and have found ways to maintain the service, albeit at a lower level, but then at the moment, they have less funding.  Whether it was polishing our egos or not, they did say more than once, "following your advice we looked at this" or "we have done this".

However in education matters, this is often far less the case.  My impression has been that decisions are made as knee-jerk reactions, without letting the previous set of changes embed themselves and demonstrate worth, and due to the loudest set of shouting in the media.  I am not totally sure how often Apprenticeships have been re-branded since they were reintroduced in the late 90s, but the myriad brands have not helped employers or prospective employers identify with the product.  Similarly, new qualifications are brought in and then dropped at the drop of a hat - who remembers GNVQs?  And if you have one, how do you explain it to people now?

It seems that my impressions are not totally without foundation for once as well.  This article on the BBC website heralds a report that says just that - politicians do not look at the evidence when changing education policy and practice.  For those of us who have recently been following the mantra of "evidence based policy making", this will be something of a shock, but then - are politicians going to take any notice of this report either?

Are Quangos truly evil?

A new Government and new ideas - for some it is a time of new hope and excitement as people get to grips with their new responsibilities and start to make their mark on the country.  For many, many more it is a frustrating time as we try to second guess what the new Government will do and wait for permission to carry on with the process of running the country and its key services.  The rhetoric is amazing but what does it mean for the person in the office?

One of the big new ideas appears to be to remove Quangos and get rid of all that awful red tape that has been put in place in the last 12 years of Labour.  Bizarrely, quangos and all that red tape was there before too, but had different names.  Local enterprise boards that are being contemplated were Training and Enterprise Councils, and their Business Links.  All that red tape about learners was the Individualised Student Record and the Annual Pupil Census and the TFW8 (Training for Work form 8) instead of the Individual Learner Record and something too secret to mention from DWP (Department for Work and Pensions).

The reason some of these quangos at least are out there is because the huge departments at Moorfoot and Longbenton among others have been scaled down pretty drastically, but the work remains, it is just done by an organisation with a contract and some other responsibilities thrown in like being nice to other organisations with a related remit.  But as it is a separate organisation, you can gradually over time make working conditions intolerable and force people off work with stress and still feel good about Civil Service bashing.  The unions are fragmented and unable to help staff to regain self-respect and a reasonable work-life balance or indeed a working day of less than 9 hours, excluding travel.  Many of these individuals are therefore looking forward to the time when their last vestige of benefit in working for the public good kicks in - their frozen Civil Service pension.  Let's hope they are allowed to keep it.

The bid idea of removing red tape and oversight and leaving schools, colleges and hospitals free to determine their own priorities and spending only works until one of them, or someone related but removed from them, slips up and suddenly there has to be regulation to control the use of public funds.  Until the MP expenses scandal, Colleges were being encouraged to drive up quality and standards themselves and through peer review. When the scandal broke, self-regulation was consigned to the scrap heap - if MPs couldn't be trusted to behave, how could Colleges possibly cope with the responsibility of improving themselves?

So the cycle starts again, and the big ideas will be touted round.  Public servants will lose their jobs, small consultancies will go through hard times (this NEVER happens to the Big Five), lots of new brochures and policies will be printed and lots of red tape repealed or banned, until the next slip up and misuse of public funds is spotted and it all builds again.  In about a decade I will be able to recycle this article and sprinkle just a little more cynicism into it, having been round the loop again.

Nasty developments

Over the past month I have noticed a nasty little tendency cropping up on my blogs, and if it happening to me, it is happening to others as well.  I have to say it appears to happen less on this thread than on my other one, and is I think something to do with the level of traffic a site receives.

So what is this little gremlin?  It is Spam, which appears to congratulate the author for being so altruistic as to share information, but as it is badly spelt (often coming from Masters Dissertation students allegedly - but I think not), and sometimes includes a reference to a website to visit, it is obviously not from the heart.  Particularly as the spelling mistakes are the same for many different authors.

I have zero tolerance towards spam e-mail, spam contributions to my weblogs, and to cold sales calls placed to any of my phones, and that includes Skype.  Working remotely, working as a freelancer is difficult enough at the best of times, but this sort of rubbish steals my time in that I have to remove it, while giving me concerns as to whether it is causing any other damage.

If anyone who does do spam, spam contributions or cold calling sales is reading this - please think about what you are doing and imagine you are trying to do something very important yourself, but are impeded by this activity, or your wife, mother, sister, girlfriend is getting this hassle.  Make a New Year's resolution to do something valuable with your life instead of this!

Participation in Learning: half full or half empty?

Such is the volume of news reporting and issuing of statements, that two stories were published by the BBC within 24 hours of each other that give the average reader no hope of understanding how well the UK is doing in terms of skills.

First was what should be an extremely good news story - more than 1 in 4 of the entire population of England is in education and training.  While that will underestimate the number of people actually learning, it is a tremendous statistic.  I could spend a happy while picking holes in it, with particular reference to the Unique Learner Number, but that would be nitpicking.  The expansion in participation in education and training is a marvellous thing and some of the statistics quoted tell a marvellous story.  Read the story for yourself here - but if you don't have time, know that in the late 1950s about 10% of school leavers achieved 5 O-levels while today the minimum performance level for a school is 30% of 16 year olds achieving 5 A*to C GCSEs, with some Grammar schools achieving 99% or 100% of this and many bog standard schools achieving at least 50%.

The writing of the story tries hard to make this a bad news story by asking how all this education is going to be paid for, but "as any fule kno" gaining skills and qualifications pays for itself both for the individual and for the economy as a whole, so stop asking silly questions.

Where does the downer come from then?  A day later the University and College Union points out that in their analysis the UK is falling being in the qualifications race and that there is not enough investment in education and skills.  While the UK has made tremendous strides (according to the first article), any progress is trashed as inadequate and lower than our competitors in the second.  We are allegedly behind such economic dragons as Portugal and Greece in the qualifications race and must do better.

Two things strike me about some of this latter report - is the quality of education the same, are they comparing like with like, is graduateness in the UK as good, better or worse than graduateness in Greece or Portugal or the USA.  At one time, it used to be said a UK degree was as intellectually challenging as a USA masters, while the West Germans used to make the same unfavourable comparison between a German degree and a UK degree (I never believed that however).  Secondly, if Portugal and Greece have such better education and qualification rates than the UK, why are the PIGS countries of Europe regarded as the drag on European economic growth?  (PIGS by the way - Portugal, Italy, Greece, Spain.)

Another happy thought is that the financial services companies that sparked off the global slowdown employed some of the brightest mathematical PhD students from the best universities - and look where that took us all!

European Milk Strike

Normally I keep my private passions away from this blog and discuss them on my other one, but I feel this topic is something more linked to work thoughts as well as home life.

Did you know that dairy farmers across Europe, but not Denmark and probably not the UK, have been taking industrial action in protest at the farm gate milk prices unilaterally set by the dairies and other milk transformers?  It has been going on for quite a while and with some dramatic protests.  I reported on local action in this posting, while a European association has been set up, complete with website, to press the case, and recently nearly 1,000 tractors made the image you can see at this site in a field near Annencis to make their point.

Free marketeers would argue that there is excess supply and that therefore the low prices offered to farmers at 8 cents a litre below the cost of production are just a manifestation of the laws of supply and demand.  The fact that prices in supermarkets for milk, cream, butter, cheese and other milk related products have not fallen at all doesn't seem to come into the argument.  Sharing the margins from the consumer or cutting the prices to the consumer have not happened - so the market isn't really working.

It was also not that long ago that prices for milk products rocketed because of excess demand in the system before the credit crunch (although I can't for the life of me think who takes out a mortgage to buy yoghurt).  Supply is not something you can turn on and off that easily however - cows take a few years to breed and get to milking while the milking parlours farmers have to have to extract and store the milk need to meet ever stricter hygiene rules.  Cows also need to be fed, even if no one is buying the milk, and create a landscape that is cherished by all, not just the farmers.

Finally, there are surely some products that are essential to the nation - clean air, clean water, basic foodstuffs, energy.  If all our milk were to come from emerging nations, but then someone was able to pay more, our supply would be cut off, and we would be without the means of production ourselves (look at parallels in coal, gas, steel, aluminium, copper).

Personally I am prepared to pay a bit more for my dairy products, knowing that I am keeping an economy and way of life going.  Having said that, I try to make sure I buy my milk and milk products from someone who is also a producer or at least doesn't profiteer from the producer.  Bizarrely, such a purchasing policy also means I get to consume nicer products!

Who changed the definition of NEETS?

I have spent years monitoring NEETS - those not in education, employment or training.

First time round, we called it the Success Index and it was run in Coventry and Warwickshire and showed the first destination post-GCSEs for young people aged 16 or so in the sub-region.  It was published towards the end of the autumn term and was used as a measure of success in inspiring young people in the area. 

The original definition was that a young person had to be in education or training with work, and we used to have a lot of debate with Connexions about a job without training - due to really tricky semantics.  If a young person was on a training programme with Marks & Spencer or AXA, they often weren't counted as being in a positive destination as the training wasn't to a recognised qualification.  However the unqualified young man who was helping his Dad out as a scaffolder was a tricky one - should we grass them up to the HSE?

Then the concept of NEETS came up, so that whole debate was put on the back burner, as you could be in education, employment or training, and it was a positive outcome.  Then people started to get clever and think about what happened at the end of year 12, so ever more sophisticated methods for tracking young people were needed, and Connexions case workers were at the front of collecting the data.  Now when we just did the Success Index, we did find that people who went to public school for their sixth form education ended up as non-positive outcomes, as they were generally lost to the state careers system - is that the same for NEETs?

Today I find that there is a whole new category of NEET - actually we used to call them unemployed - young people aged 19 to 24 not in education, employment or training.  Here we get into seriously tricky water on definitions.  What about young mothers?  What about unpaid carers? What about unpaid postgraduate academics?  What about school leavers and graduates on gap years?  What about volunteers?  As one gets older, the range of options and life choices that are totally valid but don't sit comfortably within the categories of in employment or in education or in training get ever more numerous. 

Why call these people NEETs?  It takes the strength away from the 16 - 18 definition.  These people are unemployed or inactive.