Evaluation, statistics and OR

Business Survival Rates

On Monday I went with my husband to the local Chambre de Commerces for a half day introductory course on setting up in business in France.  Having done a similar thing in the UK a number of year's ago with enterprise agencies, we were pretty familiar with a lot of what was presented.  We were slightly shocked at the confusion as to which Chambre you need to register with, which depends on what you intend to do, and the fact that you could need to dual register, but that is a totally different story.

The session started with some statistics from INSEE on business survival rates - with the usual scare of perhaps half making it through to 3 years trading and only about 40% surviving to 5 years.  The lady running the course did comment that it was widely recognised that survival rates were favourably influenced by start ups receiving help from their adviser at the relevant Chambre.  Given that fact, another member of the course asked whether age affected survival rates, but that was not available.

Now that set me thinking - as you would expect.  There must be information collected on the age and gender of business start ups, not least through HMRC and Companies House, but how to look at the factors that make a successful start up?  If it is a one person business, that is relatively easy, provided the data is captured.  If that person takes on an employee, then it is also possible to allocate success to the manager (it may not be fair, but it is possible).

The difficulty is where you start up with some one or some many.  Can you allocate equal shares of someone's personal characteristics?  How do you measure influence?  This is what gets me to a certain extent about women's business support as you lose eligibility as soon as you work with a man, and yet men make up about half the population.

So, an interesting question, and one to which I don't have an answer, but if anyone has a nice big data set and a bit of money, I am more than happy to spend some time working up a few statistical models to see if I can sort out some determinants of business success based on age and on gender.

School League Tables

It's that time of year again - the school league tables for GCSE and A-levels are out.  As usual schools, education authorities and other interested parties are concerned that the tables are misleading, don't represent their achievements, put undue pressure on staff and pupils, etc.

It is one of the "joys" of performance statistics that however much you come up with what you consider a clean number to describe performance, someone will quibble with it or declare it unfair.  Currently GCSE passes are unfair because they favour girls, some independent schools do international qualifications that don't count for league tables, they don't take into account the prior performance of the pupils or the socio-economic context of the school.

Value added indicators have now been added - you hear less about them being unfair, but my quibble with them is that they are tortuous to calculate and how do you describe them to the average parent at the end of a hard day at work?

Then you get all the issues about branding young people failures if they don't meet the current base threshold (5 GCSE A* to C passes including Maths and English), and how schools fiddle the figures by not letting some poorly performing pupils take any exams if they will bring down the school performance figure.  Plus consternation from educationalists as to the validity of some of the qualifications that count towards that 5 good passes total.

Let's face it - there won't be a measure that pleases everyone.  But that minimum standard - 5 good GCSE or equivalent passes - drives public spending and policy on learning and training.  It should be the aim of society to furnish every young person that is capable with that minimum standard of education that will make the rest of their lives that bit easier, and the best time to do that is by the age of 16.

Critical Realism v Scientific Method

I have taken the opportunity of the Christmas/New Year break to start to look at critical realism, as part of a review of systems thinking and approaches.  This may brand me a somewhat sad individual, but the concepts and thinking involved really require a time of reflection and quiet rather than the daily urge to earn a crust.

In doing my doctorate in Operational Research and management methods, I used and discussed the scientific method.  This was the process to formulate an hypothesis, collect data and test it to determine if the hypothesis can be accepted or rejected, perhaps an odd feedback loop to tweak things a bit, and then once proved or not, write up and ideally use the resulting theory.

My research was undertaken in the late 80s and was academically driven so I was not aware of the newer concepts being discussed particularly around social sciences and geography that were appearing in less formal arena.  It wasn't until I was external supervisor for a geography postgraduate at Coventry University that the concepts of critical realism started to encroach on my thinking.

John Mingers has done a lot of work to see if critical realism is a valid alternative for OR to the scientific method.  Not all studies in OR are now really easy for the scientific method, particularly as the manufacturing and engineering base reduces, and messes to be looked at are far more likely to be people and services driven.

Two processes are proposed for critical realism that can be applied to an OR issue - DREI and RRREI.  DREI being Description, Retroduction, Elimination and Identification, while RRREI is Resolution, Redescription, Retroduction, Elimination and Identification.  Personally I prefer the RRREI version, but it is important to note what the phases mean.  Resolution here is not resolving the problem that you are looking at (part of the old scientific method, but a last stage), rather it is resolving the event or phenomena into its component parts and how they are related.  Retroduction is where you postulate hypotheses for mechanisms or structures that if they existed would give rise to the phenomena you are looking at.

Much of the work that I do in skills research cannot be simply explained or modelled, and it is here that often the scientific method doesn't quite help.  The critical realism approach could provide a better framework within which to work.

My main caveat to all this is that the client never really needs to see the philosophical constructs that one uses to work on their project and certainly never needs to hear phrases and words like critical realism, retroduction or transitive dimensions.  As even concepts like employer engagement can trigger long reports trying to define their meaning, the KISS principle has to be paramount outside of academia!