Categoría: Behavioral Economics

Air safety: When statistics are used to kill the messenger

Long time ago, I observed that big and long-range planes -with a few exceptions- always had a safety record better than the one more little planes had. Many explanations were given to this fact: Biggest planes get the most experienced crews, big planes are more carefully crafted… it was easier than that: The most dangerous phases in flight are on ground or near to the ground. Once the plane is at cruise level, the risk is far lower. Of course, biggest planes make long flights or, in other terms, every 10 flown hours a big plane could perform, as an average, one landing while a little one could land 10 times. In a statistical report based in flown hours…which one is them is going to appear as safer? Of course, the big one. If statistics are not carefully read, someone would started to be worried about the high accidentability rate of little planes, if compared with the big ones.

Now, the American NTSB has discovered that helicopters are dangerous: http://www.flightglobal.com/news/articles/ntsb-adds-helicopters-ga-weather-to-quotmost-wantedquot-394947/  and the explanation could be similar, especially if they address the HEMS activity: Emergency medical services are given an extremely short answering time. That means flying with machines that can be cold at the moment of performing an exigent take-off, for instance, very near to a hospital or populated places where they need to make an almost vertical take-off. Once they are airborne, they need to prepare a landing near to an accident place. The place can have buildings, unmarked electrical wires and, of course, it can be from fully flat at sea level to a high mountain place. Is the helicopter risky or the risk is in the operation?

Of course, precisely because the operation is risky, everything has to be as careful as possible but making statistical comparisons with other operations is not the right approach. Analyze in which phase of the flight accidents happen; if the pilot does not have full freedom to choose the place to land, at least, choose an adequate place for the base. Some accidents happened while doctor onboard was seeing that they were very near to an electrical wire and assumed that the pilot had seen it…all the eyes are welcome, even the non-specialized ones. Other times, non-specialized people asked and pressed for landing in crazy places or rostering and missions are prepared ignoring experience and fatigue issues. That is, there is a lot of work to do in this field but, please, do not use statistical reports to justify that by comparing things that are really hard to compare.

Does CRM work? Some questions about it

Let’s start by clarifying something: CRM is not the same that Human Factors concern. This is a very specific way to channelize this concern in a very specific context, that is, the cockpit…even though, CRM philosophy has been applied to Maintenance through MRM and other fields where a real teamwork is required.

Should we have to improve CRM training or is it not the right way? Do we have to improve indicators quality or should we be more worried about the environment in which these indicators appear?…

An anecdote: A psychologist, working in a kind of jail for teenagers had observed something around the years: The center had a sequence known as «the process», whose resemblance to Kafka work seemed to be more than accidental, and inmates were evaluated according to visible behavior markers included in «the process». Once all the markers in the list appeared, the inmate was set free. The psychologist observed that smarted inmates, not best ones, were the ones able to pass the process because in a very short time were able to exhibit the desired behavior. Of course, once out of the center, they behave as they liked and, if they were caught again, they would exhibit again the required behavior to go out.

Some CRM approaches are very near to this model. The evaluator looks for behavioral markers whose optimum values are kindly offered by evaluated people and, once passed the evaluation, they can behave in agreement with their real drive, whatever it is coincident with the CRM model or not.

Many behaviorist psychologists say that the key is which behavioral markers are selected. They can even argue that this model works in clinical psychology. They are right but, perhaps, they are not fully right and, furthermore, they are wrong in the most relevant part:

We cannot try to use the model from clinical psychology because there is a fundamental flaw: In clinical psychology, the patient goes by himself asking for a solution because his own behavior is felt like a problem. If, through the treatment, the psychologist is able to suppress the undesired behavior, the patient himself will be in charge of making this situation remain. The patient wants to change.

If, instead of speaking about clinical psychology, we focus in undesired behaviors from teamwork perspective, things do not work that way:  Unwanted behaviors for the organization or the team could be highly appreciated by the one who exhibits them. Hence, they can dissappear while they are observed but, if so, it does not mean learning but, perhaps, craftiness from the observed person.

For a real change, three variables have to change at the same time: Competence, Coordination and Commitment. Training is useful if the problem to be solved is about competence. It does not work if the organization does not make a serious effort to avoid contradictory messages and, of course, it is useless if there is not commitment by individuals, that is, if the intention to change is not clear or, simply, it does not exist.

Very often, instead of a real change, solutions appear under the shape of shortcuts. These shortcuts try to subvert the fact that the three variables are required and, furthermore, they are required at the same time. Instead of this, it is easier to look for the symptom, that is, the behavioral marker.

Once a visible marker is available, the problem is redefined: It is not about attitude anymore; it is about improving the marker.  Of course, this is not new and everyone knows that the symptomatic solution does not work. Tavistock consultants use to speak about «snake oil» as an example of useless fluid offered by someone who knows it does not work to any other who knows the same. However, even knowing it, they can buy the snake oil because it satisfies the own interest…for instance, not being accused of inaction about the problem.

The symptomatic solution goes on even in front of full evidence against it. At the end of the day, who sells it makes a profit and who buys it save the face. The next step should be alleging that the solution does not perform at the expected level and, hence, we should improve it.

Once there, some crossed interests make hard changing things for anyone who has something to lose. It is risky telling that «The Emperor is naked».Instead of that, there is a high probability that people will start to praise the new Emperor gown. 

Summarizing, training is useful to change if there is in advance a desire to change. Behavioral markers are useful if they can be observed under conditions where the observed person does not know to be observed. Does CRM meet these conditions? There is an alternative: Showing in a clear and undisputed way that the suggested behavior gets better results than the exhibited by the person to be trained. Again…does CRM meet this condition?

Certainly, we could find behavioral markers that, for deep psychology lovers, are predictive. However, this is a very dangerous road that some people followed in selection processes. This could easily become a kind of witch-hunting. As an anecdote, a recruiter was very proud about his magical question to know a candidate: His magic question was asking for the name of the second wife of Fernando the Catholic. For him, this question could provide him a lot of keys about the normal behavior of the candidate. Surprisingly, these keys dissappeared if the candidate happened to know the right answer.

If behavioral markers have a questionnable value and looking for other behaviors with a remote relation with the required ones, it should be required looking in different places if we want a real CRM instead of pressure to agreement -misunderstood teamwork- or theatrical exercises aimed to provide the desired behavior to the observer.

There is a lot of work to do but, perhaps, in different ways that the ones already stepped:

  1. Recruiting investment: Recruiting cannot be driven only by technical ability since it can be acquired by someone with basic competences. Southwest Airlines is said to have rejected as a pilot a candidate because he addressed in a rude way to a receptionist. Is it a mistake?
  2. Clear messages from Management: Teamwork does not appear with messages like «We’ll have to get along» but having shared goals and respect among tem members avoiding watertight compartments. Are we prizing the «cow-boy», the «hero» or the professional with the guts to make a hard decision using all the capabilities of the team under his command?
  3. CRM Evaluation from practicioners: Anyone can have a bad day but, if on continuous bases, someone is poorly evaluated by those in the same team, something is wrong, whatever could say the observer in the training process. If someone can think that this is against CRM, think twice:Forget for a moment CRM: Do pilots behave in the same way in a simulator exercise under observation and in a real plane?
  4. Building a teamwork environment: If someone feels that his behavior is problematic, there is a giant step to change. If, by the other side, he sees himself as «the boss» and he is delighted to have met himself, there is not way for a real change.

No shortcuts.CRM is a key for air safety improvement but it requires much more than behavioral markers and exercises where observers and observed people seem to be more concerned about looking polite than about solving problems using the full potential of a team.

When Profits and Safety are in different places: An historic approach to Aviation

All of us heard that Aviation is the safest Transportation way. That is basically true but, if 94% of accidents happen while on ground or near to ground, we should think that some flight phases have a risk level to be studied.

 It’s true that some activities bring an intrinsic risk and Safety means balancing acceptable risk level .vs. efficiency. Aviation is in that common situation but it has its own problems: The lack of an external assessment made the safety-related decisions to be inside a little group of manufacturers, regulators and operators. Consumers listen the mantra “Aviation is the safest Transportation way” but they cannot know if some decisions could drive Aviation to leave that privileged position.

 A little summary of technology evolution in the big manufacturers could show how and why some decisions were made and how, in the best possible scenario, these decisions meant losing an opportunity to improve safety level. In the worst one, they should mean a net decrease in safety level:

Once jets appeared, safety increased as a consequence of higher engines reliability. At the same time, navigation improvements like ground-based stations (VOR-DME), inertial systems and, later, GPS appeared too.

However, at the same time that these  and other improvements appeared, like making zero visibility landings possible, some other changes whose contribution could be considered as negative appeared too.

One of the best known cases is the engines number, especially in long haul flights. Decades ago, the standard practice for transoceanic flights was using four engine planes. The only exceptions were DC-10 and Lockheed Tristar with three engines. However, in places like U.S.A., long flights where, if required, planes could land before their planned destination, were performed by big planes with only two engines.

Boeing, one of the main manufacturers, would use this fact to say that engines reliability could allow transoceanic flights with twin planes. Of course, maintaining two engines is cheaper than maintaining four and, then, operators should have a strong incentive to embrace the Boeing position but…can we say that crossing an Ocean with two engines is as safe as doing it with four engines, keeping the remaining parameters constant?

Intuition says that it’s not true but messages trying to oppose this simple fact started to appear. Among them, we can hear that a twin modern plane is safer than a four-engine old plane. Nobody said that, if so, the parameter setting safety level should be how old the plane was. Then, the right option should be…a modern plane with four engines.

 Airbus, the other big manufacturer, complained because at that moment did not have its own twin planes to perform transoceanic flights but, some time after, they would accept this option starting their own twin planes for these long haul flights. This path –complain followed by acceptance and imitation- has been repeated regarding different issues: One of the manufacturers proposes an efficiency improvement, “its” regulator accepts the change asking for some improvements and the other manufacturer keeps complaining until the moment they have a plane that can compete in that scenario.

In the specific case about twin engines, regulators imposed a rule asking the operators to keep a certain distance from airports that could be in their way. That made twin planes design longer routes and, of course, that meant time and fuel expenses. However, since statistical information showed that engines reliability is very high, the time span allowed to fly with only one engine working while loaded with passengers was increasing until the present situation. Now, we have planes that are certified to fly with only one engine working until arriving to the nearest airport…assuming that it could be 5 hours and a half far. Is that safe?

We don’t really know how safe it is. Of course, it is efficient because that means that a twin engine certified in that way can fly virtually through any imaginable route. Statistics say that it’s safe but the big bulk of data about reliability does not come from laboratories but from flying planes and that’s where statistics could fail: Engines reliability makes that big amount of data come from flights where both engines have been working in uneventful flights. We can add that twin planes have more remaining power than four-engine planes due to the exigence that, if an engine fails after a moment during take-off, the plane has to be able to take-off with only one engine. Of course, the four-engine plane has to be able to perform this action with three engines, not with one.

In other words, during cruise time, the engines of a twin plane work in a low effort situation that, of course, can have a favorable impact in reliability. The question that statistical reports could not address because of lack of the right sample should be: Once one engine failed, the remaining one starts to work in a much more exigent situation. Does it keep the same reliability level that it had while both engines were working? Is that reliability enough to guarantee the flight under these conditions for more than 5 hours? Actually, the lack of a definitive answer to this question made the regulators to ask for a condition instead: The remaining engine should not get out of normal parameters while providing all the required power to keep the plane airborne.

At least, we could have some doubts about it but, since the decision was made among “insiders” without any kind of external check, nobody questioned it and, nowadays, the most common practice at boarding a transoceanic flight, is doing it in a twin plane. We will attend to the masks and lifejackets show but it’s unlikely that some could say:

“By the way, the engines in this plane are so reliable that, in the very unlikely event that one of them fails, we can fly with full safety with the remaining one until reaching the nearest airport, no more than 5 hours and a half far”.

How many users are informed about this little detail with they board a plane with the intention of crossing an Ocean? This is only and example because it’s not the only field where improvement followed by complains and acceptance was the common behavior.

Engines number is an issue especially visible –for obvious reasons- but a similar case can be observed in matters like codkpit crewmembers decrease or automation. Right now, there is not a single passengers plane from any of the big manufacturers bringing flight engineer. In this case, Airbus was the innovator in its A310 model and, like in the engines issue, we could ask if removing the flight engineer has made Aviation more or less safe.

Boeing was the one complaining in this case but…it happened to be designing its models 757 and 767 that, in the final configuration, would be launched without a flight engineer.

Is a flight engineer important for safety? Our starting point should be a very easy one: The job of a pilot does not know the concept of “average workload”. It goes from urgencies and stress to boredom and viceversa. In a noneventful flight overflying an Ocean and without traffic problems, there are not many things to do. The plane can fly without a flight engineer and even without pilots. They remain in their place “just-in-case”, that is, in a situation quite similar –with some differences- to the one we can find in a firemen place. However, when things become complex, there is a natural división of tasks: One of the pilots flies the plane while the other one takes care of navigation and communications and, if there is a serious technical problem, they have to try to fix it…it seems that someone is missing.

This absence was very clear in 1998 in Swissair-111, where a cabin smoke situation should make a MD-11, without a flight engineer, crash. In a few moments, they passed from an uneventful flight prepared to cross Atlantic Ocean to a burning hell where they had to land in an unknown airport, to find the place and runways orientation, radio frequencies…while keeping the plane controlled, throwing fuel and trying to know the origin of the fire to extinguish it.

The accident research, performed by “insiders” did not address this issue. Two people cockpit was already considered as a given, even though another almost identical plane –DC10- with flight engineer could have invited them to make the comparison. Of course, nobody can say that having a flight engineer should have saved the plane but the workload that pilots confronted should have been far lower.

This issue was not addressed neither when a plane from Air Transatt landed at Azores islands with both engines stopped. That happened because they were losing fuel and a wrong fuel management made the pilots transfer fuel to the tank that was losing it. Should it have happened if someone had been devoted to analyze carefully fuel flow and how the whole process was working? Perhaps not but this scenario was simply ignored.

Flight engineers dissappeared because automation appeared and that started a new problem: Pilots started to lose skills for manual flying and it drove to a situation named “automation paradox”:

Automation gets an easier user interface but this is a mirage: A cockpit with less controls and cleaner from a visual scope does not mean that the plane is simpler. Actually, it’s a much more complex plane. For instance, every Boeing 747 generation has been decreasing the number of controls in the cockpit. Even though, newer planes are more complex and that’s how the automation paradox works:

Training is centered in interface design instead of internal design. That’s why we find planes more and more complex and users who know less and less about them. A single comparison can be made with Windows systems, almost universal in personal IT. Of course, it allows much more things than the old DOS but…DOS never got blocked. Unlike DOS, Windows is much more powerful but, if blocked, the user does not have available options.

The question should be if we can admit a Windows-like system in an environment where risk is an intrinsic part of the activity. The system allows more things and can be properly managed without being an expert but, if it fails, there are not options for the average user.

“Fly-by-wire” system was introduced by Airbus in commercial Aviation, with the Concorde exception, and it confronted complains from Boeing. We have to say that Boeing had a high experience in fly-by-wire systems because of its military aircrafts. Again, we find a situation where efficiency is bigger even though some pilots complain about facts like losing kinestesic feeling. In a traditional plane, a hand on the controls can be enough to know how the plane is flying and if there is a problem with speed, center of gravity and others. In fly-by-wire planes, by default, this feeling does not exist (Boeing kept it in its planes but, to do so, they had to “craft” the feeling since the controls by themselves don’t not provide it).

 This absence could partially explain some major accidents, labeled “Human Error” or “Lack of Training” without anybody analyzing what features of the design could drive to an error like, for instance, a defective sensor triggering an automatic response without the pilots knowing what’s going on.

 What is the situation right now? If we check the last planes from the big manufacturers, we can get some clues: Boeing 787 .vs. Airbus A350. Both are big twin and long-haul planes, there is not a flight engineer, they are highly automated and they both have fly-by-wire system. Coincidence? Not at all. Through a dynamic of unquestionned changes agreed by insiders and without knowledge by the consumers, the winner will be always the most efficient solution. Then, both manufacturers finished with two models that share a good part of the philosophy. There are differences –electric .vs. hydraulic controls, feeling .vs. no-feeling from the controls, more or less use of composite materials, lithium .vs. traditional batteries…- but the main parameters are the same.

 Issues that were discussed time ago are seen as already decided. The decision always favored the most efficient option, not the safest one. Could that be changed? Of course, but it’s not possible if everything keeps working as an “insiders game” instead of giving clear and transparent information outside.

We should understand too the position of «insiders»: A case like GermanWings was enough for some people -like NYT- to question the plane before knowing what really happened. A few days ago, we had an accident with a big military plane manufactured by Airbus and some people started already to question the safety of a single manufacturer…perhaps someone near to the other one?

Information has to flow freely but, at the same time, many people make a living from scandal and it’s hard to find the right point: Truth and nothing but the truth and, at the same time, deactivate the ones who want to find or manufacture a scandal. Nowadays, the environment is very closed and in that environment efficiency will have always the upper hand…even in cases where it shouldn’t. By the other side, we have to be careful enough to address real problems instead of invented ones. The examples used here can be illustrated not only with the referenced cases but with some others whose mention has been avoided.

Del parado como del cerdo todo se aprovecha…por parte de algunos

Sé que la afirmación es cruel pero después de ver, entre otras cosas, cómo los partidos políticos y sindicatos roban fondos destinados a la formación de los desempleados o meten en EREs realizados por empresas en crisis a esbirros suyos que jamás habían trabajado ahí…¿qué otra cosa puede decirse?

Pues bien, esta misma mañana me llega una forma más entre las muchas e ingeniosas que hay de robar a los parados:

Una empresa con un nombre muy sonoro, naturalmente en inglés, busca colaboradores expertos a los que, a cambio de una tarjeta y el honor de utilizar su nombre al facturar cobra una cuota de entrada que varía entre 18 y 30.000 euros y, después, una cifra del orden del 10 al 30% sobre facturación.

Naturalmente, los clientes se los tiene que buscar su víctima y se supone que se le abrirán las puertas del Universo una vez que aparezca ante los potenciales clientes con tan prestigioso nombre en su tarjeta. Estas «empresas» además ofrecen a menudo los servicios de un call-center destinado a conseguir entrevistas de forma que no haya que ir a hacer ventas a puerta fría. También cobran por entrevista conseguida, tanto si es con el Director General como con el encargado de la limpieza…y lo mejor de todo: Es absolutamente legal con lo que este tipo de estafadores sin escrúpulos están saliendo como las setas en otoño.

¿No es bastante dura la situación de desempleo para, además, tener que estar atentos a no ser estafados por este tipo de carroñeros?

Human Resources and Mathematical Fictions

It is hard to find more discussed and less solved issues than how to quantify Human Resources. We have looked for tools to evaluate jobs, to evaluate performance and at what percentage objectives were met. Some people tried to quantify in percentage terms how and individual and a job fit and, even, many people tried to obtain the ROI over training. Someone recovered Q index, aimed to quantify speculative investments, to convert it into the main variable for Intellectual Capital measurement, etc..

 Trying to get everything quantified is so absurd as denying a priori any possibility of quantification. However, some points deserve to be clarified:

New economy is the new motto but measurement and control instruments and, above all, business mentality is defined by engineers and economists and, hence, organizations are conceived as machines that have to be designed, adjusted, repaired and measured. However, it is a common fact that rigor demanded about meeting objectives is not used in the definition of the indicators. That brought something that is called here Mathematical Fictions.

A basic design principle should be that any indicator can be more precise than the thing it tries to indicate whatever the number of decimal digits we could use. When someone insists in keeping a wrong indicator, consequences appear and they are never good:

  •  Management behavior is driver by an indicator that can be misguided due to sneaky type of the variable supposedly indicated. It is worth remembering what happened when some Governments decided that the main priority in Social Security was reducing the number of days in waiting lists instead of the fluffy “improving Public Health System”. A common misbehavior should be to give priority to less time consuming interventions to reduce the number of citizens delaying the most importan tones.
  • There is a development of measurement systems whose costs are not paid by the supposed improvement to get from them. In other words, control becomes an objective instead of a vehicle since control advantages do not cover costs of building and maintenance of the control. For instance, some companies trying to control abuse in photocopies ask for a form for every single photocopy making the control much more expensive than the controlled resource.
  • Mathematical fictions appear when some weight variables that, in the best situation, are only useful for a situation and lose its value if the situation changes. Attemps relative to Intellectual Capital are a good example but we commit the same error if we try to obtain percents of people-job adjustment to use them as to foresee success in a recruiting process.
  • Above all, numbers are a language that is valid for some terrains but not for others. Written information is commonly rejected with “smart talk trap” arguments but the real fact is that we can perceive fake arguments easier in written or verbal statements than if they come wrapped in numbers. People use to be far less exigent about indicators design than about written reports.
  • Even though we always try to use numbers as “objective” indicators, the ability to handle these numbers by many people is surprisingly low. We do not need to speak about the journalist that wrote that Galapagos Islands are hundreds of thousands of kilometers far from Ecuador coast or the common mistake between American billion or European billion. We can show two easy examples about how numbers can lose any objectivity due to bad use:

After the accident of Concorde in Paris, 2001, media reported that it was the safest plane in the world. If we consider that, at that time, only fourteen planes of the type were flying instead of the thousands of not-so-exclusive planes, it is not surprising that an accident never happened before and, hence, nobody can say from it to be the safest plane. The sample was very short to say that.

Another example: In a public statement, the Iberia airline said that travelling by plane is 22 times safer than doing it by car. Does it mean that a minute spent in a plane is 22 times safer than a minute spent in a car? Far from it. This statement can be true or false depending of another variable: Exposure time. A Madrid-Barcelona flight lasts seven times less than a trip by car. However, if we try to contrast one hour inside a plane with an hour inside a car, results could be very far from these 22 times.

The only objective of these examples is showing how numbers can mislead too and we are less prepared to detect the trick than when we have to deal with written language.

These are old problems but –we have to insist- that does not mean they are solved and, perhaps, we should to arrive to the Savater idea in the sense that we do not deal with problems but with questions. Hence, we cannot expect a “solution” but contingent answer that never will close forever the question.

If we work with this in mind, measurement should acquire a new meaning. If we have contingent measurements and we are willing to build them seriously and to change them when they become useless, we could solve some –not all of them- problems linked to measurement. However, problems will arise when measurement is used to inform third parties and that could limit the possibility to change.

An example from Human Resources field can clarify this idea:

Some years ago, job evaluation systems had a real crisis. Competencies models came from this crisis but they have problems to for measurement. However, knowing why job evaluation systems started to be displaced is very revealing:

Even though there are not big differences among the most popular job evaluation systems, we will use Know-How, Problem Solving and Accountability, using a single table to compare different jobs in these three factors is brilliant. However, it has some problems hard to avoid:

  • Reducing to a single currency, the point, all the ratings coming from the three factors implies the existence of a “mathematical artifact” to weight the ratings and, hence, priming some factors over others.
  • If, after that, there are gross deviations from market levels, exceptions were required and these go directly against one of the main values that justified the system: Fairness.

Although these problems, job evaluation systems left an interesting legacy not very used: Before converting ratings into points, that is, before starting mathematical fictions, we have to rate every single factor. We have there a high quality information, for instance, to plan professional paths. A 13 points difference does not explain anything but a difference between D and E, if they are clearly defined, are a good index for a Human Resources manager.

If that is so…why is unused this potential of the system? There is an easy answer: Because job evaluation systems have been used as a salary negotiation tool and that brings another problem: Quantifiers have a bad design and, furthermore, they have been used for goals different from the original one.

The use of mix comittees for salary bargaining, among other factors, has nullified the analytical potential of job evaluation systems. Once a job is rated in a way, it is hard to know if this rating is real or it comes from the vagaries of the bargaining process.

While job evaluation remained as an internal tool of Human Resources area, it worked fine. If a system started to work poorly, it could be ignored or changed. However, if this system starts to be a main piece in the bargaining, it losses these features and, hence, its use as a Human Resources tool dissapears.

Something similar happens if we speak about Balanced Scorecard or Intellectual Capital. If we analyze both models, we’ll find that there is only a different variable and a different emphasis: We could say, without bending too much the concepts, that the Kaplan and Norton model is equal to Intellectual Capital plus financial side but there is another difference more relevant:

Balanced Scorecard is conceived as a tool for internal control. That implies that changes are easy while Intellectual Capital was created to give information to third parties. Hence, measurement has to be more permanent, less flexible and…less useful.

Actually, there are many examples to be used where the double use of a tool nullifies at least another one. The same idea of “Double Accounting” implies criticism. However, pretending that a system designed to give information to third parties can be, at the same time and with the same criteria, an effective tool for control, is quite near to ScFi.

Competencies systems have too its own part of mathematical fiction. It is hard to créate a system able to capture all the competencies and to avoid overlapping among them. If this is already hard…how is it possible to weight variables to define job-occupant adjustment? How many times are we evaluating the same thing under different names? When can we weight a competence? Is this value absolute or should it depend on contingencies? Summarizing….is it not a mathematical nonsense aimed to get a look of objectivity and, just-in-case, to justify a mistake?

This is not a declaration against measurement and, even less, against mathematics but against the symplistic use of it. “Do it as simple as possible but no more” is a good idea that is often forgotten.

Many of the figures that we use, not only in Human Resources, are real fiction ornated with a supposed objectivity coming from the use of a numeric language whose drawbacks are quite serious. Numeric language can be useful to write a symphony but nobody would use it to compose poetry (except if someone decides to use the cheap trick of converting letters into numbers) and, however, there is a general opinion about numbers as universal language or, as Intellectual Capital starters said, “numbers are the commonly accepted currency in the business language”.

We need to show not only momentaneous situations but dynamics and how to explain them. That requires written explanations that, certainly, can misguide but, at least, we are better equipped to detect it than if it come wrapped in numbers.

Three myths in technology design and HCI: Back to basics

It has been a coincidence driven by the anniversary of Spanair accident but, for a few days, comments about the train accident in Santiago de Compostela and about the Spanair accident appeared together. Both have a  common feature beyond, of course, a high and deadly cost. This feature could be stated like this: «A lapsus cannot drive to a major accident. If it happens, something is wrong in the system as a whole».

The operator -pilot, train driver or whoever- can be responsible if there is negligence or clear violation but a lapsus should be avoided by the environment and, if it is not possible, its consequences should be decreased or nullified by the system. Clearly, it did not happen in any of these cases but…what was the generic problem? There are some myths related to technology development that should be explicity addressed and they are not:

  • First myth: There is not an intrinsic difference between open and closed systems. If a system is labeled as open, that comes only from ignorance and technology development can convert it into a closed one: To be short and clear, a closed system is one where everything can be foreseen and, hence, it is possible to work with explicit instructions or procedures while an open one has different sources of interaction from outside or inside and it makes impossible to foresee all posible disturbances. If we accept the myth as a truth, no knowledge beyond operative level is required from the operator once technology reached the right point to consider a system as closed. Normative approach should be enough since every disturbance can be foreseen.

Kim Vicente, in his Cognitive Work Analysis used a good metaphor to attack this idea: Is it better having specific instructions to arrive to a place or is it better a map? Specific instructions can be optimized but they fail under closed streets, traffic jams and many other situations. A map is not so optimized but it provides resources under unforeseen situations. What if the map is so complex that including it in the training program should be very expensive? What if the operator was used to a roadmap and now he has to learn how to read an aeronautical or topographic map? If the myth works, there is not problem. Closed street and traffic jams do not exist and, if they do, they always happen in specific places that can be foreseen.

  • Second myth: A system where the operator has a passive role can be designed in a way that enables situation awareness. Perhaps to address this myth properly, we should go back to a classic experiment in Psychology:  http://bit.ly/175gKIc where a cat is transporting another one in a cart. Supposedly, the visual learning of both cats should be the same since they have a common information. However, results say that it does not happen. The transporting cat get a much better visual learning than the transported one. We don’t really need the cats nor the experiment no know that. Many of us can go a lot of times to a place while other person is driving. What happens when we are asked to go alone to that place? Probably, we did not learn how to go. If this happens with cats and with many of us…is it reasonnable to believe that the operator will be able to solve an unplanned situation where he has been fully out of the loop? Some designs could be removing continuous feedback features because they are hard and expensive to keep and, supposedly, they add nothing to the system. Time ago, a pilot in a highly automated plane told me: «Before, I was able to drive the plane; now the plane drives me»…this is other way to describe the present situation.
  • Third myth: Availability bias: We are going to do our best with our resources. This can be a common approach by designers: What can we offer with the things that we have or we can develop at a reasonnable cost? Perhaps that is not the right question. Many things that we do in our daily life could be packed in an algorythm and, hence, automated. Are we stealing pieces of situation awareness at doing so? Are we converting the «map» into «instructions» without resources if these instructions cannot be applied? However, for the last decades, designers have been behaving like that: Providing an output under the shape of a light, a screen or a sound it quite easy while handles, hydraulic lines working -and transmitting- pressure and many other mechanical devices are harder and expensive to include.

Perhaps whe should remember again «our cat» and how visual and auditive cues could not be enough. The right question is never about what technology is able to provide but about what is the situation awareness that the operator has at any moment and what are his capabilities and resources to solve an unplanned problem. Once we answer this question, perhaps some surprises could appear. For instance, we could learn that not everything that can be done, has to be done and, by the same token, some things that should be done have not a cheap and reliable technology available. Starting a design trying to provide everything that technology can provide is a mistake and, sometimes, this mistake is subtle enough to pass undetected for years.

Many recent accidents are pointing to these design flaws, not only Spanair and Renfe ones:  Automated pilots that get data from faulty sensors (Turkish Airlines, AF447 or Birgenair) , stick-shakers that can be programmed -instead of behaving as the natural reaction of a plane near to stall- provoking an over-reaction from fatigued pilots (Colgan), indicators where a single value can mean opposite things (Three-Mile Island) and many others.

It’s clear that we live in a technological civilization. That means assuming some risks, even catasthropic ones, like an EPM or a massive solar storm. However, there are other minor and current risks that should be controlled. Having people to solve the problems while, at the same time, we steal them the resources they should need to do that is unrealistic. If, driven by cost-consciousness, we assume that unforeseen situations are below one in thousand million and, hence, this is an acceptable risk, be coherent: Eliminate the human operator. By the other side, if we think that unforeseen situations can appear and have to be managed, we have to provide people with the right means to do so.  Both are valid and legitimate ways to behave. Removing resources -including the ones that allow situation awareness- and, once the unforeseen situation appears, having an operator as a breaker to burn speaking of «lack of training», «inadequate procedure compliance» and other common labels is not a right nor legitimate way. Of course, accidents will happen even if everything is properly done but, at least, the accidents waiting to happen should be removed.

Discusiones bizantinas sobre competencias, inteligencias emocionales, titulaciones y otros

Cuando llegó la moda de la inteligencia emocional de la mano de Daniel Goleman, estuve entre las voces discordantes que afirmaban que el concepto y, sobre todo, el uso que se hacía del mismo no tenían ningún sentido. Que existen unas características personales que son determinantes para el éxito o fracaso es claro; si a eso se le quiere llamar inteligencia emocional sea. Es un nombre más marquetinero que preciso pero puede aceptarse.

Lo que ya no es aceptable es desbarrar…y se desbarra cuando se hacen afirmaciones como que el 80% del éxito es atribuible a la inteligencia emocional por encima de la inteligencia en el sentido que la hemos entendido siempre.  Se desbarra también cuando se dice que las competencias son mucho más importantes que las titulaciones en el éxito profesional…sin pretender ni en un caso ni en el otro negar la importancia de los factores a los que se les quiere dar tanta primacia. El problema es mucho más simple: Es una cuestión de secuencia y no de porcentaje.

Expliquémoslo con un ejemplo sencillo: ¿Qué es más importante para el éxito de un cirujano? ¿Su título académico o su habilidad y capacidades en el quirófano?…Por supuesto, la pregunta tiene trampa y, además, ésta es muy visible: Para entrar al quirófano armado de un bisturí, el cirujano necesita tener un título académico y, por tanto, el segundo filtro -habilidad y capacidades- se aplica sobre los que han pasado el primero -título- y, por tanto, no podemos establecer una comparación en términos de porcentaje entre titulación y capacidades: Elijo a las personas por sus capacidades entre los que tienen la titulación. ¿Cómo puedo hablar entonces de porcentajes relativos de importancia entre lo uno y lo otro?

Por supuesto, es un caso extremo pero aplicable a los conceptos a los que me refería con la idea de discusiones bizantinas: Alguien puede desempeñarse espléndidamente gracias a su inteligencia emocional pero la entrada al terreno de juego se la ganará con la inteligencia, en los términos que todos conocemos de toda la vida. ¿Podemos decir que para muchos puestos, una vez que se ha pasado un umbral de cociente intelectual -por utilizar una métrica conocida de todos-  es más favorable mejorar en la capacidad de interacción con los demás que añadir 10 puntos más de cociente intelectual? Probablemente sí pero si las cosas funcionan así, es decir, si el acceso lo defino mediante un valor umbral y el desempeño como una selección por otros criterios entre los que han superado ese umbral ¿puedo decir cosas como que la inteligencia emocional es responsable del 80% del éxito? No deja de ser una majadería si no le añado «entre aquéllos que tengan un nivel intelectual como mínimo medio-alto» y, si le hago este añadido, deja de ser una majadería para convertirse en una simpleza.

Idéntico razonamiento podemos aplicar a la disquisición sobre la importancia relativa de competencias y títulos: ¿Cuántas ofertas aparecen -o aparecían cuando las había- en que se pide «titulación superior»? Obsérvese que les da lo mismo que sea una ingeniería de caminos que una licenciatura en filatelia avanzada, en el caso de que tal cosa exista. Hay profesiones que exigen un título específico para el acceso -médico, farmacéutico, maquinista ferroviario…- y en éstas la discusión ya es absurda: Se selecciona por competencias entre los que tienen el título y, por tanto, no tiene sentido comparar la importancia relativa de uno y otro factor. Incluso entre los casos que no exigen explícitamente un título o habilitación profesional, suele darse un cierto sesgo haciendo que las condiciones de acceso no sean las mismas.

En conclusión, no podemos comparar la importancia relativa de dos factores cuando uno de ellos hace referencia al acceso al puesto mientras que el otro hace referencia al desempeño una vez que se está en el puesto. Es como comparar el tocino con la velocidad…y además, usando porcentajes para que quede más «científico».

El origen de la crisis al descubierto

Compromiso

 

Cuando ayer mismo recibí esta cita en un folleto publicitario, me resultó difícil de creer. Algunos no entienden que el compromiso no se rige por las leyes de mercado sino que en él funciona la reciprocidad más absoluta. No me resulta difícil de creer el dato aunque tenga serias dudas sobre el tipo de diseño que pueda realizarse para llegar a conseguirlo. Sin embargo, hay una pregunta mucho más serie que unos tantos por ciento arriba o abajo:

¿Qué porcentaje de las empresas muestra compromiso a largo plazo con sus empleados?

Insisto: Aquí no funciona el mercado y el compromiso es recíproco. O lo hay o no lo hay y son muchas, muchísimas las empresas que se llenan la boca con el capital humano como el mejor de sus activos a la vez que lo tratan como lastre del que desprenderse cuando es necesario. Sería interesante hacer el cálculo recíproco ¿o ni siquiera es necesario?

Technology development model, safety and security: Time to change?

Many people had been warning about a wrong technological development years ago; you can find an example in the frame of this blog under the name «Improving Air Safety through Organizational Learning». However, the development model did not change at all and nobody paid attention. Statistical information, in a high level analysis, could justify this behavior but, in the last years, many things have happened that can be seen as a serious warning hard to ignore. The so-called  «black swans», meaning facts impossible to forecast, start to become a full flock. Some cases:

  1. Economic crisis: We can speak about greed, opportunism and many other things. The fact is that the financial system was so complex that it was impossible to have a real surveillance and many people understand parts of it but ignore secondary effects in different parts of the same system. Perhaps, the best explanation about what happened is a humoresque one: http://youtu.be/mzJmTCYmo9g Opening any newspaper can show how hard is an agreement about the diagnostics and, hence, about the solutions. Depending on the expert one asks, forecasts are going to be different. Even, with emergent issues like BitCoin, there is not an agreement among experts about what the consequences are going to be.
  2. AF447: This air accident should mark a change: Experts with interest in the market went running to show that, at the end, everything was coming from a sensor and a human error. Is that right? If a faulty sensor and a tired pilot are enough to crash a plane…something is seriously wrong about air safety. Journalists ready to help manufacturers and regulators emphasized the time required by the pilot to go back to the cockpit. Have you ever seen the layout of a long-haul plane? Perhaps, you have observed a door in the middle of the plane where crewmembers come and go through. This door allows crewmembers to access the bunks that are downstairs. Now, watch in hand, try to find how long it takes to arrive from the bunk to the cockpit. Why bunks were removed from a position near to cockpit? The reality is that crewmembers got confused and a system that does not give enough information about what happens is, at least, questionnable because under unplanned events, it can produce confusion and inability to find the right action to perform.
  3. Cyberattacks: About two years ago, Iranian said to be able to get control over a U.S. drone forcing to land. The feasibility of that was rejected by U.S. officers but, only days ago, something happened inviting to think that it was a real possibility: http://www.businessweek.com/articles/2013-04-12/hacking-an-airplane-with-only-an-android-phone If a mobile phone is enough to get control over a manned plane…can we seriously affirm that it is not possible with more advanced technology over an unmanned one? Aviation is not the only activity where things like these can happen: http://arstechnica.com/information-technology/2013/04/the-spammer-who-logged-into-my-pc-and-installed-microsoft-office/ It’s said months ago that something bigger than 11S could come from powerful cyberattacks that could make useless vital installations. Even some technology managers are worried about the possibility of an undetected cyberattack that could convert a sophisticated weapon into something useless at the critical moment. Beyond cyberattacks, the feasibility of an EMP (http://en.wikipedia.org/wiki/Electromagnetic_pulse ) with similar but longer term effects is another real and present danger.
  4. Food fraud: Is it so hard identifying the kind of meat -if so-  that hamburgers have inside? Perhaps it is hard for users but…also for regulators? Why every other day a new scandal appears with products containing something that it is not in the declared composition? Not only horse or moose meat but dogs…what else?
  5. Aviation safety: Beyond AF447…is the average passenger informed and concerned about what is the real safety level? Does the passenger know that, when crossing an Ocean in a twin plane, if one of the engines fails, the plane is certified to fly for hours with the remaining engine? Manufacturing processes are the ones that are certified or, as some people say, different? http://www.thedailybeast.com/newsweek/2012/03/19/is-boeing-s-737-an-airplane-prone-to-problems.html . What about the air quality onboard? http://www.achooallergy.com/air-quality-airplane-cabins.asp What about the practice in some airlines of having a flight student as a first officer paying to fly in a plane with passengers?…

In short, technology has kept the same track and it is more and more hard to understand and check. Regulators cannot be trusted -independently of their knowledge or professional attitude- if final user cannot check their work. If users cannot check how the regulators protect their interests, the expected results should be that the regulators should take care of their own interest, that not always could be the same that the ones of users.

The old Rassmussen rule, «the operator has to be able to run cognitively the program that the system is running», is not followed time ago. However, as times goes by, we find that the problem goes beyond operators. Designers themselves understand specific parts without a clear understanding of the full product and, hence, unable to foresee consequences coming from interactions among different parts of the system. AF447 was a big warning light but it is not the only one and, perhaps, it is not even the worst one. It should be the right time for an assessment of technology development model. Otherwise, consequences will be worse and worse.

«Seconds to Disaster» by Glenn Meade and Ray Ronan

When I read my doctoral dissertation -you can find two versions in the blog frame depending of your kind of interest: Aviation or Organizational Studies- I remember especially one of the persons who had to evaluate it, Secundino Valladares. He said: «Now, I’m sure that I will never fly again». I cannot blame him: To justify every finding, I put two or three paragraphs, extracted from official reports about major Aviation accidents. It was quite easy to reach that conclusion. After reading the book by Meade and Ronan, I have started to think of myself as a soft nun, regarding the kind of things they bring to the discussion.

For instance, the existence of compromises between regulators and main manufacturers is crystal-clear and there are many facts that can show how the European regulator does not look with the same eyes at both, Boeing and Airbus, and the same can be said about FAA but, of course, in the opposite side. Even though, the chapter that the authors devote to Boeing 737NG shows something far beyond a «friendly eye». Nothing new; some of us are old enough to remember what happened with DC-10 and how 325 avoidable deaths (Turkish Airlines 981) were required to fix a problem that was previously known. The authors speak too about AF447 -you will find in this blog several posts dealing with AF447- and, for a moment, I had the feeling of not being alone with my conclussions about this case: The stamp «Lack of Training» is very comfortable to close a report avoiding entering in design issues. However, this stamps does not answer the main question: If that is true…why did you have people lacking training to fly a big plane over Atlantic Ocean? We can go beyond: Is that a training practice by Air France or is it shared worldwide? Still, we can go beyond: Is it possible, due to design complexity, provide pilots and engineers with the training level that they could require under an extreme situation? Once we get here, we could be around the root of the problem: Profit aimed design.

A few days ago, I published something about how average passengers boarding a twin plane for a long-haul trip does not know what are the rules: He does not know that, in the event of an engine-stop, the plane is certified to fly more than 5 hours from the nearest airport with only one engine working and full of passengers. This and many of the things that the authors of «Seconds to Disaster» say are unknown to the flying public. Perhaps, this is the first thing to change if we want to change something.