Friday, July 20, 2018

How Complex Systems Fail

Highlights from How Complex Systems Fail:

  • Title) How Failure is Attributed to Proximate Cause
  • 1) It is the presence of these hazards that drives the creation of defenses against hazard that characterize these systems
  • 2) multiple layers of defense
  • 3) Catastrophe requires multiple failures
  • 4) Eradication of all latent failures is limited primarily by economic cost but also because it is difficult before the fact to see how such failures might contribute to an accident
  • 5) complex systems run as broken systems
  • 5) The system continues to function because it contains so many redundancies and because people can make it function, despite the presence of many flaws
  • 6) It is impossible to eliminate the potential for such catastrophic failure
  • 7) Post-accident attribution accident to a ‘root cause’ is fundamentally wrong
  • 7) There are multiple contributors to accidents
  • 7) social, cultural need to blame specific, localized forces or events for outcomes
  • 8) Knowledge of the outcome makes it seem that events leading to the outcome should have appeared more salient to practitioners at the time than was actually the case
  • 8)  It seems that practitioners “should have known”
  • 9) dual roles
  • 9) Outsiders rarely acknowledge the duality of this role
  • 11) After an accident, practitioner actions may be regarded as ‘errors’ or ‘violations’
  • 11) biased by hindsight and ignore the other driving forces, especially production pressure
  • 13) Human expertise in complex systems is constantly changing
  • 13) need to replace experts who leave
  • 14) Change introduces new forms of failure
  • 14) overt: open and observable
  • 14) use of new technology
  • 14) decrease the number of low consequence but high frequency failures
  • 14) create opportunities for new, low frequency but high consequence failures
  • 14) Not uncommonly, these new, rare catastrophes have even greater impact than those eliminated by the new technology
  • 14) hard to see the contribution of technology to the failure
  • What to do? Freeze a system from all and any change?
  • 15) post-accident remedies usually increase the coupling and complexity
  • 16) Safety is a characteristic of systems and not of their components
  • 18) Failure free operations require experience with failure
  • "Game day" and failure injection

No comments: