IEC 61508 et al - when required?

This is a general question about standards that address potentially
hazardous failures, especially in software.  My question, as simply as
I can phrase it, is: If a company develops software that controls
potentially hazardous equipment, when should that software be
developed in compliance with formal standards?

A simple example - a software/hardware system controls a large motor.
A software bug could cause the motor to turn on at the wrong time or
otherwise behave incorrectly, which could cause injury or death.  What
standards, if any, come into play for the development of that
software?

I get confused reading the IEC 61508 FAQ: "[IEC 61508] covers possible
hazards caused by failure of the safety functions to be performed by
the E/E/PE safety-related systems, as distinct from hazards arising
from the E/E/PE equipment itself (for example electric shock etc)."

So would the motor turning on because of bug be a failure of the
safety functions, or a hazard arising from the equipment itself?

Any insights or pointers to reading material greatly appreciated.

KK6GM wrote:

> This is a general question about standards that address potentially
> hazardous failures, especially in software.  My question, as simply as
> I can phrase it, is: If a company develops software that controls
> potentially hazardous equipment, when should that software be
> developed in compliance with formal standards?

I shall make the assumption that this is an academic question. If it is 
being asked because you want to develop a safety critical control system 
then please seek appropriate professional help.

Professional developers of Safety Related and Safety Critical Systems will 
have researched, reviewed and appraised which standards they should be 
applying to the products they are developing. There are very few places in 
the world where you can import something that does not adhere to some 
standards. So, you should apply appropriate standards always.
 
> A simple example - a software/hardware system controls a large motor.
> A software bug could cause the motor to turn on at the wrong time or
> otherwise behave incorrectly, which could cause injury or death.  What
> standards, if any, come into play for the development of that
> software?

In Europe this would fall under the Machinery Safety Directive. A number of 
standards apply. From a response I made in another thread:-

[Depending on the industry sector you are dealing with you may find that
 IEC61508 is not necessarily the right one to comply with. IEC61508 is a
 generic standard from which the industry specific standards were to be
 developed.

 Functional Safety

 IEC61508 "Functional Safety of Electrical, Electronic and Programmable
 Electronic Systems".

 IEC61511 "Functional safety Safety instrumented systems for the process
 industry sector"

 IEC61513 "Nuclear power plants – Instrumentation and control for systems
 important to safety – General requirements for systems"

 IEC62061 "Safety of Machinery - Functional safety of safety-related
 electrical, electronic and programmable electronic control systems.

 ISO13849 "Safety of Machinery - Safety-related parts of control systems"

 Of course, there are also the non-industry specific ones:

 Machine Safety
 ISO 14118 "Safety of machinery -- Prevention of unexpected start-up"

 IEC 60204 "Safety of machinery - Electrical equipment of machines"

 ..and your document management needs to be up to scratch too.
 Document Control and Configuration Management
 IEC 82045-1:2001 “Document Management”]
 
> I get confused reading the IEC 61508 FAQ: "[IEC 61508] covers possible
> hazards caused by failure of the safety functions to be performed by
> the E/E/PE safety-related systems, as distinct from hazards arising
> from the E/E/PE equipment itself (for example electric shock etc)."

Electrical Installation standards such as BS-7671:2008 "Requirements for 
Electrical Installations" deal with some of that aspect.
 
> So would the motor turning on because of bug be a failure of the
> safety functions, or a hazard arising from the equipment itself?

The answer to that will be down to the architecture of your system and how 
you structure it to provide the protective measures. "Inherent safety of 
design" is better than "containment measures" for example. 
 
> Any insights or pointers to reading material greatly appreciated.

With over 200 titles in my personal library that is a bit of a dangerous 
question to ask. However try these three for starters.

"Lifecycle Management for Dependability" by Felix Redmill and Chris Dale 
ISBN 3-540-76073-3.

"Human factors in Safety Critical Systems" by Felix Redmill and jane Rajan, 
ISBN 0-7506-2715-8.

"Better Embedded Systems Software" by Philip Koopman, ISBN 978-0-9844490-0-2

Reading some of the standards above in conjunction with the above three 
books would get you a bit of a way in to doing the right things and getting 
your system architectural thinking moving along the right lines.

Hope that is a helpful start for you.

-- 
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

Op 04-Apr-11 22:20, KK6GM schreef:
> This is a general question about standards that address potentially
> hazardous failures, especially in software.  My question, as simply as
> I can phrase it, is: If a company develops software that controls
> potentially hazardous equipment, when should that software be
> developed in compliance with formal standards?

Not necessarily, it depends on the hardware.

> A simple example - a software/hardware system controls a large motor.
> A software bug could cause the motor to turn on at the wrong time or
> otherwise behave incorrectly, which could cause injury or death.  What
> standards, if any, come into play for the development of that
> software?
>
> I get confused reading the IEC 61508 FAQ: "[IEC 61508] covers possible
> hazards caused by failure of the safety functions to be performed by
> the E/E/PE safety-related systems, as distinct from hazards arising
> from the E/E/PE equipment itself (for example electric shock etc)."
>
> So would the motor turning on because of bug be a failure of the
> safety functions, or a hazard arising from the equipment itself?
>
> Any insights or pointers to reading material greatly appreciated.

In my line of business (production equipment) the typical solution is to 
make the equipment safe at hardware level in such a way that there is no 
way for the software to make it unsafe. Machine parts that may inflict 
bodily harm are protected by hardware interlocks which prevents the 
machine from becoming active if those parts are accessible (e.g. when an 
equipment door is open). The safety circuits do have to meet stringent 
safety standards, and do require special attention in the hardware 
design (e.g. ordinary relays may not be used in a safety circuit because 
their contacts may stick). Once you can prove the hardware design is 
such that the software cannot create dangerous situations then the 
software does not have to meet any safety standards.

I don't know if making the hardware inherently safe is an option for 
you. If it is, it is probably the easiest solution. If the software has 
to meet safety standards it does not only affect how your code and your 
software development process, but also limits the tools (compilers) and 
operating systems you can use.

Thanks for all the comments so far.  I understand that hardware
interlocks can shield against many software faults, and may be enough
protection in some instances, but I'm more curious about those
instances where hardware interlocks alone do not suffice.  Take the
Segway for example.  In the general case where software is controlling
outputs that can provide or channel significant energy (mechanical,
electrical, heat, etc), where that energy output could cause damage
and/or injury if the software misbehaves, how does one determine if
the software should be developed to any particular body of standards?
I'd add that I'm mostly interested in the situation in the USA.

KK6GM wrote:

> Thanks for all the comments so far.  I understand that hardware
> interlocks can shield against many software faults, and may be enough
> protection in some instances, but I'm more curious about those
> instances where hardware interlocks alone do not suffice.  Take the
> Segway for example.  In the general case where software is controlling
> outputs that can provide or channel significant energy (mechanical,
> electrical, heat, etc), where that energy output could cause damage
> and/or injury if the software misbehaves, how does one determine if
> the software should be developed to any particular body of standards?
> I'd add that I'm mostly interested in the situation in the USA.

While hardware interlocking might be suitable in some situations this is not 
always practical. If software and/or electronic control systems failing 
could cause dangerous problems then some other mitigation needs to be put in 
place. In such situations you may have systems in parallel (2oo3, 3oo4, 4oo5 
etc) where a vote is taken to confirm that the action to be taken is 
probably the best course. One does, however, have to be wary of the 
introduction of systemic errors into all channels of control. Very careful 
consideration of the voting mechanism is also needed.

As for the standards to apply, if you are doing flight systems for aircraft 
(where interlocking is somewhat impractical) you would look to standards 
like DO-178B (or C). If it was for UK railways you might look at the 
standards from the Rail Industry Association (there are equivalents in other 
countries). For nuclear power plant then IEC61513 would most likely apply.

The considerations and techniques you will employ must all answer the 
question of how you are going to mitigate the risks posed by the hazards 
that are uncovered during the analysis phases (plural here as I expect that 
you will analyse the system hazards many times during the development 
process).

With the Segway item you mention I would expect that there is an opportunity 
for interlocking the energy output to the motors by the use of a deadman's 
handle arrangement so that the device will stop if this is released by the 
rider.

I am sure that if you wish to present a more concrete system architecture 
for the system type that you wish to study in more detail, there will be 
quite a few on this list that would be happy to openly discuss this.

-- 
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

Op 05-Apr-11 20:54, KK6GM schreef:
> Thanks for all the comments so far.  I understand that hardware
> interlocks can shield against many software faults, and may be enough
> protection in some instances, but I'm more curious about those
> instances where hardware interlocks alone do not suffice.  Take the
> Segway for example.  In the general case where software is controlling
> outputs that can provide or channel significant energy (mechanical,
> electrical, heat, etc), where that energy output could cause damage
> and/or injury if the software misbehaves, how does one determine if
> the software should be developed to any particular body of standards?
> I'd add that I'm mostly interested in the situation in the USA.

You might want to look into UL1998, though it is difficult to tell if it 
is applicable to your project without knowing the specifics of your 
project and it intended application area/classification. And even with 
the specifics of the project it is not always clear cut which safety 
standards must be adhered to, there is often room for interpretation and 
negotiation. In some cases legislation may dictate certain safety 
standards, but often its a choice of the manufacturer based on market 
demands and/or for legal reasons ('cover your ass') in case things do go 
wrong. In case of the USA the standards you will have to meet may vary 
between states.

Sometimes apparently small aspects can have a big impact on which 
standards the software has to meet. I recommend you consult an expert 
before making any major design decisions.

Anyway if your software has to meet a certain safety standard, its 
requirements has to be taken into account from the very start of the 
project. It is not just about your code, but also about your tools, your 
software development process, traceability...etc. Things that are nearly 
impossible to implement afterwards.

On Apr 6, 1:22=A0pm, Dombo <do...@disposable.invalid> wrote:
> Op 05-Apr-11 20:54, KK6GM schreef:
>
> > Thanks for all the comments so far. =A0I understand that hardware
> > interlocks can shield against many software faults, and may be enough
> > protection in some instances, but I'm more curious about those
> > instances where hardware interlocks alone do not suffice. =A0Take the
> > Segway for example. =A0In the general case where software is controllin=
g
> > outputs that can provide or channel significant energy (mechanical,
> > electrical, heat, etc), where that energy output could cause damage
> > and/or injury if the software misbehaves, how does one determine if
> > the software should be developed to any particular body of standards?
> > I'd add that I'm mostly interested in the situation in the USA.
>
> You might want to look into UL1998, though it is difficult to tell if it
> is applicable to your project without knowing the specifics of your
> project and it intended application area/classification. And even with
> the specifics of the project it is not always clear cut which safety
> standards must be adhered to, there is often room for interpretation and
> negotiation. In some cases legislation may dictate certain safety
> standards, but often its a choice of the manufacturer based on market
> demands and/or for legal reasons ('cover your ass') in case things do go
> wrong. In case of the USA the standards you will have to meet may vary
> between states.
>
> Sometimes apparently small aspects can have a big impact on which
> standards the software has to meet. I recommend you consult an expert
> before making any major design decisions.
>
> Anyway if your software has to meet a certain safety standard, its
> requirements has to be taken into account from the very start of the
> project. It is not just about your code, but also about your tools, your
> software development process, traceability...etc. Things that are nearly
> impossible to implement afterwards.

I don't have a particular application at the moment, although there
are some possibilities in the future.

Maybe I should frame my question in a more general way.  If software
controls equipment that could cause damage/injury/death, and hardware
protection cannot fully mitigate the dangers of software failures, is
there ever a case where that software could be developed outside of
any formal standard, without that decision being professional and/or
legal negligence?

KK6GM wrote:

> I don't have a particular application at the moment, although there
> are some possibilities in the future.

So, my assumption that this question was academic in nature was correct 
then.
 
> Maybe I should frame my question in a more general way.  If software
> controls equipment that could cause damage/injury/death, and hardware
> protection cannot fully mitigate the dangers of software failures, is
> there ever a case where that software could be developed outside of
> any formal standard, without that decision being professional and/or
> legal negligence?

When a system you designed, constructed or programmed, fails you will need 
to have a very good defence in court. How good your defence will be depends 
on whether or not you followed "current best practice", adhered to current 
and/or emergent standards, the effort you expended in design analysis, 
system architecture and appropriate assessment of the hazards that your 
system poses. So, amongst all the design work you will need to analyse those 
hazards, assess the risks and design mitigations to ensure that the 
consequences of the hazards being realised are minimised.

The references I quoted in previous posts deal with these aspects quite 
thoroughly. There are some papers in the proceedings of the Safety Critical 
Systems Club which give an excellent treatment on the construction of Safety 
Cases and how these need to stand tests that satisfy "scientific" acceptance 
criteria. 

So, developing such software outside the framework of appropriate standards 
and current best practice could leave you somewhat exposed to the 
possibility of litigation and severe penalties.

-- 
********************************************************************
Paul E. Bennett...............<email://Paul_E.Bennett@topmail.co.uk>
Forth based HIDECS Consultancy
Mob: +44 (0)7811-639972
Tel: +44 (0)1235-510979
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

On 06/04/2011 22:00, KK6GM wrote:
> On Apr 6, 1:22 pm, Dombo<do...@disposable.invalid>  wrote:
>> Op 05-Apr-11 20:54, KK6GM schreef:
>>
>>> Thanks for all the comments so far.  I understand that hardware
>>> interlocks can shield against many software faults, and may be enough
>>> protection in some instances, but I'm more curious about those
>>> instances where hardware interlocks alone do not suffice.  Take the
>>> Segway for example.  In the general case where software is controlling
>>> outputs that can provide or channel significant energy (mechanical,
>>> electrical, heat, etc), where that energy output could cause damage
>>> and/or injury if the software misbehaves, how does one determine if
>>> the software should be developed to any particular body of standards?
>>> I'd add that I'm mostly interested in the situation in the USA.
>>
>> You might want to look into UL1998, though it is difficult to tell if it
>> is applicable to your project without knowing the specifics of your
>> project and it intended application area/classification. And even with
>> the specifics of the project it is not always clear cut which safety
>> standards must be adhered to, there is often room for interpretation and
>> negotiation. In some cases legislation may dictate certain safety
>> standards, but often its a choice of the manufacturer based on market
>> demands and/or for legal reasons ('cover your ass') in case things do go
>> wrong. In case of the USA the standards you will have to meet may vary
>> between states.
>>
>> Sometimes apparently small aspects can have a big impact on which
>> standards the software has to meet. I recommend you consult an expert
>> before making any major design decisions.
>>
>> Anyway if your software has to meet a certain safety standard, its
>> requirements has to be taken into account from the very start of the
>> project. It is not just about your code, but also about your tools, your
>> software development process, traceability...etc. Things that are nearly
>> impossible to implement afterwards.
>
> I don't have a particular application at the moment, although there
> are some possibilities in the future.
>
> Maybe I should frame my question in a more general way.  If software
> controls equipment that could cause damage/injury/death, and hardware
> protection cannot fully mitigate the dangers of software failures, is
> there ever a case where that software could be developed outside of
> any formal standard, without that decision being professional and/or
> legal negligence?

I think you are approaching the subject of standards in a somewhat 
backwards manner.  Dependant on the customer/industry you may be 
required to complete a project according to one or more standards.  In 
the case of standards such as 61508, they will include a mandatory 
hazard analysis which will then lead you into what you need to do to 
make the product "safe".

This is opposed to thinking you have a hazard and then going off to find 
a suitable standard.

These standards deal primarily with how you go about the process of 
developing the product rather than the detail of the design, how you 
complete the design is up to you but if you don't a least demonstrate 
state of the art techniques then in the event of an accident as a result 
of poor design then you will find it hard to defend yourself.

I know this repeats much of what Paul said, but it's worth repeating.

Even in the abscense of a requirement to follow any particular 
standards, if you identify a risk and you do nothing about it then you 
are liable, absolutely no defence.  Even if you didn't identify a risk, 
if that risk is considered "obvious" by experts in the field then you 
will still be liable.

In the case of 61508 and it's related standards, you need a big team to 
execute the process, it's not a task to be undertaken on your own.  In 
fact you can't according to the standard because you must at least 
idenify a functional safety manager who should be independant from the 
engineering function.  Usually this person reports to board level 
through a non engineering director such as the quality director.

There are other independant roles required as well such as for testing 
and requirements gathering.  Oh and on the subject of requirements, it's 
getting more common that the use of DOORS is mandated.

Dave