electronica Nov 2014
42
D
EVELOPMENT
T
OOLS
New software quality assurance
methods keep everything under control
By Dr. Stefan Weisse and Andreas Gajda,
PLS
Software quality assurance
in electronic control devices,
according to the ISO 26262
standard, is often a fairly difficult
task in practice. A new non-invasive
method now allows for the first time
determining the branch coverage
even with optimized code.
With the complexity of control software
in safety-relevant systems, which has virtu-
ally exploded in recent years, developers are
now more dependent than ever on reliable
methods for verifying and assessing the func-
tional reliability of manually created source
code. The focus is currently primarily on
control-flow-oriented testing methods. Their
great advantage is the largely automatable
application with specially generated artificial
stimuli or parallel to functional tests. Thus,
these test methods are very often recom-
mended or required (IEC 61508, ISO 26262)
as part of standardized processes for quality
assurance of products with software-based
control systems.
Control-flow-oriented methods typically
ensure that in the control processes described,
on the basis of high-level languages all pos-
sible branches in the control-flow-generated
stimuli using special or using functional
tests within the operational environment of
the system are achieved. The different levels
of abstraction of control-flow-oriented test
methods in terms of coverage of the control
flow fall broadly into the following categories:
coverage at function level, coverage at source
text lines level, coverage at statement/com-
mand level, coverage of the branches – edge
coverage, coverage of the paths of a module,
and coverage of the individual conditions
and decision terms. For the exact definition
of each category, there are similar but not
completely identical definitions from sev-
eral authors. Classification according to the
DO-178B standard is the basis for the follow-
ing information.
When classifying these categories, measure-
ment of the coverage at source text level is
always assumed. For this, typical methods
use test instrumentation code additionally
built-into the test code, by means of which
the corresponding measurement data are
determined. The measurement itself can then,
based on a static code simulation or by trans-
lation of the generated overall code from test
and instrumentation codes, be carried out in
machine code and then test-run to determine
the measured values. However, this method
has several serious disadvantages. Due to
the instrumentation, in particular the instal-
lation of measurement probes, the run-time
behavior of the system will change. The addi-
tional code necessary for the instrumenta-
tion obviously also increases the overall size
of the application to be tested in the test case.
Instrumentation is only sensibly applicable
in non-optimized code. For real-time-criti-
cal control systems, the use of this method is
therefore very limited, because adherence to
the time behavior as well as maintaining the
memory layout are absolutely necessary for
parallel execution of coverage and function
tests. For the test on such real-time-critical
systems, methods by which the data required
for coverage measurement are determined
non-invasively seemmuch better suited. Since
in this case, the installation of additional code
is excluded, other methods must be used here
for obtaining the required measurement data.
The following describes how the con-
trol-flow-information, for example as part of
the debug-information, can be additionally
provided by the compiler. As a result, this
information must not be embedded as addi-
tional instrumentation code in the program
structure, but rather only used in the evalu-
ation of trace information to determine the
coverage.
The code optimization can change the struc-
ture of the machine code in such a way that
subsequently a clear mapping of the basic
blocks of the source code level on basic blocks
of the machine code level is no longer possible.
Thus, during optimization, for example, basic
blocks are duplicated, combined or elimi-
