Save Justification for C/C++ Code Coverage Results

When reviewing Polyspace® Test™ code coverage results, you might want to justify or exclude certain results. You can store these justifications in a reusable file. You can create the reusable file by using the Polyspace Platform user interface. If you do not have access to the user interface, you can create this file manually using a text editor.

After creating the reusable justifications file, use it to:

Export Code Coverage Justifications from UI

When you review code coverage results, you might justify portions of the result. Save your justifications into a reusable file using the Polyspace Platform user interface:

  1. Calculate code coverage of your tests.

  2. Open the coverage results. Review the results.

  3. To justify a result, select an item in the Results List. In the Result Details, select Justified as the Status.

  4. After you complete the result review, export your justifications. Click Save to save your justifications in a *.psprof.filter file in the same location as the code coverage results.

After saving the justifications into a reusable file, you can apply the justification every time you calculate the coverage of the same source code. You can also share this file with other projects that use the same source code.

Create Code Coverage Justification File Manually

If you have access to the Polyspace Platform user interface, export the justifications into a file using the user interface. Creating these files manually is error-prone and can result in unexpected behavior.

If you do not have access to the user interface, you can use a text editor to manually create a file that contains code coverage justifications.

Structure of a Justifications File

A justification file is an XML file containing a filter node. This node contains one or more rule nodes. Attributes of the rule node describe which result you want to justify or exclude. For instance, this XML snippet contains a filter node and a valid rule node:

<?xml version="1.0" encoding="utf-8"?>
<filter>
    <rule type="DECISION_OUTCOME" outcomeIndex="1" fileName="src.c" functionName="foo"  index="1" mode="JUSTIFIED" rationale="False outcome not relevant"/>
</filter>
To create a justification file:

  1. Create a filter node in a utf-8 encoded XML file:

  2. Within the filter node, create a rule child node. Each rule node attaches a justification to an element of your code coverage result. In the rule node, describe which element of your code coverage result you want to justify by specifying these attributes:

    • type — Specify what kind of results you want to justify. The type attribute has these valid values:

      1. FILE — Justify all coverage results in a file.

      2. FUNCTION — Justify all coverage results in a function.

      3. DECISION — Justify the coverage results of all outcomes in a decision.

      4. CONDITION — Justify the coverage results of all outcomes in a condition.

      5. DECISION_OUTCOME — Justify the coverage results of one specific outcome of a decision.

      6. CONDITION_OUTCOME — Justify the coverage results of one specific outcome of a condition.

      7. MCDC_OUTCOME — Justify the coverage result of MCDC coverage metric.

      8. RELBOUND_OUTCOME — Justify the coverage result of relational boundary coverage metric.

    • fileName — Specify the name of the file that contains the code coverage element you want to justify.

    • functionName — Specify the name of the function that contains the code coverage element you want to justify.

    • index — Specify the index of the decision or condition that you want to justify. Polyspace counts the index separately for each function and the index count starts at the beginning of a function from zero.

    • condIndex — Specify the index of a condition that is a component of a decision.

    • outcomeIndex — Specify the index of an outcome of a decision, condition, MC/DC, or relational boundary.

    • expr — Specify the expression in the source code that you want to justify.

    • source — Specify the source of the rule. Valid values for this attribute are:

      • USER — Specifies that the rule is user-defined

      • IMPORT — Specifies that the rule is imported from a preexisting justification file.

      • CODE_PROVER — Specifies that the rule comes from a Polyspace Code Prover™ result file.

      • BUG_FINDER — Specifies that the rule comes from a Polyspace Bug Finder™ result file.

  3. After identifying the code coverage result you want to justify, specify how you want to justify the results by using these attributes

    • mode — Specify how the justifications are applied. Valid values are:

      • NONE — The result is not justified.

      • JUSTIFIED — The specified result is shown as justified.

      • EXCLUDED — The specified result is excluded from the result.

    • rationale — Specify a string containing the rationale for justifying an outcome.

Example: Justify Missing Coverage

This example shows how you can justify missing coverage when you do not have access to the Polyspace Platform user interface. In this workflow, you create a justification file manually and then apply the justification in the command line.

Consider this code and its corresponding tests:

  • Source Code:

    int foo(unsigned int A, unsigned int B, unsigned int C){
    
    if(A*B*C>=0){
    if(A||B&&C)
        return 1;
    else return 0;    
    }else{
    return -1;
    }
}

  • Test cases:

    • foo(0,0,1)==0

    • foo(0,1,1)==1

    • foo(0,1,0)==0

    • foo(1,0,1)==1

When you calculate code coverage of these tests, you see that the decision coverage is ¾ or 75%. The decision outcome (A*B*C>=0)==false is not covered. But because all A, B, and C are unsigned integer, this decision outcome does not need to be covered. You can justify the missing coverage by applying a justification file.

To justify the decision outcome (A*B*C>=0)==false:

  • Create an XML file with a filter node.

    <?xml version="1.0" encoding="utf-8"?>
<filter>
</filter>

  • In the filter node, specify a rule that identifies the decision outcome (A*B*C>=0)==false as justified:

    <?xml version="1.0" encoding="utf-8"?>
<filter>
    <rule type="DECISION_OUTCOME" outcomeIndex="1" fileName="src.c" functionName="foo"  index="1" mode="JUSTIFIED" rationale="False outcome not relevant"/>
</filter>
    Here:

    • type="DECISION_OUTCOME" specifies that the rule justifies a decision outcome.

    • outcomeIndex="1" specifies that the false outcome is justified.

    • fileName and functionName specifies the function to which the decision belongs.

    • index="1" specifies that the intended decision outcome is in the first decision of the function.

    • mode="JUSTIFIED" specified that the specified decision outcome is justified.

    • The string specified as rationale explains why the decision outcome is justified.

    Save the file as filt.psprof.filter.

  • Apply this justification when you generate the report:

    polyspace-code-profiler -report -html -report-dir reportFolder -filter-files filt.psprof.filter  result.psprof
    Here, filt.psprof.filter is the justification file and result.psprof is the code coverage result to which you apply the justification. The report containing the justified code coverage results is stored in reportFolder.

    After importing the justification, the decision metric is highlighted in cyan, indicating that the uncovered decision outcome is justified.

Once you justify the missing coverage of an outcome, the justification propagates. See Rules for Propagating Code Coverage Justifications Across Multiple Results.

See Also

Topics