Report

{% set tcases_count = dset.progress.UNKN + dset.progress.PASS + dset.progress.FAIL %} {% set tcases_completed = dset.progress.PASS + dset.progress.FAIL %} {% set trun_bg_color = "secondary" %} {% set trun_bg_color = "success" if dset.status == "PASS" else trun_bg_color %} {% set trun_bg_color = "danger" if dset.status == "FAIL" else trun_bg_color %} {% set trun_color = "white" %} {% set trun_color = "black" if dset.status == "PASS" else trun_color %} {% set trun_color = "black" if dset.status == "FAIL" else trun_color %}

A total of {{ tcases_count }} testcases across {{ dset.testplans | length }} testplans(s).

Test execution is {{ ((tcases_completed / tcases_count) * 100)|int }}% complete {% if dset.stamp and dset.stamp.begin %} , started {{ dset.stamp.begin | stamp_to_datetime | strftime("%b. %d %Y") }} {% endif %} {% if dset.stamp and dset.stamp.end %} , ended {{ dset.stamp.end | stamp_to_datetime | strftime("%b. %d %Y") }} , spanning a wall-clock period of {{ "{0:0.2f}".format(((dset.stamp.end - dset.stamp.begin) / 60.0)/60.0) }} hours {% endif %}

Testcase Status Count
PASS {{ dset.progress.PASS }}
FAIL {{ dset.progress.FAIL }}
UNKN {{ dset.progress.UNKN }}
Input given to the runner in the form command-line arguments
{% for key, value in dset.args.__dict__ | dictsort %} {% endfor %}
Name Value
{{ key }} {{ value }}
Configuration of the CIJOE installation
{% for key, value in dset.conf|dictsort %} {% endfor %}
Name Value
{{ key }} {{ value }}

Results

{% for tplan in dset.testplans %} {% set tplan_bg_color = "secondary" %} {% set tplan_bg_color = "success" if tplan.status == "PASS" else tplan_bg_color %} {% set tplan_bg_color = "danger" if tplan.status == "FAIL" else tplan_bg_color %} {% set tplan_color = "white" %} {% set tplan_color = "black" if tplan.status == "PASS" else tplan_color %} {% set tplan_color = "black" if tplan.status == "FAIL" else tplan_color %}

Below is the progress for testcases within this testplan. See the tabs for environment definitions, hooks, and output produced by running testplan-hooks.

Testcase Status Count
PASS {{ tplan.progress.PASS }}
FAIL {{ tplan.progress.FAIL }}
UNKN {{ tplan.progress.UNKN }}

A total of {{ tplan.evars | length }} environment variables are defined. Testsuites inherit testrun environment variables and testcases inherit testsuite environment variables. These environment variables and then defined before executing hooks and testcases, prior to sourcing in the environment definition.

    {% for evar in tplan.evars %}
  • {{ evar }}: {{ tplan.evars[evar] }}
  • {% endfor %}

A total of {{ tplan.hnames | length }} hooks are used per testrun.

    {% for hname in tplan.hnames %}
  • {{ hname }}
  • {% endfor %}

A total of {{ tplan.aux_list | length }} auxilary files, produced by testplan hooks, are listed below

    {% for aux in tplan.aux_list %}
  • {{ aux }}
  • {% endfor %}

Output produced by executing testplan hooks is provided below

{{ tplan.log_content | safe }}
{% for tsuite in tplan.testsuites %} {% set tsuite_bg_color = "secondary" %} {% set tsuite_bg_color = "success" if tsuite.status == "PASS" else tsuite_bg_color %} {% set tsuite_bg_color = "danger" if tsuite.status == "FAIL" else tsuite_bg_color %} {% set tsuite_color = "white" %} {% set tsuite_color = "black" if tsuite.status == "PASS" else tsuite_color %} {% set tsuite_color = "black" if tsuite.status == "FAIL" else tsuite_color %}

The {{ tsuite.name }} testsuite contains {{ tsuite.testcases | length }} testcases.

The testcases are listed below along with testcase status UNKN/PASS/FAIL, descr, source definition, auxilary files, and run log.

    {% for tcase in tsuite.testcases %} {% set ident = tplan.ident +"_"+ tcase.ident %} {% set tcase_color = "secondary" %} {% set tcase_color = "success" if tcase.status == "PASS" else tcase_color %} {% set tcase_color = "danger" if tcase.status == "FAIL" else tcase_color %}
  • {% set show_preq = tcase.status_preq and tcase.status_preq != "UNKN" %} {% if show_preq %} {% set preq_color = "secondary" %} {% set preq_color = "success" if tcase.status_preq == "PASS" else preq_color %} {% set preq_color = "danger" if tcase.status_preq == "FAIL" else preq_color %} {% endif %}
    {% if tcase.wallc %} {% endif %} {% if show_preq %} {% endif %}
    {{ tcase.descr if tcase.descr else "UNDOCUMENTED" }}
    {{ tcase.descr_long if tcase.descr_long else "UNDOCUMENTED" }}

    Source code for the testcase definition

    {{ tcase.src_content }}

    A total of {{ tcase.evars | length }} evars are defined for the testcase before sourcing environment definition

      {% for evar in tcase.evars %}
    • {{ evar }}: {{ tcase.evars[evar] }}
    • {% endfor %}

    A total of {{ tcase.hnames | length }} hooks are run before/after the testcase

      {% for hname in tcase.hnames %}
    • {{ hname }}
    • {% endfor %}

    A total of {{ tcase.aux_list | length }} auxilary files produced by testcase hooks are listed below

      {% for aux in tcase.aux_list %}
    • {{ aux }}
    • {% endfor %}

    Log of output from testcase and hooks executed before and/or after the testcase

    {{ tcase.log_content | safe }}
    {% if show_preq %}

    Log of output from testcase analysis

    {{ tcase.analysis_content | safe }}
    {% endif %}
  • {% endfor %}

A total of {{ tsuite.evars | length }} environment vars are defined for this testsuite, the testcases will define these prior to sourcing the environment definition

    {% for evar in tsuite.evars %}
  • {{ evar }}: {{ tsuite.evars[evar] }}
  • {% endfor %}

A total of {{ tsuite.hnames | length }} hooks per test suite. That is, these hooks are run before and after all the testcases have run.

    {% for hname in tsuite.hnames %}
  • {{ hname }}
  • {% endfor %}

A total of {{ tsuite.aux_list | length }} auxilary files produced by testsuite hooks are listed below

    {% for aux in tsuite.aux_list %}
  • {{ aux }}
  • {% endfor %}

Output produced by executing testsuite hooks is provided below

{{ tsuite.log_content | safe }}

{% endfor %}
{% endfor %}

EOL

In the presence of failure, recall, this too shall pass