Logo: Relish

  1. Sign in

Project: Cucumber

Publisher: Cucumber

JUnit output formatter

  • @spawn

In order for developers to create test reports with ant
Cucumber should be able to output JUnit xml files

Background
Given
the standard step definitions
And
a file named "features/one_passing_one_failing.feature" with: 
Feature: One passing scenario, one failing scenario

  Scenario: Passing
    Given this step passes

  Scenario: Failing
    Given this step fails
And
a file named "features/some_subdirectory/one_passing_one_failing.feature" with: 
Feature: Subdirectory - One passing scenario, one failing scenario

  Scenario: Passing
    Given this step passes

  Scenario: Failing
    Given this step fails
And
a file named "features/pending.feature" with: 
Feature: Pending step

  Scenario: Pending
    Given this step is pending

  Scenario: Undefined
    Given this step is undefined
And
a file named "features/pending.feature" with: 
Feature: Pending step

  Scenario: Pending
    Given this step is pending

  Scenario: Undefined
    Given this step is undefined
And
a file named "features/scenario_outline.feature" with: 
Feature: Scenario outlines

  Scenario Outline: Using scenario outlines
    Given this step <type>

    Examples:
      | type   |
      | passes |
      | fails  |
Scenarios
  1. one feature, one passing scenario, one failing scenario
  2. one feature in a subdirectory, one passing scenario, one failing scenario
  3. pending and undefined steps are reported as skipped
  4. pending and undefined steps with strict option should fail
  5. run all features
  6. show correct error message if no --out is passed
  7. one feature, one scenario outline, two examples: one passing, one failing
  8. one feature, one scenario outline, two examples: one passing, one failing with --expand option
one feature, one passing scenario, one failing scenario
When
I run cucumber --format junit --out tmp/ features/one_passing_one_failing.feature
Then
it should fail with:
And
the junit output file "tmp/TEST-features-one_passing_one_failing.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="One passing scenario, one failing scenario">
<testcase classname="One passing scenario, one failing scenario" name="Passing" time="0.05">
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="One passing scenario, one failing scenario" name="Failing" time="0.05">
  <failure message="failed Failing" type="failed">
    <![CDATA[Scenario: Failing

Given this step fails

Message:
]]>
    <![CDATA[ (RuntimeError)
./features/step_definitions/steps.rb:4:in /^this step fails$/'
features/one_passing_one_failing.feature:7:in Given this step fails']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>
one feature in a subdirectory, one passing scenario, one failing scenario
When
I run cucumber --format junit --out tmp/ features/some_subdirectory/one_passing_one_failing.feature --require features
Then
it should fail with:
And
the junit output file "tmp/TEST-features-some_subdirectory-one_passing_one_failing.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Subdirectory - One passing scenario, one failing scenario">
<testcase classname="Subdirectory - One passing scenario, one failing scenario" name="Passing" time="0.05">
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="Subdirectory - One passing scenario, one failing scenario" name="Failing" time="0.05">
  <failure message="failed Failing" type="failed">
    <![CDATA[Scenario: Failing

Given this step fails

Message:
]]>
    <![CDATA[ (RuntimeError)
./features/step_definitions/steps.rb:4:in /^this step fails$/'
features/some_subdirectory/one_passing_one_failing.feature:7:in Given this step fails']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>
pending and undefined steps are reported as skipped
When
I run cucumber --format junit --out tmp/ features/pending.feature
Then
it should pass with:
And
the junit output file "tmp/TEST-features-pending.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="0" errors="0" skipped="2" tests="2" time="0.05" name="Pending step">
<testcase classname="Pending step" name="Pending" time="0.05">
  <skipped/>
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="Pending step" name="Undefined" time="0.05">
  <skipped/>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>
pending and undefined steps with strict option should fail
When
I run cucumber --format junit --out tmp/ features/pending.feature --strict
Then
it should fail with:
And
the junit output file "tmp/TEST-features-pending.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="2" errors="0" skipped="0" tests="2" time="0.05" name="Pending step">
<testcase classname="Pending step" name="Pending" time="0.05">
  <failure message="pending Pending" type="pending">
    <![CDATA[Scenario: Pending

Given this step is pending

Message:
]]>
    <![CDATA[TODO (Cucumber::Pending)
./features/step_definitions/steps.rb:3:in /^this step is pending$/'
features/pending.feature:4:in Given this step is pending']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="Pending step" name="Undefined" time="0.05">
  <failure message="undefined Undefined" type="undefined">
    <![CDATA[Scenario: Undefined

Given this step is undefined

Message:
]]>
    <![CDATA[Undefined step: "this step is undefined" (Cucumber::Undefined)
features/pending.feature:7:in `Given this step is undefined']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>
run all features
When
I run cucumber --format junit --out tmp/ features
Then
it should fail with:
And
a file named "tmp/TEST-features-one_passing_one_failing.xml" should exist
And
a file named "tmp/TEST-features-pending.xml" should exist
show correct error message if no --out is passed
When
I run cucumber --format junit features
Then
the stderr should not contain: 
can't convert .* into String \(TypeError\)
And
the stderr should contain: 
You *must* specify --out DIR for the junit formatter
one feature, one scenario outline, two examples: one passing, one failing
When
I run cucumber --format junit --out tmp/ features/scenario_outline.feature
Then
it should fail with:
And
the junit output file "tmp/TEST-features-scenario_outline.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Scenario outlines">
<testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | passes |)" time="0.05">
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | fails |)" time="0.05">
  <failure message="failed Using scenario outlines (outline example : | fails |)" type="failed">
    <![CDATA[Scenario Outline: Using scenario outlines

Example row: | fails |

Message:
]]>
    <![CDATA[ (RuntimeError)
./features/step_definitions/steps.rb:4:in /^this step fails$/'
features/scenario_outline.feature:9:in Given this step fails'
features/scenario_outline.feature:4:in `Given this step <type>']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>
one feature, one scenario outline, two examples: one passing, one failing with --expand option
When
I run cucumber --expand --format junit --out tmp/ features/scenario_outline.feature
Then
it should fail with exactly:
And
the junit output file "tmp/TEST-features-scenario_outline.xml" should contain: 
<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" errors="0" skipped="0" tests="2" time="0.05" name="Scenario outlines">
<testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | passes |)" time="0.05">
  <system-out/>
  <system-err/>
</testcase>
<testcase classname="Scenario outlines" name="Using scenario outlines (outline example : | fails |)" time="0.05">
  <failure message="failed Using scenario outlines (outline example : | fails |)" type="failed">
    <![CDATA[Scenario Outline: Using scenario outlines

Example row: | fails |

Message:
]]>
    <![CDATA[ (RuntimeError)
./features/step_definitions/steps.rb:4:in /^this step fails$/'
features/scenario_outline.feature:9:in Given this step fails'
features/scenario_outline.feature:4:in `Given this step <type>']]>
  </failure>
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>

Last published about 5 years ago by mattwynne.