flakemap — inspection report

Test reliability across CI history, computed from JUnit, NUnit XML and TRX.

runs 1 tests 18 run range Capture → Capture window 2026-10-18 → 2026-10-18
3 input / ordering warnings — inspect before using this report
0broken
0flaky
0healthy
18insufficient data
0test-runs passed on retry
2test-runs excluded

Tests that fail together

7 tests in 1 exact outcome cohort. Each group has the same normalized outcomes, retry states, skips and missing runs. A shared history is a lead to investigate, not proof of a common cause.

2 tests have ambiguous outcomes and are excluded from grouping. See their individual evidence below.

Download all shared-outcome evidence

Showing 1 of 1 cohorts, ordered by affected test-runs. Final failures and passed-on-retry results remain separate.

7 tests share 1 final-failure runs and 0 recovered runs

Matching run history

Final fail or error (1)
Capture

Members (7)

Showing 7 of 7 affected test-runs. Source positions are 1-based result ordinals. Long text and retry lists are previewed; the JSON download retains every record.

Scroll the evidence table horizontally to inspect every column.

Original failures and recovered retries
Run / environmentTestReported outcomeSource and message
Capture
Environment not recorded
BrokenSetup.NeverReachedAerrorCapture.xml :: testcase 1

NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure.

Capture
Environment not recorded
BrokenSetup.NeverReachedBerrorCapture.xml :: testcase 2

NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure.

Capture
Environment not recorded
InvalidSignature.MissingArgumenterrorCapture.xml :: testcase 4

NUnit label=Invalid: No arguments were provided

Capture
Environment not recorded
MultipleAssertions.TwoFailuresfailCapture.xml :: testcase 5

Multiple failures or warnings in test:

Capture
Environment not recorded
Outcomes.ErrorerrorCapture.xml :: testcase 6

NUnit label=Error: System.InvalidOperationException : Controlled unhandled exception.

Capture
Environment not recorded
Outcomes.FailurefailCapture.xml :: testcase 8

Controlled assertion failure.

Capture
Environment not recorded
Pair<Int32,Double>.CheckfailCapture.xml :: testcase 17

Controlled generic fixture outcome.

Test × run heatmap

Tests under NativeNUnit

BrokenSetup.NeverReachedA 100%

NativeNUnit.BrokenSetup.NeverReachedA

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure. ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure.
BrokenSetup.NeverReachedB 100%

NativeNUnit.BrokenSetup.NeverReachedB

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure. ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6NUnit label=Error, site=Parent: OneTimeSetUp: System.InvalidOperationException : Controlled one-time setup failure.
BrokenTeardown.PassedBody 0%

NativeNUnit.BrokenTeardown.PassedBody

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
InvalidSignature.MissingArgument 100%

NativeNUnit.InvalidSignature.MissingArgument

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • NUnit label=Invalid: No arguments were provided ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6NUnit label=Invalid: No arguments were provided
MultipleAssertions.TwoFailures 100%

NativeNUnit.MultipleAssertions.TwoFailures

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • Multiple failures or warnings in test: ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6Multiple failures or warnings in test:
Outcomes.Error 100%

NativeNUnit.Outcomes.Error

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • NUnit label=Error: System.InvalidOperationException : Controlled unhandled exception. ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6NUnit label=Error: System.InvalidOperationException : Controlled unhandled exception.
Outcomes.Explicit n/a

NativeNUnit.Outcomes.Explicit

Final failure rate (95% CI)unknown: no usable outcomes
Usable outcomesruns: 0; skipped: 1; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)0
Outcomes.Failure 100%

NativeNUnit.Outcomes.Failure

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • Controlled assertion failure. ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6Controlled assertion failure.
Outcomes.Ignored n/a

NativeNUnit.Outcomes.Ignored

Final failure rate (95% CI)unknown: no usable outcomes
Usable outcomesruns: 0; skipped: 1; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)0
Outcomes.Inconclusive n/a

NativeNUnit.Outcomes.Inconclusive

Final failure rate (95% CI)unknown: no usable outcomes
Usable outcomesruns: 0; skipped: 0; excluded: 1
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
ActivityUnavailable: ambiguous outcomes interrupt the failure history.
Timing observations (without retries)0

Recorded evidence

Sources identify a file and its 1-based testcase position. Failure records follow XML order. Attempt timing and total retry cost are not inferred.

Capture — excluded

NUnit result 'Inconclusive' does not establish pass, failure or skip

Capture.xml :: testcase 10 — reported unknown

Controlled inconclusive result.

Outcomes.row.with.dots("a.b") 0%

NativeNUnit.Outcomes.row.with.dots("a.b")

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
Outcomes.row.with.dots("x,y") 0%

NativeNUnit.Outcomes.row.with.dots("x,y")

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
Outcomes.Pass 0%

NativeNUnit.Outcomes.Pass

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
Outcomes.RetryRecovered 0%

NativeNUnit.Outcomes.RetryRecovered

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
Outcomes.Skipped n/a

NativeNUnit.Outcomes.Skipped

Final failure rate (95% CI)unknown: no usable outcomes
Usable outcomesruns: 0; skipped: 1; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)0
Outcomes.Warning n/a

NativeNUnit.Outcomes.Warning

Final failure rate (95% CI)unknown: no usable outcomes
Usable outcomesruns: 0; skipped: 0; excluded: 1
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
ActivityUnavailable: ambiguous outcomes interrupt the failure history.
Timing observations (without retries)0

Recorded evidence

Sources identify a file and its 1-based testcase position. Failure records follow XML order. Attempt timing and total retry cost are not inferred.

Capture — excluded

NUnit result 'Warning' does not establish pass, failure or skip

Capture.xml :: testcase 16 — reported unknown

Controlled warning, not a proven pass.

Pair<Int32,Double>.Check 100%

NativeNUnit.Pair<Int32,Double>.Check

Final failure rate (95% CI)100% [21%–100%] · 1 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Activity1 failures: too few to place.
Timing observations (without retries)1
Failure messages
  • Controlled generic fixture outcome. ×1

Failures, newest first

RunSame commit, other runsOther tests failedNamed in output, or message
Capture
2026-10-18 12:46
commit not recorded6Controlled generic fixture outcome.
Pair<Int32,String>.Check 0%

NativeNUnit.Pair<Int32,String>.Check

Final failure rate (95% CI)0% [0%–79%] · 0 of 1 runs
Usable outcomesruns: 1; skipped: 0; excluded: 0
Passed on retry0 (explicit failed attempts before a pass)
Retry exhaustion0 (every recorded attempt failed)
Flip rateunavailable: no usable consecutive pairs
Same-context rerun disagreementno commits with multiple runs observed
Flakiness score0.000
Timing observations (without retries)1
pass passed on retry fail error skip not run unknown / excluded Rates use the final outcome of each run; retry attempts do not increase the sample size. Rows sorted broken → new → flaky → quiet → insufficient data → healthy, then by score. Columns are runs in chronological order. Click a row for detail. Hover a cell for the run.