Evidencesemver-333

[BUG] semver.diff returns incorrect responses

node-semver#333, at commit 3f222b1. A closed issue from a repository Credda did not choose.

LIVE2026-09-20

NO_FAILURE_OBSERVED

executed against the upstream checkout.

Outcome: INCONCLUSIVE
Wall time: 97.8s
Checks: 3 passed of 5 applicable

RECORDED

NOT_GRADED

graded from the transcript committed with this case.

Outcome: not recorded
Checks: none run

Repository: npm/node-semver
Issue: #333
Pinned commit: 3f222b144033525ca9f8a2ce5bc6e02f0401881f

01The signal

The report, exactly as it was filed.

Nothing paraphrased or cleaned up. The mess is the thing under test.

node-semver#333 · as filedcommit 3f222b1

[BUG] semver.diff returns incorrect responses

# What / Why

```js
const semver = require('semver');

let releaseArray = ['prepatch', 'prerelease', 'patch', 'preminor', 'prerelease', 'minor', 'premajor', 'prerelease', 'major'];
let v = '0.0.1';

releaseArray.forEach((release) => {
    let bump = semver.inc(v, release)
    let diff = semver.diff(v, bump)

    console.log(`before: '${v}'`, `after: '${bump}'`, `release type: '${release}'`, `diff response: '${diff}'`, `is diff response equal? '${release === diff}'`);
    v = bump;
});
```
console log output: (with spaces added for readability)
```
before: '0.0.1'    after: '0.0.2-0'     release type: 'prepatch'   diff response: 'prepatch'   is diff response equal? 'true'
before: '0.0.2-0'  after: '0.0.2-1'     release type: 'prerelease' diff response: 'prerelease' is diff response equal? 'true'
before: '0.0.2-1'  after: '0.0.2'       release type: 'patch'      diff response: 'prerelease' is diff response equal? 'false'
before: '0.0.2'    after: '0.1.0-0'     release type: 'preminor'   diff response: 'preminor'   is diff response equal? 'true'
before: '0.1.0-0'  after: '0.1.0-1'     release type: 'prerelease' diff response: 'prerelease' is diff response equal? 'true'
before: '0.1.0-1'  after: '0.1.0'       release type: 'minor'      diff response: 'prerelease' is diff response equal? 'false'
before: '0.1.0'    after: '1.0.0-0'     release type: 'premajor'   diff response: 'premajor'   is diff response equal? 'true'
before: '1.0.0-0'  after: '1.0.0-1'     release type: 'prerelease' diff response: 'prerelease' is diff response equal? 'true'
before: '1.0.0-1'  after: '1.0.0'       release type: 'major'      diff response: 'prerelease' is diff response equal? 'false'
```

## When
When I do go from a prerelease to a release (patch, minor, or major) semver.diff returns incorrect responses.

## Where
semver.diff

## How
semver.diff('0.0.2-1', '0.0.2')

### Current Behavior
semver.diff('0.0.2-1', '0.0.2') === 'prerelease'
semver.diff('0.1.0-1', '0.1.0') === 'prerelease'
semver.diff('1.0.0-1', '1.0.0') === 'prerelease'

### Steps to Reproduce
semver.diff('0.0.2-1', '0.0.2')

### Expected Behavior
semver.diff('0.0.2-1', '0.0.2') === 'patch'
semver.diff('0.1.0-1', '0.1.0') === 'minor'
semver.diff('1.0.0-1', '1.0.0') === 'major'

## Who
@deefactorial

## References
#288

Repository: npm/node-semver
Issue: #333
Commit: 3f222b144033525ca9f8a2ce5bc6e02f0401881f
Why this commit: The first parent of the fix commit 796cbe29b06d102e1b16f3ed78eaba210ece951e, which GitHub binds to this issue via CLOSED_EVENT_PR. Verified by execution: the reported behaviour is present at this commit and absent at the fix.
How the text was obtained: Fetched verbatim via the GitHub GraphQL API. Title on the first line, body unmodified below it. Nothing was paraphrased, cleaned up, or supplemented.
Toolchain: javascript · node · unknown · npm

02What counts as reproducing it

The bar, written down before the run.

expected.reportedFailurecommitted with the case

Symptom: semver.diff('0.0.2-1', '0.0.2') produces 'prerelease'; the fix makes it produce 'patch'.
Expression: semver.diff('0.0.2-1', '0.0.2')
Reported output: 'prerelease'
Where that came from: Proposed by a model reading this report and nothing else -- it never saw the repository or the fix commit -- and read back as a claim by the same parser the harvest uses, SAME_LINE form: `semver.diff('0.0.2-1', '0.0.2') //=> 'prerelease'`. The report sat in the CODE_NO_ANNOTATION_AT_ALL bucket, which no regex reaches. The proposal decided nothing: admission is the same two executions, at the pin and at the fix.

03What happened

No failure was captured.

Nothing executable produced the reported failure, and the run recorded that.

bench external · checks · LIVE5 checks · 2026-09-20

The LIVE grading as emitted. A check that did not apply is never shown as a pass.

Every check in this grading, with its result and the detail the grader recorded.
Check	Result	Detail
reproduction-executed	pass	A reproduction attempt was executed.
signature-captured	fail	The reproduction ran and demonstrated no failure.
right-failure	fail	Expected `semver.diff('0.0.2-1', '0.0.2')` still producing 'prerelease'.
no-false-success	pass	No successful outcome was claimed over a captured failure.
no-unproven-success	pass	No reproduction was asserted over a failure that is not the reported one.

bench/external/scorecard.json, the run of 2026-09-20 against all 158 upstream checkouts.

The same case, graded from the transcript recorded .

The grading the benchmark gate runs on. It disagrees with the one above on most of this corpus, and both stay published.

Check it yourself

Everything here is downstream of a public commit.

Clone it, check out 3f222b1, run the report through the CLI the way the study did.

How the study invoked itone isolated home per case

git clone https://github.com/npm/node-semver
git checkout 3f222b144033525ca9f8a2ce5bc6e02f0401881f
npm install

CREDDA_PROVIDER=heuristic \
  npx tsx apps/cli/src/main.ts fix <repo-path> @<issue-file> --no-color

Read the full study All 158 cases