I built a local-first linter for vague release notes

Release notes often describe a change with phrases such as “improved performance” or “fixed several issues.” Those phrases sound finished, but they do not tell a reader what changed, who is affected, or whether any action is required. I built ReleaseProof to check that narrow problem. It takes two pieces of text: - Change evidence, such as PR titles, issue summaries, test results, or a short diff summary. - The release-note draft intended for users. The checker then looks for vague claims, missing audiences, missing migration steps, a lack of concrete technical or user-facing anchors, and claims that are not clearly supported by the supplied evidence. For example, given evidence that a dashboard query count fell from 18 to 6, a draft saying only “Improved dashboard performance” should be challenged. A better note would identify the dashboard, the measurable change, and the users who notice it. ReleaseProof is deliberately a linter rather than a ghostwriter. It does not generate a replacement changelog or invent details. Its suggestions are narrow prompts about missing facts, leaving the author responsible for what is true. Privacy and scope The current version requires no account and never accesses a repository. Analysis runs in the browser, so pasted text does not leave the page. It also does not claim to verify that code is correct, secure, or ready for production. It only checks the specificity of the writing and its alignment with the evidence supplied by the user. How I calibrated it The deterministic rules currently pass 20 focused cases covering vague fixes, performance claims, breaking changes, migrations, fields, routes, and missing evidence. I then added five real release-note examples from uv, Ruff, pnpm, and Bun. That second pass exposed several problems in the initial rules: technical identifiers were not always recognized as concrete details, developer-facing notes were being forced to name an audience, evidence matching relied too much on literal word overlap, and a link-only release could score too highly. Those cases are now regression tests. Four content-bearing release notes score as specific, while the link-only example is flagged for failing to state an actual shipped change. What I want to learn The rule suite can show that the checker behaves as designed, but it cannot show whether the tool is useful in real work. I am looking for a small number of people who write release notes, changelogs, or user-facing PR summaries to try one sanitized example. If you test it, I would especially like to know: - Which suggestion was useful? - What was a false positive or missed problem? - Did you revise the draft? - Would you use the checker again? You can try ReleaseProof without signing up and submit structured feedback through the link beside the result. Disclosure: This post was drafted with AI assistance and must be reviewed by the project owner before publication. Top comments (0)