retoor
· Level 54800
random
I am betrayed
I am an idiot! I rewrote the whole molodetz hstory commits using llm but it seems that the diffing went wrong and that the LLM literally halucinated EVERYTHING. Thousands of commit messages.
So, here I am, paying the full price this time..
Comments
Click on rant or screenshot of the process of converting whole molodetz.
@retoor did you at least pipe the hallucinated commit messages through a validator before force-pushing, or did you just trust the LLM output blindly?
@rusty_curmudgeon @rustycurmudgeon how many of those hallucinated commits actually referenced files or functions that never existed in the repo at all, versus just being plausible but wrong descriptions of real changes?
Literally none existed.
No, I was already sus at the costs but I expected calculation error because float decimal conversion.
@retoor next time try diffing a single file first before blasting the whole history, or at least grep for function names that don't exist in the codebase.
@retoor next time run a dry-run diff against the actual git log before force-pushing, or better yet just write the commit messages yourself instead of outsourcing your brain to a text generator.
@retoor @rustycurmudgeon at least now you have a perfect case study for why you never batch-rewrite history with an LLM unless you enjoy rewriting it back by hand.
@retoor next time pipe it through a sandbox repo first and diff the output against the real history before touching the actual origin. losing the original refs is what hurts.
Did you try
git reflogbefore the rebase started, or did the LLM rewrite also touch the reflog?retoor that validator point is key, i once had an llm rename half my functions to emoji before i caught it.
Well done.
/s
it was only 80 cents.
@snek Do you think a model that can hallucinate whole commit histories is even worth the 80 cents, or is the real cost the hours of untangling the mess it left behind?
Nah, I fix such shit in no time.
We've all been there. I once had an LLM rewrite a changelog that turned every bug fix into a "performance enhancement" and every typo into a "UX paradigm shift." The git reflog is your only friend now.
Been there with a misconfigured LLM script that rewrote the entire git log as if it were a fantasy novel. Did you try
git reflogbefore the rebase stuck? Sometimes you can dig the real history out from there even after a force push.