Stop Prompting and Start Engineering: Treating LLMs as Unreliable Functions

Most developers start with AI by writing a long prompt and hoping the model returns a valid JSON object. This works 80 percent of the time. In production, that 20 percent failure rate is a disaster. It leads to runtime errors, broken UI components, and endless debugging sessions where you try to "tweak the wording" of a prompt to fix a bug.

I spent three months building a feature that relied on an LLM to categorize user feedback. Every time I updated the prompt to fix one edge case, it broke two other things. I realized I was treating the AI as a magic box rather than a software component. Here is how to move from guessing to engineering.

The Fallacy of the Perfect Prompt

Prompt engineering is often sold as a way to get the perfect answer. But LLMs are probabilistic, not deterministic. Even with the temperature set to zero, you cannot guarantee a consistent output format across different model versions or high-load periods.

If your code assumes the AI will always return a specific key in a JSON object, your code is fragile. You are essentially calling a function that might change its return type at any moment without warning.

Implementing the Wrapper Pattern

Instead of trusting the output, build a validation layer between the LLM and your business logic. Treat the LLM output as "untrusted user input."

• First, define a strict schema for what you expect. Use a library like Zod or Pydantic to validate the response immediately after it arrives. If the validation fails, do not let the data reach your database or your frontend.
• Second, implement a retry loop with a fallback. If the LLM returns a malformed response, send the error message back to the LLM and ask it to fix the specific formatting error. This is far more effective than just retrying the same prompt three times.

Constraining the Output Space

One of the biggest mistakes is asking the AI to "describe" something. Descriptions are variable. Instead, force the AI to choose from a predefined list of enums.

Instead of:

"Categorize this feedback as positive, negative, or neutral."

Try:

"Categorize this feedback. You must choose exactly one value from this list: [POSITIVE, NEGATIVE, NEUTRAL]. Do not include any other text in your response."

By limiting the output space, you make the validation layer simpler and the results more predictable.

Testing for Regressions

How do you know if a prompt change actually improved the system? You cannot rely on a few manual tests. You need a golden dataset.

Create a JSON file containing 50 to 100 examples of inputs and the expected outputs. Every time you change a prompt, run your entire dataset through the model. If you fix the "edge case" but change five other correct answers to incorrect ones, your prompt is a regression.

This turns prompt engineering into a measurable process. You are no longer guessing if it feels "better." You have a percentage accuracy score.

Concrete Takeaway

Stop trying to write the perfect prompt. It does not exist. Instead, build a system that expects the AI to fail.

• Validate every response against a strict schema.
• Use enums to limit the output range.
• Build a test suite of inputs and expected outputs to track regressions.

When you treat the LLM as an unreliable function, you can build reliable software around it.