AI employee: how to check a news-processing workflow before publication

A content workflow has several distinct failure points: receiving a source, deciding whether it is a duplicate, producing a draft and authorizing publication. This article lays out checks for those boundaries using my content factory as context. The checks are proposed test scenarios, not results from an executed test suite.

The factory is a team of AI agents in Telegram. It receives material from 13 news sources, filters repeated items, highlights topics, prepares posts in the author's voice, checks facts and adds an image. A person reviews the preview; my approval is required before publication.

Telegram is the known interaction surface here. I am not specifying unverified package versions, queue technology or deployment details.

Define a boundary before testing it

Start with a single stage. For source intake, the input is news material from the specified sources. The output is material available for the next stage, with enough context to find the original. The observable question is whether the reviewer can trace an item back to the correct source.

A useful test set includes:

• a normal item
• an item missing a number
• an unavailable source

These are hypothetical inputs for a reader to construct, not events I am claiming happened in my system.

Expected behavior should distinguish missing data from a factual value. If a source is unavailable, a reviewer should not be told that the source was successfully checked.

Do not infer that a missing item was filtered as a duplicate until intake is known to have received it. A failure earlier in the chain and a decision later in the chain can have the same visible result: an item does not appear in the preview. Separate observations at the two boundaries make diagnosis possible.

Test filtering with counterexamples

A duplicate filter has two kinds of error to consider. It can leave two reports of the same event in the result, or it can remove a distinct event because the wording looks similar. The latter can be missed if the reviewer looks only at retained items.

Construct two pairs for a local review: the same event described with different headlines, and different events using similar terms. Compare the retained and rejected material with the source text. The expected result is one representative of the repeated event and both distinct events retained. This is a proposed check; no classifier accuracy is asserted.

When changing a rule, repeat the earlier checks too. A rule tuned to catch a missed duplicate may start suppressing distinct news. Recording the source and the reason for a decision gives a human reviewer something to inspect when that happens. It does not require revealing private credentials or production data.

Treat a draft as an intermediate artifact

The drafting stage turns a selected topic into text in the author's voice. Fact checking and image selection add further review requirements. A polished paragraph can still turn a forecast into an observed event, or carry over an incorrect number. Review the actual claim against the source, not only whether the paragraph reads fluently.

For a manual test:

• pick a sentence with a number and find that number in the input
• pick a sentence describing a future event and check whether the draft preserves that status

If an input lacks a figure, the draft should not fill the gap with a plausible-looking estimate. These checks are examples for a reader to run on their own material.

An image needs review too. The existence of a file does not establish that it depicts the topic or is appropriate to use. In my workflow, the preview presents the text and image for review before publication.

Make approval a separate state

The system prepares a preview, and I can request changes. The post is published only after my yes. The factory has published 84 posts, none without my approval. The specific boundary is between preparing the material and authorizing an external action.

For a design review, explicitly consider a preview with no approval, one that was rejected for changes, and one approved by its owner. The first two must not be treated as publication authorization. The last can move forward only under the actual application's rules. These are desired scenarios for testing, not claims that a particular state machine or test suite is implemented.

If a process times out after attempting an external action, an unknown result is not proof of failure. Check the target before retrying; otherwise a second attempt may duplicate the action. This is a general operational recommendation, not a description of a measured incident in the factory.

Keep corrections reviewable

Since October 28, the factory has compared my edits with channel rules and added to those rules to avoid repeating mistakes. A changed rule can improve a later draft, but it does not replace review of the actual result. Keep the original error and a counterexample when evaluating a change so that the correction does not silently break a case that previously worked.

In my process, preparing a post went from about an hour to roughly 5 minutes to a finished preview. That is not a latency guarantee for another stack. It is also not proof that each intermediate stage is error-free.

For a broader explanation of why I separate preparation from human decisions, see what tasks an AI employee can take on. For your own implementation, choose one boundary, define its input and expected output, construct a counterexample and inspect the result before adding another autonomous action.

This article was created with the help of AI.