Report finds that Microsoft Copilot's human reviewers can see users' uploaded photos and sexualized AI edits

In a typical assignment, a contractor receives the user's prompt and original image, along with two edits produced by Copilot, and must decide which version better follows the instructions. The assessment covers four areas. Reviewers check whether Copilot completed the requested edit, left unrelated parts of the image unchanged, avoided...

Read Entire Article

In brief: Human contractors reviewing Microsoft Copilot's image editor can see users' prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material. Yet the reviewers are not there to flag the content - they are paid to decide how well Copilot carried out the request.

How Copilot edits are evaluated

In a typical assignment, a contractor receives the user's prompt and original image, along with two edits produced by Copilot, and must decide which version better follows the instructions.

The assessment covers four areas. Reviewers check whether Copilot:

• Completed the requested edit.
• Left unrelated parts of the image unchanged.
• Avoided visible defects.
• Produced a strong image overall.

If the prompt asks the system to add a cup to a table, for example, the cup should be the only change.

The instructions reviewed by 404 Media also tell contractors to rely on subjective judgment.

"Trust your intuition - when you glance at the two edited images side by side, which one immediately feels like the better edit? Your gut reaction as a human viewer matters," one set of instructions says.

"When in doubt go with your first impression," the instructions continue. "Human intuition is good at catching subtle quality differences that are hard to articulate."

Sexualized and disturbing material

But many assignments go beyond routine image editing. Contractors said that they had seen requests to enlarge women's breasts, shorten their skirts, reveal more of their legs or place people in sexual positions. Other assignments included upskirt photos, pro-anorexia material and fetish images involving children's cartoon characters.

"Faces are always uncensored, and many of the prompts are sexual in nature and dubiously consensual," one contractor told 404 Media.

The publication granted the worker anonymity because the person was not permitted to speak to the press.

The limits of reviewers' roles

The reviewers are not being hired to moderate those requests or determine whether Copilot should have accepted them. Their job is to judge the quality of its response. In some cases, that means deciding which output more closely matches a user's request for a sexualized edit.

"Who is writing these prompts and who is deciding that basically generating porn is what Copilot is now focused on? It's hard to take things seriously when my focus has to be what model generated the appropriate bust size or which middle-aged woman was put in the appropriate sexually suggestive position," one contractor wrote on an internal message board.

Other contractors said they sometimes opened assignments without knowing what they contained.

"I just came across an image set that consisted of eight upskirt photos," one reviewer wrote while asking for an unsafe-content warning.

Other posts mentioned an apparent animal sacrifice and prompts involving sexual acts or positions. One contractor wrote, "I recoiled," after encountering pro-anorexia content.

Scale and use of contractor data

The documents indicate that at least hundreds of human reviewers sometimes see Copilot submissions. That means photos users upload for editing may be viewed by a contractor as part of Microsoft's process for evaluating the model.

At least one company hiring contractors for the work is Prolific, which offers human feedback for preference tuning, safety evaluations and AI benchmarks. Prolific guidelines included in the material acknowledge that generative AI can expose workers to content that researchers did not expect.

Prolific did not respond to 404 Media's request for comment.

A Microsoft spokesperson told the publication, "Microsoft uses customer data as described in our terms of use, including to improve our products and enforce our code of conduct."

Model improvement versus content moderation

Tech companies have long used contractors to moderate online content and label data used to train AI systems. The Copilot assignments described in the documents serve a different purpose.

Rather than screening material for safety, contractors are comparing outputs to help improve the image editor's accuracy, consistency and ability to follow prompts.