Google accused of using novices to verify Gemini AI answers

There’s no denying that the AI ​​still has its fair share of unreliable moments, but one hopes that at least its estimates will be accurate. However, last week Google said it had instructed contract employees evaluating Gemini not miss any prompts regardless of their experience, TechCrunch reports based on internal guidelines it has reviewed. Google shared a preview of the Gemini 2.0 earlier this month.

Google has reportedly instructed GlobalLogic, an outsourcing firm whose contractors evaluate AI-generated output, not to have reviewers miss prompts beyond their expertise. Previously, contractors could choose to skip any prompt that was far outside of their expertise — such as asking a doctor about the law. The guidelines read: “If you do not have the critical experience (eg, coding, math) to evaluate this prompt, please skip this assignment.”

It now claims that contractors were instructed, “You must not skip prompts that require specialized domain knowledge” and that they must “evaluate the parts of the prompt that you understand,” adding a note that this is not an area in which which they have knowledge of. Obviously, the only times contracts can be omitted now are if a large part of the information is missing or has harmful content that requires special consent forms for assessment.

One performer aptly responded to the changes, stating, “I thought the point of the omission was to increase accuracy by giving someone better?”

Google did not respond to a request for comment.