GOOGLE - Search Quality Evaluator Guidelines
I read blogs, comments and training every day relating to Google guidelines, many of which don't specifically mention Google but are presented as tips, tricks, proper protocol and etiquette. I guess I just presumed this information came from those with extensive experience, or someone studying statistics from Google Analytics for years and years. I have never heard anyone mention that Google actually hires "raters" to browse, read and rate websites from "all over the world," to assist Google with their ranking evaluations.
The "raters" are required to be familiar with search engines and "an expert inGoogle search," but in the same paragraph they are told to "experiment with using operators (e.g., quotes or a dash) in your searches or try using Google’s advanced search option." They are provided links to Wikipedia to understand anti-virus and spyware, along with a list of file types that are generally safe.
The whole second chapter is a mini-course, includes definitions of common terminology and a detailed section on how to search for reputation information.
I can probably determine a lot about the "raters" by the instructions they are given for the task. I can only speculate as to how thorough a "rater" will be as they review every element of a website.
I just happened to come across this document today. A 157-page PDF dated May 11, 2017, entitled Google's Search Quality Evaluator Guidelines and represented to be Googles most current set of standards. Very informative and a must-read for everyone once you get to a certain point in training. Basically, if you don't understand it then you are not ready for the information.
I know there are probably many who have this and probably use it as a template, but there are also many that probably know nothing about it. I know my website doesn't meet all of these standards.
Here is the link. Read and Heed.