Want to Make Up to $2007 working indirectly for Google? Find out how HERE
Web Search Evaluator – A complete authoritative guide

Web Search Evaluator – A complete authoritative guide

This blog post may contain affiliate links. Please read the full disclosure here.

14 Shares

Web Search Evaluator

This post will tell you anything and everything you need to know about the web search evaluator job.

I will lay threadbare all the inside details about this job that is not available online. I am an active evaluator and hence I know everything about this job.

If you are a stay at home mama, a college student, an introvert, jobless or anyone who is looking to work online, then this is by far, the best online job you will ever find in the entire world wide web.

And it is much more stable and consistent than survey sites.

Hey, I made $2007 working as a web search evaluator aka search engine evaluator being an Indian although we are paid half of what Americans or Europeans are paid.

Find out what it is and how to work as one –

What is Web Search Evaluator aka Search Engine Evaluation?

Web Search Evaluation aka Search Engine evaluation is a process of evaluating and rating websites and web-pages based on the General Guidelines given by Google or Microsoft. It involves finding out the quality of websites and how relevant or helpful a result is for the queries issued by user.

Look at this image below –

This is what search engine evaluator does

A user (like you and me) issued a query world war ii in his/her mobile device.  And he/she got the result as shown above. (Other results below are truncated due to lack of space)

The result is pretty helpful to a user issuing the query world war ii

Go ahead, type something similar if you feel like.

Now, in search engine evaluation, we have to rate how relevant or useful a particular result is to a given query.

Results could be web-pages, websites, YouTube videos, apps, pdf files, songs, and so on. We also got to evaluate the quality of web-pages or websites.

So in short, a web search evaluation aka search engine evaluation is a process of evaluating websites and web-pages based on the parameters outlined in General Guidelines of Google ( or Microsoft) and also finding out how relevant or useful a particular result is for a given query.

Due to NDA I signed with them, I cannot disclose any specific tasks that we do inside.

Terminologies simplified 

There are different terms used for people who work as a search engine evaluator or web search evaluator or Google Rater. They are also known as internet analyst, raters and so on. They all mean the same thing.

Here are four different terms with which they are known –

  • Web Search Evaluator
  • Search Engine Evaluator
  • Personalized Internet Assessor
  • Internet Safety Analyst
  • Internet Analyst
  • Google Rater
  • Rater

I will be using these terms interchangeably here. They will all mean the same thing. Please don’t get confused.

Different companies uses different terms for them

Appen (previously Leapforce) likes to call them internet analyst.

Raterlabs likes to call them internet analyst

Lionbridge calls it personalized internet assessors. They also refer to it as raters or as internet safety analyst

But they all mean the same thing – Google Rater, search engine evaluator aka web search evaluator.

What do Search Engine Evaluators do?

A search engine evaluator evaluates web-pages and websites and judges its quality. However, a search engine evaluators main work is to rate the relevance and helpfulness of a result to a query issued by a user.

How to work as search engine evaluators?

The process involves applying to the companies mentioned below and waiting for them to get back to you. It can take anywhere from few days to few months. If they don’t get back to you within 6 months, you may re-apply again.

But till its 6 months, do not re-apply with different email accounts because in your first application itself, your IP address would be captured. Re-applying again with different emails would only send you back to the waiting queue.

Initial Requirements Checklist.

  • Have in-depth, up-to-date familiarity with social culture, media, and web culture
  • Excellent comprehension and written communication skills in English
  • Broad range of interests, with specific areas of expertise a plus
  • University degree or equivalent experience (degrees in-progress are acceptable). Advanced degrees a plus. (In my opinion, it is  not really required. Just apply for the positions available).
  • Excellent web research skills and analytical abilities.
  • Ability to work independently with minimal supervision
  • Possess a high-speed internet connection (Cable Modem, Fiber, DSL, etc.)
  • Use of an Android phone version 4.1 or higher, Windows phone version 8.1 or higher, or an iPhone version 4s or higher

Plus, you have to be Googly enough :). I mean, you have to be comfortable using Google to research stuffs, look for things and be comfortable looking around a bit.

Where to apply and how to apply?

Three companies are hiring at the moment –

If you want an in-depth comparison of the 4 evaluator companies, read this article HERE

Applying to these companies is pretty straight forward. Just click on the links above, you are taken to an application page, you just need to fill in your basic details, upload CV and bamm ! You are done.

Lionbridge application process is a bit lengthy though. You need to fill in all your details, academic, work experience and others, and upload CV as well.

{ NB – Leapforce was an independent company founded by a former Google employee. It split in to two – Leapforce and Raterlabs few years back. Raterlabs was created exclusively to hire American search engine evaluators. Now Appen acquired Leapforce and hence both Leapforce and Raterlabs have now became part of Appen. That’s why you would see Appen logo and branding in the previous Leapforce website.

Also you will notice that Raterlabs website design and everything is same as current Appen/Leapforce account. 

Also note that the previous Leapforce website/company is also known as Appen Connect}

Who exactly will you be working for?

Although, you will be working for the aforementioned companies, the real client you will be working for is either Google or Microsoft.

In most cases it is Google.

Google contracts these companies mentioned above, and they in turn look for raters like you and me.

So, although you won’t actually be Google employee, you will indirectly be working for Google in making the world wide web a better place.

How to write CV for search engine evaluator position?

Now writing a good CV is pretty important to be considered for this position. Thousands of people apply to these companies every month. Standing out from them is pretty tough.

Here is my take on how to write a good CV.

  • Make it single page.
  • No need of cover letter.
  • No need to attach your photo.
  • Make sure to not leave any gap years  in your academic, professional details

How long does it take to hear back from them?

It can take anywhere from few days to few months. It all depends on their needs and demand from client.

People often complain that they are yet to hear back from them. Often times, the emails from Appen or Lionbridge ends up in spam folder. And when you get an email from them, the company name “Appen or Lionbridge are not immediately visible on mobile devices.

Where do we mostly check our emails these days? Mobile phones !

Take a note of these emails and make sure to check your spam as well once you apply to these companies

Here are the emails they could email you from –

Appen/Leapforce email addresses –

 Lionbridge email addresses –

Raterlabs – 

I don’t work here or don’t know of any one who works here. So you will have to check your email including spam and other folders regularly if you apply here. And look out for the keywords like qualification test, raterlabs, and so on.

Note that you won’t see the word Appen, Lionbridge or Raterlabs in the email preview unless you click and double check. So be a bit diligent in checking your emails after applying to these companies.

Where to work as one? 

Now this is a tough one to decide. Here is a comparison table –

Appen/Leapforce
Lionbridge
Raterlabs
Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Only for Americans
One can work for unlimited number of hours except for Brazilians who are limited to 10-20 hours per week.Their hours are limited, ranging from 20 – 30 hours per week.
They give bonus hours of up to 15 hours per week when the task load is high.
Fixed 26 hours per week
Apart from search engine evaluator position one can work in multiple projects at the same time.Has multiple projects but one person can only work at one project at a time.
Only search engine evaluator project is available.
Candidates are hired as independent contractors for 6 months. Candidates are hired as independent contractors for 12 months. Hired as part-time employees.
Pay rate ranges from $3/hr to $13 per hour.
Indians are paid $3/hr, Americans are paid up to $11/hr while Europeans are paid up to $13/hr
Pay rate ranges from $6.81/hr to $13 per hour.
Indians are paid $6.81/hr, Americans are paid up to $12/hr while Europeans are paid up to $13/hr

$11 - $13/ hr depending on which state you are.
Pays $25 per referral.No referral programThey do have a referral program
Paid via Payoneer under NET 15 days terms Paid via direct bank transfer under NET 45 days termsPaid via Payoneer/direct transfer under NET 15 days term

Do they hire in my country?

Well, they hire from almost any country in the world. You will have to check here whether they are hiring in your country or not.

They don’t hire from most African countries, Pakistan, Nepal, Bhutan, Sri Lanka, Bangladesh.

Do check it out here whether your country and language is in the list or not –

How much can you earn with search engine evaluator job?

You can earn anywhere from $500 – $3800 easily, depending on where you are located. Hey I made $2007 myself, being an Indian.

If you are an American working as a part-time employee who is limited to 26 hours per week, you can make up to

26*4*$11-$13 = $1144 – $1352

Brazilians used to be limited to 20 hours per week. But now since Appen brought in a lot of projects, there isn’t any weekly limit that they got to adhere to.

But everyone else can work as many hours as you want.

How much do they pay you?

As mentioned in the table above, payments ranges anywhere from $3 per hour to $13 per hour depending on where you are located.

Projects available inside?

All the companies got multiple projects inside except for Raterlabs.

Raterlabs is the only company that hires Americans for search engine evaluator position and it only has this one position open. Sure its website mentions openings for multiple countries with several projects. But trust me, its only for Americans and there is just one project available – Search Engine Evaluator aka Internet Analyst.

Appen has multiple projects apart from this. They offer Facebook projects (social media evaluator), Instagram projects and many others. Can Americans work in this projects? Yes, except that they won’t get the evaluator position to work for.

One can work for multiple projects at the same time in Appen.

Anyone who joins Appen after April 2019, and is selected to work as a search engine evaluator aka web search evaluator is eligible for $250 bonus. $150 for working at least 20 hours in the first two weeks and another $100 for working at least 20 hours in the next two weeks.

Lionbridge as multiple positions as well like Maps (Apple), Social media evaluator and so on. One cannot work for multiple projects at the same time.

One can also not work for different companies in the same project at the same time.

How to get paid?

If you want to work for Appen or Raterlabs, you need a Payoneer account. Here is a link to create one. You will get $25 bonus if you sign up via my link. You will get that bonus as soon as you are paid $1000 to your Payoneer account.

PAYONEER SIGN UP

What about the qualification test?

There is a tough qualification test before you can work as search engine evaluator. It is one of the toughest exams that I have written in my life.

Hardly 1 out of 100 people gets through it i.e. only 1% qualifies for this position.

It is not a surprise considering the fact that the passing score is 87%

The qualification test consists of 3 parts –

  • Theoretical part
  • Practical part I
  • Practical Part II

How to pass the search engine evaluator qualification test?

Before appearing for the test, you will get a 164 page General Guidelines to study. It will be your Bible, before and after you pass the test.

As soon as you get it, read it like hell. You will need at least 20 hours to cover all of the concepts and go through all the examples.

Here is my recommended time distribution for the three part qualification test –

  • 1 and 1/2 day for the Theory part.
  • 2 and 1/2 day for Practical I
  • 3 and 1/2 day for Practical II

If you fail the test, you may get a one time retake.

Why people fail at search engine evaluator qualification test?

Most of the people fail at the third part of the exam. It is the toughest of all the parts. Some people fail even at the first part.

There are many reasons why people fail.

  • They don’t study enough or can’t study enough.
  • They don’t give much time or don’t have much time understanding the concepts.
  • They rush things through.
  • No attention to minute difference between different types of rating.

To help people crack this test, I build a course, two courses in fact –

One is for complete newbies who got no idea what search engine evaluation is.

The other one is for those who failed the qualification test . You can check it out here –

Course

Thousands apply for this positions and getting a retake chance is getting harder and harder. So don’t take risk. Enroll in one of the courses above. Learn from someone who has been at this game for almost 3 years now. Learn the top 4 reasons why people fail and how to avoid those mistakes.

Final words

I have been working as Search Engine Evaluator for almost 3 years now.  It has benefited me tremendously. It has helped me in many ways –

  • I can figure out fake news very easily.
  • I now understand what is SEO and why certain pages rank high in Google search while others don’t.
  • I take every online info with a handful of salt.

According to me, this is the best work at home job for moms, students, introverts and for anyone who likes to work from home.

Remember, you will be working for Google indirectly. And you will be working to make the internet a better place for all.

So, go ahead, make your decision and sign  up in these three companies –

You also need a Payoneer account to get paid, so sign up here and get your $25 bonus*

PAYONEER SIGN UP

Free Search Engine Evaluator Course

Do you want a free course on search engine evaluation? Do you want to learn about it from an active evaluator himself. Then click on the enroll button and get a premium lesson for free (limited time only)

This Post Has 29 Comments

  1. Hey Sam, thanks for all the awesome posts! I am thinking of applying to Appen for the internet Analyst position. But I am literally broke. i have a data pack that has a limit of like a Gb/ day. Can’t really afford a wifi for now. I plan to appear for the test using the data pack and then get a wifi connection, in case i get confirmed. Would it be a problem to make such a switch because of the IP? I am from India. What do you suggest i do? Also, i run Linux on my system. I know Appen asks us to use Windows. I don’t wish to remove Linux. Is it possible to work while booting MS Win from a usb? My last question is what is your opinion regarding rev.com?

    I’d really like to thank you for this great blog that you have been running. The best part is the accessibility. It’s a real help for someone like me. Kudos! πŸ™‚

    1. You can appear with data pack. Shouldn’t be a problem when you switch.

      About booting Windows in to Linux machine, I have no idea about it. You will have to email support for it. Once you get an email from them about test, just shoot them an email there.

      I haven’t work with Rev.com, so can’t really say. I heard that they are good for beginners. I think you should give it a try as well.
      Thanks for the kind words. Good Luck !

      1. Hi there Sam, see this is why I love your blog. You are so approachable. πŸ™‚

        I actually applied to the Web Search Evaluator position in the last quarter of 2016. They invited me for a “pre-screening assessment test”. In another email they called it the “Appen online test”. It had 3 sections and they said that the 1st one will determine if I get selected or not.

        Here is an excerpt from the email they sent :

        “Please be advised that Section 1 (Search) is the most important part of the test and your score in this Section will determine if you get selected for the project. It will be a preview of what you will be doing as a Social Media Evaluator. In this section, you will be asked to rate websites based on its relevance to a given search term. Please be objective in rating websites and base your ratings on the guidelines and examples given at the start of the exam and not on your personal opinion.”

        As you can see its a bit confusing because in the the beginning of the same email it said “We received your application for the Web Search Evaluator project. The next step of the application process is the Appen Online Test. This is a short pre-screening test which will measure the 3 key skills that are crucial for this project: Web Evaluation, English comprehension, and basic computer literacy skills. The exam would determine if you are qualified for any of our ongoing Web Search Evaluation projects.”

        it clearly says “Web Search Evaluator”…the position for which I actually applied but in the first part they call it Social Media Evaluator. I need you to clear up this confusion. Was it only a typo?

        Anyways, I did appear for the test and got a confirmation that i had passed. They invited me for “Web5Point Project” and asked to confirm for the same. Due to some confusion on my side, i was unable to confirm back in time. By the time I did, the headcount was already full.

        Later they offered me a Social Media Evaluator position but I declined it since I don’t use social media.

        I read somewhere on your blog that you need to have Gmail as your primary account but all of this communication was done on a non-Gmail account. I think I am still registered with them through the same . I even did a few small transcription and survey tasks. Do you think I need to change it to Gmail. Like would I just need to update my email through the settings? Also, is the test that i passed the main thing or do you need to pass another one during qualification (which they invited me for)? Like are there 2 tests? Because this one was called “pre-screening”? Which one is the main deal? I didn’t read the Google guideline pdf for the one I appeared.

        Thanks dude! πŸ™‚

        1. Alright I understood you now. You were talking about Appen Global. Their project was from Microsoft and that’s the reason why you had a pre-screener test and with it being the most important. And no, we never worked in “Web5PointProject” and I have been doing this since 2016.

          I am talking about Search Engine Evaluator project with Leapforce, now known as Appen Connect. Appen Global acquired it in late 2017 and has since rebranded it as Appen Connect.

          This is the site where I work as SEE and this is how it looks like – https://connect.appen.com/qrp/public/jobs?uref=6c2ff9f87034def5768bd6b5bb65d9df

          In the search engine evaluator project of Appen Connect, there is

        2. no pre-screener test
        3. you need to read the 169 page PDF Guidelines to pass the exam
        4. this is a 3 part exam, based on the Guideline and the its concepts
        5. and you do need Gmail to sign up
        6. The same project is being offered by Lionbridge as well.

          I would recommend you read this and decide which search engine evaluator company to go with – https://www.outsidethatcubicle.com/search-engine-evaluator-companies/

          1. Thanks for clearing up the confusion. I get it now. So earlier one had to appear for “pre-screening” and then for another test, based on the guidelines during qualification? I checked it out and found that I can log in to Appen Global with my non-Gmail id. Here is the link:
            https://go.appen.com/users/sign_in

            But can’t do that here:
            https://connect.appen.com/qrp/core/login

            Inside my Global account i just checked the tests which i cleared in 2016. Here are the results :
            Status Score Attempts
            Search Screening Test / English Comprehension Passed 94% (60%) 1(1)
            Search Screening Test / Technical Skills Passed 100% (60%) 1(1)
            Search Screening Test / Search Passed 76% (70%) 1(1)

            Do you think thats an ok score? Would the new test be more difficult?

            Also, why do they not just migrate my account to Appen Connect? Its a bit confusing to sign up at 2 places for the same company. I have done a few small tasks till this year inside my Global account itself! Btw, would I need to reupload all my documents into Global? I mentioned in my resume to them that i was pursuing an undergrad degree . But the thing is I actually dropped out of it later. Do you think that will be a problem? Do they actually ask for the scanned copies of the degrees? XD

          2. It is not like earlier they had a different type of test and now they have another. Its just that you appeared for a search engine evaluator project from Microsoft and I believe that Appen Global no longer works with Microsoft ( we can’t know these things for sure obviously).

            You will not be able to log in to Appen Connect because it operates independently although owned by Appen Global. I would recommend you to sign up to Appen Connect with gmail, and look for SEE projects or check out other companies I sent in the previous link.

            Regarding your scores , I don’t know what is a good one because I didn’t work in that project in Appen Global. We don’t have such score inside Appen Connect.

            Migration was done last year. A lot of people came in from Appen Global to Appen Connect, not sure if you were aware about it or may be it was just for some projects. And no you don’t need to upload scanned copies of your degree.

          3. Interesting. So, they basically take one’s word for the degree? Is Lionbridge also this relaxed? Anyways, at the end of the last month and again this week I was notified by them of job openings for the analyst position. I just want to be really well prepared before I give the test. Should I finish the Google guidelines first and then apply when I feel confident enough or should I just apply anyways? I was worried if I would get enough time to be prepared in case I get a sudden reply!

            And yeah you are right. I remember way back in 2013-14 they used to work with Bing (MS). They used to send a guideline pdf which made it clear.

          4. Well..it doesn’t mean that we have to take advantage of it right. As far as I know they don’t do background verification. They might in future, who knows? Yeah, if you wanna finish up guidelines, by no means go ahead. And you can check out my premium courses for practice as well – https://searchengineevaluator.net/

            Learn the exact reason why people fail at this test. That alone is worth 10x the price of the course.

  2. Hi, I wondered if you have previously worked for ZeroChaos, does that eliminate you for working for one of the other companies as a search engine evaluator/ rater etc.? Also in terms of NDA, can we include the experience on the resume (obviously not disclosing the client)? Would having it on the resume help or hurt chances of getting into a different company?

    1. Hi Marie, I haven’t worked with ZeroChaos and I am not too sure if they even hire evaluators. Last I heard was that they have shut down rating project.

      I may be wrong though. Please verify.

      And yes it would hurt instead of help you with getting projects in other related companies. Specially if you are going for Lionbridge. Appen/Raterlabs not so much.
      I wouldn’t mention my experience of working as a rater with any company when applying to other companies.

  3. I’m using a PC Windows 7. Do you think that would be issue? Should I upgrade to a higher version?

  4. Hi Sam,
    I want to know how many hours per week should one work minimum. And also should I work continuously for one hour or can I check in whenever I am free and stop the work when I am busy. Is there any catch with this regard?

    1. There is no weekly minimum for search engine evaluator job. However you need to complete up to a certain number of tasks in 30 rolling days. Completing that set number of tasks is possible in like 5 – 15 hours, depends on the duration of tasks because every tasks are of different duration.

      Don’t have to work continuously for an hour. You can work for a min ( if thats the task duration), submit the work and then stop.
      You can work whenever you want and stop whenever you want, as long as there are tasks available. During weekends and US holidays, you don’t generally get tasks.

  5. Hey Sam
    I actually work for both Appen and LB. But the clients are different. And the email addresses are also different. Do you think they will find it out?

  6. Hi Sam,
    I’m on registering the project, Arrow with Appen and Ihave a new gmail account named Arrow project since they request me to do.
    But I haven’t had a exam yet. My status as below:

    1. COMPLETED – Register for project
    2. IN PROGRESS – Basic qualification package to be reviewed by Appen Butler Hill Pty Ltd.

    But I got a mail from LionBridge today offering me Internet Assessor.
    I’d like to go with LionBridge. Would you please let me know what shall I do?

      1. Do I need to ask Appen to drop
        me off from the Arrow list even if I have not taken the exam yet?

        1. Hey so sorry for the late reply. It is okay.Your name will be automatically dropped off within 60 days.

          1. Thank you Sam!! Have a nice day!

  7. Hi Sam, I applied with Appen recently and was provided with four projects viz Tx Messenger Qualification, Tigris, Crescent and Noyo to qualify for. I’m more interested in the Search Engine Evaluator role but when I select Search Evaluation from the type field in the All Projects, the page returns with no open projects.

    Would you be kind enough and let me know how to find open projects for the Search Engine Evaluator role and if it takes time to reflect in my Projects dashboard?

    1. Hey, if they have opening for search engine evaluation, that project will be listed under your > All Projects> I am not yet to qualified for. The name of thi projcet is called “Yukon”

  8. thanks bro ? for your fast reply, and patiently waiting for your next kick ass article,… also from previous comment does this lionbridge guys allow people to work more than 12months(extend)?, they are very tough nuts to crack even in the selection stages.

    1. Your contract is extended as long as your monthly quality check meets their expected scores. They are much more lenient comapared to Leapforce/Appen when it comes to quality. Their quality head is an Indian guy called Ashwin.

      1. thanks bro for all your patience??

  9. from the comparison table it says you can work for 12 months for Lionbridge and 6 months for Appen, I know that you started from 2016 and 1) how exactly did you manage to keep working for them all these 3 years given the company merging and 6 month contracts?, after each 6 months what happens exactly, 2) exactly how long could one work for Appen/LF, i recently worked as a social media evaluator for appen and after completing 1 year my contract was closed and when I applied for this WSE they said I could go for Hindi test, but I am not fluent(can read but not so write)in hindi, so I put that on hold, but on my ‘connect’ page I could still see this qualify button for yukon task, At this same time I am aware of Lionbridge, but the work time is less compared to appen, but rate is higher and selection process is tough.3) what should I go for in your opinion, should I go for Appen, as i am already in their system or Lionbridge if gets selected. which is better in terms of work culture and professionalism. you are one of the people i know to have worked for LF/appen all these years, do you anyone who managed to do the same with Lionbridge. Hope you could answer these and keep posting new digital nomad stuffs.

    1. You worked in Nile project (Facebook).It lasts for 12 -18 months. It lasted me 12 months. I also work as search engine evaluator(also knwn as rater aka internet analyst aka personalized internet assessor). The contract for that position is 6 months in Leapforce while it is for 1 year in Lionbrige.
      As long as you maintain good quality scores ( monthly reviews), your contract is renewed. If not, you will be kicked out. My quality check scores have been good all this time.

      So you can work for a long time as long as you got good quality scores.

      You can go for Lionbridge for personalized internet assessor. Appen also hires search engine evaluators. They call them internet analyst. Test is the same for both the companies. It is tough.

      Apart from this position, Lionbridge also got other positions – Online Map analyst is one of them. Very tough exam. I have cracked it though.

      Professionalism is same I must say. Leapforce was better. After its acquisition, not so good anymore. Lionbridge is more lenient when it comes to quality checks. As lons as they don’t know that you worked for Leapforce/ Appen , and as long as the project is different (they have sam projects where the clients are same), you can apply.
      I hope that answers your question.
      More digital nomad stuffs are coming out. Create a blog under 5 mins challenge is coming up. Video already made. πŸ™‚
      Keep checking.

Leave a Reply

Close