Want to Make Up to $2007 working indirectly for Google? Find out how HERE
Web Search Evaluator – A complete authoritative guide

Web Search Evaluator – A complete authoritative guide

This blog post may contain affiliate links. Please read the full disclosure here.


Web Search Evaluator

Are you a stay at home mama? Got kids crying around?

Then you will love working as a web search evaluator aka search engine evaluator. This is a perfect stay at home job for moms, for college students and for anyone who loves working from home.

And it is much more stable and consistent than survey sites. Hey, I made $2007 working as a web search evaluator aka search engine evaluator being an Indian although we are paid half of what Americans or Europeans are paid.

Find out what it is and how to work as one –

What is Web Search Evaluator aka Search Engine Evaluation?

Web Search Evaluation aka Search Engine evaluation is a process of evaluating and rating websites and web-pages based on the General Guidelines given by Google or Microsoft. It involves finding out the quality of websites and how relevant or helpful a result is for the queries issued by user.

Look at this image below –

This is what search engine evaluator does

A user (like you and me) issued a query world war ii in his/her mobile device.  And he/she got the result as shown above. (Other results below are truncated due to lack of space)

The result is pretty helpful to a user issuing the query world war ii

Go ahead, type something similar if you feel like.

Now, in search engine evaluation, we have to rate how relevant or useful a particular result is to a given query.

Results could be web-pages, websites, YouTube videos, apps, pdf files, songs, and so on. We also got to evaluate the quality of web-pages or websites.

So in short, a web search evaluation aka search engine evaluation is a process of evaluating websites and web-pages based on the parameters outlined in General Guidelines of Google ( or Microsoft) and also finding out how relevant or useful a particular result is for a given query.

Due to NDA I signed with them, I cannot disclose any specific tasks that we do inside.

Terminologies simplified 

There are different terms used for people who work as a search engine evaluator or web search evaluator or Google Rater. They are also known as internet analyst, raters and so on. They all mean the same thing.

Here are four different terms with which they are known –

  • Web Search Evaluator
  • Search Engine Evaluator
  • Personalized Internet Assessor
  • Internet Safety Analyst
  • Internet Analyst
  • Google Rater
  • Rater

I will be using these terms interchangeably here. They will all mean the same thing. Please don’t get confused.

Different companies uses different terms for them

Appen (previously Leapforce) likes to call them internet analyst.

Raterlabs likes to call them internet analyst

Lionbridge calls it personalized internet assessors. They also refer to it as raters or as internet safety analyst

But they all mean the same thing – Google Rater, search engine evaluator aka web search evaluator.

What do Search Engine Evaluators do?

A search engine evaluator evaluates web-pages and websites and judges its quality. However, a search engine evaluators main work is to rate the relevance and helpfulness of a result to a query issued by a user.

How to work as search engine evaluators?

The process involves applying to the companies mentioned below and waiting for them to get back to you. It can take anywhere from few days to few months. If they don’t get back to you within 6 months, you may re-apply again.

But till its 6 months, do not re-apply with different email accounts because in your first application itself, your IP address would be captured. Re-applying again with different emails would only send you back to the waiting queue.

Initial Requirements Checklist.

  • Have in-depth, up-to-date familiarity with social culture, media, and web culture
  • Excellent comprehension and written communication skills in English
  • Broad range of interests, with specific areas of expertise a plus
  • University degree or equivalent experience (degrees in-progress are acceptable). Advanced degrees a plus. (In my opinion, it is  not really required. Just apply for the positions available).
  • Excellent web research skills and analytical abilities.
  • Ability to work independently with minimal supervision
  • Possess a high-speed internet connection (Cable Modem, Fiber, DSL, etc.)
  • Use of an Android phone version 4.1 or higher, Windows phone version 8.1 or higher, or an iPhone version 4s or higher

Plus, you have to be Googly enough :). I mean, you have to be comfortable using Google to research stuffs, look for things and be comfortable looking around a bit.

Where to apply and how to apply?

Three companies are hiring at the moment –

If you want an in-depth comparison of the 4 evaluator companies, read this article HERE

Applying to these companies is pretty straight forward. Just click on the links above, you are taken to an application page, you just need to fill in your basic details, upload CV and bamm ! You are done.

Lionbridge application process is a bit lengthy though. You need to fill in all your details, academic, work experience and others, and upload CV as well.

{ NB – Leapforce was an independent company founded by a former Google employee. It split in to two – Leapforce and Raterlabs few years back. Raterlabs was created exclusively to hire American search engine evaluators. Now Appen acquired Leapforce and hence both Leapforce and Raterlabs have now became part of Appen. That’s why you would see Appen logo and branding in the previous Leapforce website.

Also you will notice that Raterlabs website design and everything is same as current Appen/Leapforce account. 

Also note that the previous Leapforce website/company is also known as Appen Connect}

Who exactly will you be working for?

Although, you will be working for the aforementioned companies, the real client you will be working for is either Google or Microsoft.

In most cases it is Google.

Google contracts these companies mentioned above, and they in turn look for raters like you and me.

So, although you won’t actually be Google employee, you will indirectly be working for Google in making the world wide web a better place.

How to write CV for search engine evaluator position?

Now writing a good CV is pretty important to be considered for this position. Thousands of people apply to these companies every month. Standing out from them is pretty tough.

Here is my take on how to write a good CV.

  • Make it single page.
  • No need of cover letter.
  • No need to attach your photo.
  • Make sure to not leave any gap years  in your academic, professional details

How long does it take to hear back from them?

It can take anywhere from few days to few months. It all depends on their needs and demand from client.

People often complain that they are yet to hear back from them. Often times, the emails from Appen or Lionbridge ends up in spam folder. And when you get an email from them, the company name “Appen or Lionbridge are not immediately visible on mobile devices.

Where do we mostly check our emails these days? Mobile phones !

Take a note of these emails and make sure to check your spam as well once you apply to these companies

Here are the emails they could email you from –

Appen/Leapforce email addresses –

 Lionbridge email addresses –

Raterlabs – 

I don’t work here or don’t know of any one who works here. So you will have to check your email including spam and other folders regularly if you apply here. And look out for the keywords like qualification test, raterlabs, and so on.

Note that you won’t see the word Appen, Lionbridge or Raterlabs in the email preview unless you click and double check. So be a bit diligent in checking your emails after applying to these companies.

Where to work as one? 

Now this is a tough one to decide. Here is a comparison table –

Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Hires worldwide for all positions except Americans who wants to work as search engine evaluator.Only for Americans
One can work for unlimited number of hours except for Brazilians who are limited to 10-20 hours per week.Their hours are limited, ranging from 20 – 30 hours per week.
They give bonus hours of up to 15 hours per week when the task load is high.
Fixed 26 hours per week
Apart from search engine evaluator position one can work in multiple projects at the same time.Has multiple projects but one person can only work at one project at a time.
Only search engine evaluator project is available.
Candidates are hired as independent contractors for 6 months. Candidates are hired as independent contractors for 12 months. Hired as part-time employees.
Pay rate ranges from $3/hr to $13 per hour.
Indians are paid $3/hr, Americans are paid up to $11/hr while Europeans are paid up to $13/hr
Pay rate ranges from $6.81/hr to $13 per hour.
Indians are paid $6.81/hr, Americans are paid up to $12/hr while Europeans are paid up to $13/hr

$11 - $13/ hr depending on which state you are.
Pays $25 per referral.No referral programThey do have a referral program
Paid via Payoneer under NET 15 days terms Paid via direct bank transfer under NET 45 days termsPaid via Payoneer/direct transfer under NET 15 days term

Do they hire in my country?

Well, they hire from almost any country in the world. You will have to check here whether they are hiring in your country or not.

They don’t hire from most African countries, Pakistan, Nepal, Bhutan, Sri Lanka, Bangladesh.

Do check it out here whether your country and language is in the list or not –

How much can you earn with search engine evaluator job?

You can earn anywhere from $500 – $3800 easily, depending on where you are located. Hey I made $2007 myself, being an Indian.

If you are an American working as a part-time employee who is limited to 26 hours per week, you can make up to

26*4*$11-$13 = $1144 – $1352

Brazilians used to be limited to 20 hours per week. But now since Appen brought in a lot of projects, there isn’t any weekly limit that they got to adhere to.

But everyone else can work as many hours as you want.

How much do they pay you?

As mentioned in the table above, payments ranges anywhere from $3 per hour to $13 per hour depending on where you are located.

Projects available inside?

All the companies got multiple projects inside except for Raterlabs.

Raterlabs is the only company that hires Americans for search engine evaluator position and it only has this one position open. Sure its website mentions openings for multiple countries with several projects. But trust me, its only for Americans and there is just one project available – Search Engine Evaluator aka Internet Analyst.

Appen has multiple projects apart from this. They offer Facebook projects (social media evaluator), Instagram projects and many others. Can Americans work in this projects? Yes, except that they won’t get the evaluator position to work for.

One can work for multiple projects at the same time in Appen.

Anyone who joins Appen after April 2019, and is selected to work as a search engine evaluator aka web search evaluator is eligible for $250 bonus. $150 for working at least 20 hours in the first two weeks and another $100 for working at least 20 hours in the next two weeks.

Lionbridge as multiple positions as well like Maps (Apple), Social media evaluator and so on. One cannot work for multiple projects at the same time.

One can also not work for different companies in the same project at the same time.

How to get paid?

If you want to work for Appen or Raterlabs, you need a Payoneer account. Here is a link to create one. You will get $25 bonus if you sign up via my link. You will get that bonus as soon as you are paid $1000 to your Payoneer account.


What about the qualification test?

There is a tough qualification test before you can work as search engine evaluator. It is one of the toughest exams that I have written in my life.

Hardly 1 out of 100 people gets through it i.e. only 1% qualifies for this position.

It is not a surprise considering the fact that the passing score is 87%

The qualification test consists of 3 parts –

  • Theoretical part
  • Practical part I
  • Practical Part II

How to pass the search engine evaluator qualification test?

Before appearing for the test, you will get a 164 page General Guidelines to study. It will be your Bible, before and after you pass the test.

As soon as you get it, read it like hell. You will need at least 20 hours to cover all of the concepts and go through all the examples.

Here is my recommended time distribution for the three part qualification test –

  • 1 and 1/2 day for the Theory part.
  • 2 and 1/2 day for Practical I
  • 3 and 1/2 day for Practical II

If you fail the test, you may get a one time retake.

Why people fail at search engine evaluator qualification test?

Most of the people fail at the third part of the exam. It is the toughest of all the parts. Some people fail even at the first part.

There are many reasons why people fail.

  • They don’t study enough or can’t study enough.
  • They don’t give much time or don’t have much time understanding the concepts.
  • They rush things through.
  • No attention to minute difference between different types of rating.

To help people crack this test, I build a course, two courses in fact –

One is for complete newbies who got no idea what search engine evaluation is.

The other one is for those who failed the qualification test . You can check it out here –


Thousands apply for this positions and getting a retake chance is getting harder and harder. So don’t take risk. Enroll in one of the courses above. Learn from someone who has been at this game for almost 3 years now. Learn the top 4 reasons why people fail and how to avoid those mistakes.

Final words

I have been working as Search Engine Evaluator for almost 3 years now.  It has benefited me tremendously. It has helped me in many ways –

  • I can figure out fake news very easily.
  • I now understand what is SEO and why certain pages rank high in Google search while others don’t.
  • I take every online info with a handful of salt.

According to me, this is the best work at home job for moms, students, introverts and for anyone who likes to work from home.

Remember, you will be working for Google indirectly. And you will be working to make the internet a better place for all.

So, go ahead, make your decision and sign  up in these three companies –

You also need a Payoneer account to get paid, so sign up here and get your $25 bonus*


Free Search Engine Evaluator Course

Do you want a free course on search engine evaluation? Do you want to learn about it from an active evaluator himself. Then click on the enroll button and get a premium lesson for free (limited time only)

This Post Has 17 Comments

  1. Hi Sam,
    I want to know how many hours per week should one work minimum. And also should I work continuously for one hour or can I check in whenever I am free and stop the work when I am busy. Is there any catch with this regard?

    1. There is no weekly minimum for search engine evaluator job. However you need to complete up to a certain number of tasks in 30 rolling days. Completing that set number of tasks is possible in like 5 – 15 hours, depends on the duration of tasks because every tasks are of different duration.

      Don’t have to work continuously for an hour. You can work for a min ( if thats the task duration), submit the work and then stop.
      You can work whenever you want and stop whenever you want, as long as there are tasks available. During weekends and US holidays, you don’t generally get tasks.

  2. Hey Sam
    I actually work for both Appen and LB. But the clients are different. And the email addresses are also different. Do you think they will find it out?

  3. Hi Sam,
    I’m on registering the project, Arrow with Appen and Ihave a new gmail account named Arrow project since they request me to do.
    But I haven’t had a exam yet. My status as below:

    1. COMPLETED – Register for project
    2. IN PROGRESS – Basic qualification package to be reviewed by Appen Butler Hill Pty Ltd.

    But I got a mail from LionBridge today offering me Internet Assessor.
    I’d like to go with LionBridge. Would you please let me know what shall I do?

      1. Do I need to ask Appen to drop
        me off from the Arrow list even if I have not taken the exam yet?

        1. Hey so sorry for the late reply. It is okay.Your name will be automatically dropped off within 60 days.

          1. Thank you Sam!! Have a nice day!

  4. Hi Sam, I applied with Appen recently and was provided with four projects viz Tx Messenger Qualification, Tigris, Crescent and Noyo to qualify for. I’m more interested in the Search Engine Evaluator role but when I select Search Evaluation from the type field in the All Projects, the page returns with no open projects.

    Would you be kind enough and let me know how to find open projects for the Search Engine Evaluator role and if it takes time to reflect in my Projects dashboard?

    1. Hey, if they have opening for search engine evaluation, that project will be listed under your > All Projects> I am not yet to qualified for. The name of thi projcet is called “Yukon”

  5. thanks bro 👍 for your fast reply, and patiently waiting for your next kick ass article,… also from previous comment does this lionbridge guys allow people to work more than 12months(extend)?, they are very tough nuts to crack even in the selection stages.

    1. Your contract is extended as long as your monthly quality check meets their expected scores. They are much more lenient comapared to Leapforce/Appen when it comes to quality. Their quality head is an Indian guy called Ashwin.

      1. thanks bro for all your patience😊🙏

  6. from the comparison table it says you can work for 12 months for Lionbridge and 6 months for Appen, I know that you started from 2016 and 1) how exactly did you manage to keep working for them all these 3 years given the company merging and 6 month contracts?, after each 6 months what happens exactly, 2) exactly how long could one work for Appen/LF, i recently worked as a social media evaluator for appen and after completing 1 year my contract was closed and when I applied for this WSE they said I could go for Hindi test, but I am not fluent(can read but not so write)in hindi, so I put that on hold, but on my ‘connect’ page I could still see this qualify button for yukon task, At this same time I am aware of Lionbridge, but the work time is less compared to appen, but rate is higher and selection process is tough.3) what should I go for in your opinion, should I go for Appen, as i am already in their system or Lionbridge if gets selected. which is better in terms of work culture and professionalism. you are one of the people i know to have worked for LF/appen all these years, do you anyone who managed to do the same with Lionbridge. Hope you could answer these and keep posting new digital nomad stuffs.

    1. You worked in Nile project (Facebook).It lasts for 12 -18 months. It lasted me 12 months. I also work as search engine evaluator(also knwn as rater aka internet analyst aka personalized internet assessor). The contract for that position is 6 months in Leapforce while it is for 1 year in Lionbrige.
      As long as you maintain good quality scores ( monthly reviews), your contract is renewed. If not, you will be kicked out. My quality check scores have been good all this time.

      So you can work for a long time as long as you got good quality scores.

      You can go for Lionbridge for personalized internet assessor. Appen also hires search engine evaluators. They call them internet analyst. Test is the same for both the companies. It is tough.

      Apart from this position, Lionbridge also got other positions – Online Map analyst is one of them. Very tough exam. I have cracked it though.

      Professionalism is same I must say. Leapforce was better. After its acquisition, not so good anymore. Lionbridge is more lenient when it comes to quality checks. As lons as they don’t know that you worked for Leapforce/ Appen , and as long as the project is different (they have sam projects where the clients are same), you can apply.
      I hope that answers your question.
      More digital nomad stuffs are coming out. Create a blog under 5 mins challenge is coming up. Video already made. 🙂
      Keep checking.

Leave a Reply