You are currently viewing Search Engine Evaluator Exam Guide: Ace Appen, Lionbridge & Raterlabs QualificationTest

Search Engine Evaluator Exam Guide: Ace Appen, Lionbridge & Raterlabs QualificationTest


This post may contain affiliate links, which means we may receive a commission, at no extra cost to you, if you make a purchase through a link. Please see our full affiliate disclosure for further information.

Updated on 9th July 2020

What is Search Engine Evaluator Exam?

The qualification test/exam conducted by Google with the help of evaluator companies to find the next search engine evaluators is called search engine evaluator exam.

This test weeds out all other applicants and takes only the best of the best candidates who have demonstrated their potential ability for rating by scoring a certain percentage in the exam.

This grueling test serves as a barrier between the deserving and the non-deserving candidates, between the hard-working and not so hard-working candidates.

Who conducts this test?

I suspect this test is conducted by Google themselves with the help of the companies that hire evaluators, on their behalf.

Because every time there is a test in Appen, there is test in Lionbridge as well.

I may be wrong but I am pretty confident that this test is conducted by some department inside Google themselves with the help of evaluator companies.

Which companies uses this test?

This qualification test is used by the following 4 companies

to hire raters on behalf of their client (mostly Google). For iSoftStone, the client is Microsoft, so their test is different.

This post will not apply to iSoftstone qualification test.

Different names for this qualification test

The evaluator companies likes to confuse us by assigning different names to the search engine evaluator job.

Likewise they like to confuse us by naming this test differently in different countries although it is the exact same thing!

Appen might call it as

  • Internet Analyst Test
  • Project Yukon Test
  • Search Engine Evaluator Test
  • Web Search Evaluator Test

and so on.

Lionbridge likes to call it as

  • Internet Safety Evaluator Test
  • Personalized Internet Assessor Test
  • Rater Qualification Test
  • Web Search Evaluator Test, and so on

Raterlabs likes to call it

  • Rater test
  • Internet Analyst Test, and so on.

Same goes for iSoftstone.

Whatever be the name, they all mean the same damn thing!

What is the need for this rater test?

Every day hundreds of people apply to work as a search engine evaluator with the above evaluator companies.

There has to be a way to find the most capable person who has attention to minute details, who can follow the instructions and keep in mind all the KPIs and parameters, before actually rating in real life.

Hence this tough test is conducted to trim down the applicants and select the best ones that get through.

Massive Discount on my premium Search Engine Evaluator Training Courses. Check them out now!

Timer expired

When is this test conducted?

From my experience of 4+ years working in this industry, I can confidently say that this test is conducted almost all year-round.

I am not saying that it is conducted every other month. But I am willing to bet that it is conducted at least 3 – 4 times a year.

But it all depends on their client’s (Google) requirement. If they need more people for a particular country, then the evaluator companies will help conduct more number of tests there.

If they no longer require raters in a particular country, then the evaluator companies won’t conduct any tests there.

How tough is this qualification test?

This is a grueling test and one of the toughest exams that I’ve appeared in my life !

I failed in the first attempt, but somehow got through in the second (consider yourself very lucky if you are awarded second attempt these days – number of applicants is way too high to warrant giving a second chance to candidates).

Unlike school exams, where we just need to score 30 – 45% to pass the test, here you need to score at least 87 – 90% to even qualify !

How soon do we get the result?

There are three parts to this test-

  • For the first (theory) part, you get the result immediately.
  • For the second (practical) part too, you get the test result almost instantly.
  • For the final part, you won’t get the result immediately. It takes some time. I suspect the last part is examined manually while the first two parts aren’t.

Do we get to know the scores?

Never !

Think about it, if you are an employer and you hire hundreds of people every year or month, would you be comfortable or even have time to give out scores for each person while also showing where they’ve made the mistakes?

No right?

Doing so would simply make your life difficult. Same goes for this test. Your actual scores are never revealed, and rightfully so.

What is the passing percentage?

I’ve been blogging about this search evaluator job since December 2016. You can check my archives if you want to verify.

In my almost 3 years of blogging (and 3+ years of working as an evaluator), I’ve referred thousands of people to Appen (formerly Leapforce).

But hardly a few hundreds have gone on to crack this test and work as an evaluator.

I did quick math and found out that hardly 1/100 people pass this test. It is based on my math, backed up by solid data that I have.

So the passing percentage is hardly 1%.

Do we get paid for the time spent studying for the test?

Did you get paid to study in college? I guess you took student loans to pay them instead.

You got in to debt to get a degree that won’t or didn’t get you a decent job.

So please don’t complain!

What is the exam based on?

The exam is completely based on Google supplied General Guideline.

The Guideline is the Bible of search engine evaluator work. And it is the only material that you will be supplied for the test, it is the only thing that you need.

Do I have to be a technical person to appear for the test and work as an evaluator?

Not at all !

You don’t need to be a technical person, you don’t need to do math calculations or code programs or use complex software.

You should just be comfortable using a smartphone (iOS or Android) and a desktop/laptop (Windows, Mac, or Chromebook).

Search Engine Evaluator/Internet Analyst/Rater Exam Strategy for Appen, Lionbridge, Raterlabs & iSoftstone

As soon as you get the General Guidelines from the evaluator company you apply to, read it like hell. You get ten days to read and complete the test. Click on each and every link given (I mean it) and make your own notes.

Go through every screenshot examples, there are hundreds and hundreds of them.

Take special notice of minute but subtle differences in the examples. Even though the Guideline is only 168 pages long, due to hundreds of examples with links (which you have to click and study), it takes a lot of time to soak in that much information.

Here is how you should allocate your time for each part of the rater qualification test :

  • Theoretical Part: Take 1 day aka 24 hours to complete this test. This is an open book Multiple Choice Question type test. Questions are directly taken from the Guidelines. As soon as you hit Submit, you will get an email saying that you have passed the test (or failed the test). Even though it is an open book test, don’t take it lightly. Many people fail at this part of the test.
  • Practical Part I: Allot 2 and half days for this test. This has to be completed in a PC or a Mac or Linux running Chrome or Chromebook. This is a tough exam and hence be very diligent while giving the scores. Your result will be notified immediately as soon as you hit the submit button.
  • Practical Part II: Allot 3 and half days for this test. You’ll need a mobile device (not a tablet) and a PC or a Mac or to complete this test. This is the toughest of all the exams (at least it was for me).

Other helpful tips for this test

Don’t leave anything to the last day

We all tend to leave everything to the last day and rush through. I am guilty of it and probably you are too. This is not the kind of exam that you can rush through. Please follow the time management strategy detailed above. Thank me later.

Take Notes

As you read the guidelines, make sure to go through all the examples and take notes. Take note of the minute but extremely important difference between different kinds of rating examples and types.

Refer to the guidelines all the time

Always go through the guidelines before attempting any questions in real test. Remember this is an open book test. If you are stuck in a question, refer back to the guidelines and make sure that you find that page or article or example that is kind of similar to the question that you are attempting.

Control +F or Command+F  is your best friend

When referring back to the guidelines, always use “Control+F” in Windows or “Command+F” in Mac to look for the keywords contained in that question.


While appearing for the test, you don’t have to complete every question of that part in one sitting. This applies to all parts of the exam.

Finish a question ( which may have multiple sub-questions), then hit “submit and stop”, take a break, and restart the test again.

You have 90 mins to complete each set of questions ( for all parts of the exam) before the question expires.

Search Engine Evaluator Exam Free Course

Want a free 5-day course on search engine evaluator exam?

Enter your name and email in the form below

and get instant access !

Sam Narzary

After making almost $3000 online, I started this blog to document my own journey as a search engine evaluator aka Google rater, bug hunter, website usability tester, and blogger. Now I teach others how to make money online through genuine work from home opportunities. I've been featured in Huffington Post among others. You can read more about my personal struggles and successes in making money online here.

This Post Has 150 Comments

  1. Krishna

    Hi, sam this is krishna from India, if i paid to you amount how long this course is valid, I want to pay 49dolllars, how much amount i have to pay in indian currency.

    1. Sam

      You get lifetime access plus all the updates for free. Once checkout, you will be able to see the amount in INR. Let me know if you have any more questions.

      And please note that you get hundreds of practice quizzes, common mistakes, but not answers to exam questions.

  2. Paresh

    I too have applied appen in 15 days ago so far I have not received the Exam e-mail. India
    How much time does it take

    1. Sam

      It can take anywhere from few days to months. If you don’t hear back from them in 6 months, you can re-apply again.

  3. Krishna

    I got qualification exam from appen, general guidelines which they sent I am unable to understand, can u please me to pass 1st exam.

    1. Sam

      There is nothing there that a normal person can’t understand. The first part of the exam are all from the guidelines. You can check this out for any help you need –

  4. Neha

    Hi Sam,

    I received an invitation from Appen for a project.

    After some basic qualification steps,when I tried to qualify for a project, it says mention the name of the company for which you are currently working as SEE?

    I am qualifying for this project for first time. Also, I have not worked as SEE for any company earlier. What should I mention in that. Appen or none?

    1. Sam

      Leave that as blank or put in Not Available.

  5. tan

    hi sam,

    now it’s almost 3 months no enough posts to rate in Appen. therefore not able to cover production hours and also less payments. is this a normal situation?

    1. Sam

      The payment rate is the same. No decrease in it. Also are you restricted by any means?

  6. Amish Vyas

    Hi Sam
    I have just applied in lionbridge last week and just completed my profile there. The exam is still pending to clear. But later on i have found about leapforce as well and it seems interesting for me as we can work more than 20 hrs. There is no minimum hrs restrictions as well. My question is As I have not appeared for any lionbridge exam yet should i think about leapforce? Which one is better lionbridge or leapforce? And if we started working in lionbridge what we can do if we want to leave lionbridge work and wanted to start work in leapforce. And the last question is there any hourly rate of leapforce less than lionbridge in India?
    Thank you so much in advance ?

  7. Brian

    Hi Sam,

    I just took the Lionsbridge exam and I was notified that i failed within 30 minutes of finishing it, which I thought was weird because all the forum posts and even the exam guidelines themselves made it out as if it would take much more time than that. I wonder if it’s possible they might have already filled the position and didn’t even bother to actually rate my exam just telling me I failed. I really would hate to think that all the hours I invested and the work I put into getting every question as correct as possible was wasted in that way.

    Anyways I wonder do you know if a company like Appen will have a problem of me applying with them if I already tried to apply with Lionsbridge?

    1. Sam

      This test has 3 parts right. For the first 2 parts, you get the exam result immediately. So if if you have completed the first part (Theory), you would be immediately notified about its result. Same goes for the 2nd part. The scores are auto-calculated. Its only the last part that is manually verified ( in my opinion).

      But don’t worry, you can definitely request for a re-test. And do make sure to check out my course here –

  8. Eltis Dhara

    Hi sam is it okay to change the IP address?but still in secure one, because I am going to move house, but still in the same city

  9. Sufiyan

    How much appen get to qualify projects?

    1. Sam Narzary

      You can work for as many projects as you want provided you can maintain quality and time commitment.

  10. tan

    hi Sam,

    I got a mail from Appen contract will be expire on April 2020. So that’s mean i wont able to work there anymore ? can i apply for new projects in Appen?

    1. Sam Narzary

      You got a renewal email saying that your contract is valid till April 2020 or just an email saying that it will expire on April 2020?

      Moreover the contract is with individual projects and not with Appen.

      1. tan

        it says it will expire on April 2020. i work for them as a SEE.

        1. Sam Narzary

          What was your quality score for NM, PQ, SxS for the past 12 months?

  11. Varun

    Hi Sam,
    I pass Theoretical part of the Lionbridge exam but failed in Practical part 1. My question is, Can i apply again in Lionbirdge from someone else ID or information? and another question is, my brother is working in Appen, can i apply their with different IP address ? or Please tell me is there any other that i can get work in Lionbridge or Appen?

    1. Sam Narzary

      If you fail the test, you will get an option to retake the test. If you don’t get it, send them an email and request for the same politely. And if you failed in Practical 1st part, you will find the Practical II even harder.

      And make sure your bro and you don’t work in the same project – meaning both of you don’t work in search engine evaluator project.

      You can apply to Appen as well, but no two persons are allowed to work in the same company from the same household. If they find out, you will be in trouble. You have to be living in a different address to work.

      Anyways I would just recommend you to retake the test. And to make sure that you pass the test, enroll in my premium course here –

      1. Varun

        Thank you for the reply Sam. You helped me a lot.

    2. Kapil

      Lionbridge has very little work.i hardly got 20 hours a month.Support has no answer to work load problems nor do they bother.Its the worst for Indian raters.
      I switched to appen and getting projects and m happy with that coz I do more work in a week than I did with lion ridge in a month.

      1. Sam Narzary

        Awesome ! Every companies has their own pros and cons. But if yous witched from the same project in Lionbride to the same project in Appen, you will be in trouble. Hope you didn’t do that.

        Anyways glad its working for you.

      2. Tanu

        But isn’t $3 very low at Appen? How much are you getting? How much can you earn in a month?

      3. Tanu

        But isn’t $3/hr very low at Appen? How much are you getting? How much can you earn in a month?

  12. Gan

    Hi Sam,

    Can I do any project (e.g. arrow photo task) which appears under the type “search engine evaluation” in projects page as a qualifying project in order to become a search engine evaluator at Appen? or Should I wait for Yukon project to appear on my projects page as a qualifying project? I prefer search engine evaluation and social media evaluation over other small/temporary projects.

    1. Sam Narzary

      Apply for whatever project is available. You never know when Yukon or Nile would appear.

  13. Nade

    Sam, may I know the process for Internet Ads Assessor?
    Can I finish it in single day?
    Because in my second chance I might miss some task, it’s said task completed and asked to wait,,,

    1. Sam Narzary

      Hey Nade, unfortunately I’ve got no idea about Internet Ads Assessor exam. Sorry I won’t be able to help on that.

  14. tan

    hi Sam, in Appen is it ok to work 40 hours per week ? like two projects ?

    1. Sam Narzary

      Yes ! You can work for as many hours you want as long as you meet the project quality requirement and hour requirement.

  15. Neha

    Hi Sam,

    I have tried to check the all projects tab and it only shows me projects related to translations and linguistics.
    Do you have any idea how often do they upload any social media/search evaluation projects.
    Translation related projects have certain requirements to be in that specific country and knowing the desired language.
    There is one linguistic project which states the source language as English and Evaluation of machine translated sentences and no specific target language is stated.
    I am originally from India but living in Italy since 1 year as my husband works here. Will I not be able to submit an application if I have not lived in the country of residence for 5 consecutive years.
    Will they allow me submit application for any project? Can u suggest other companies wherein I can apply for similar positions.

    1. Sam Narzary

      To be a search engine evaluator or a social media evaluator, you have to be living in your country of residence for like 5 consecutive years.That could be the reason why you are not seeing any work related to search/social evaluation.

      The do hire evaluators pretty much all the year round. You can check out website testing sites like Test IO. It is good.

Leave a Reply