Want to Make Up to $4000 working indirectly for Google? Find out how HERE
Search Engine Evaluator Exam – Crack Leapforce, Appen, Raterlabs & Lionbridge Test

Search Engine Evaluator Exam – Crack Leapforce, Appen, Raterlabs & Lionbridge Test

This blog post may contain affiliate links. Please read the full disclosure here.


What is Search Engine Evaluator Exam?

The qualification test/exam conducted by Google with the help of evaluator companies to find the next search engine evaluators is called search engine evaluator exam.

This test weeds out all other applicants and takes only the best of the best candidates who have demonstrated their potential ability for rating by scoring a certain percentage in the exam. .

This grueling test serves as a barrier between the deserving and the non-deserving candidates, between the hard-working and not so hard-working candidates.

Updated August 15th 2019

Who conducts this test?

I suspect this test is conducted by Google themselves with the help of the companies that hires evaluators, on behalf of Google.

Because every time there is a test in Appen, there is test in Lionbridge as well.

I may be wrong but I am pretty confident that this test is conducted by some department inside Google themselves with the help of evaluator companies.

Which companies uses this test?

This qualification test is used by the following 4 companies

to hire raters on behalf of their client (mostly Google). I suspect iSoftstone’s client is Microsoft, so their test could be a little different.

Different names for this test

The evaluator companies likes to confuse us by assigning different names to the search engine evaluator job.

Likewise they like to confuse us by naming this test differently in different countries although it is the exact same thing!

Appen might call it as

  • Internet Analyst Test
  • Project Yukon Test
  • Search Engine Evaluator Test
  • Web Search Evaluator Test

and so on.

Lionbridge likes to call it as

  • Internet Safety Evaluator Test
  • Rater Qualification Test
  • Web Search Evaluator Test, and so on

Raterlabs likes to call it

  • Rater test
  • Internet Analyst Test, and so on.

Same goes for iSoftstone.

Whatever be the name, they all mean the same damn thing!

What is the need for this rater test?

Every day hundreds of people apply to work as a search engine evaluator with the above evaluator companies.

There has to be a way to find the most capable person who has attention to minute details, who can follow the instructions and keep in mind all the KPIs and parameters, before actually rating in real live.

Hence this tough test is conducted to trim down the applicants and select the best ones.

When is this test conducted?

From my experience of 3+ years in this industry, I can confidently say that this test is conducted almost all the year round.

I am not saying that it is conducted every other month. But I am willing to bet that it is conducted at least 3 – 4 times a year.

But it all depends on client’s (Google) requirement. If they need more people for a particular country, then the evaluator companies will help conduct more number of tests there.

If they no longer require raters in a particular country, then the evaluator companies won’t conduct any tests there.

How tough is this qualification test?

This is a grueling test and one of the toughest exams that I’ve appeared in my life !

I failed in the first attempt, but somehow got through in the second (consider yourself very lucky if you are awarded second attempt these days – number of applicants is way too high to warrant giving a second chance to candidates).

Unlike school exams, where we just need to score 30 – 45% to pass the test, here you need to score at least 87 – 90% to even qualify !

How soon do we get the result?

There are three parts of the exam –

  • For the first (theory) part, you get the result immediately.
  • For the second (practical) part, you get the result immediately too.
  • For the final part, you won’t get the result immediately. It takes some time. I suspect the last part is examined manually while the first two parts aren’t.

Do we get to know the scores?

Never !

Think about it, if you are employer and you hire hundreds of people every year or month, would you be comfortable or even have time to give out scores for each person while also showing where they’ve made the mistakes and all.

No right?

Doing so would simply make your life difficult. Same goes for this test. Your actual scores are never revealed, and rightfully so.

What is the passing percentage?

I’ve been blogging about this search evaluator job since December 2016. You can check my archives if you want to verify.

In my almost 3 years of blogging (and 3+ years of working as an evaluator), I’ve referred thousands of people to Appen (formerly Leapforce).

But hardly a few hundreds have gone on to crack this test and work as an evaluator.

I did a quick math and found our that hardly 1/100 people pass this test. It is based on my math, backed up by solid data that I have.

So the passing percentage is hardly 1%.

Do we get paid for the time spent studying for the test?

Did you get paid to study in college? I guess you took student loan to pay them instead.

You got in to debt to get a degree that won’t or didn’t get you a decent job.

So please don’t complain !

What is the exam based on?

The exam is completely based on Google supplied 166 page General Guideline.

The Guideline is the Bible of search engine evaluator work. And it is the only material that you will be supplied for the test, it is the only thing that you need.

Do I have to be a technical person to appear for the test and work as an evaluator?

Not at all !

You don’t need to be a technical person, you don’t need to do math calculations or code programs or use complex softwares.

You should just be comfortable using a smartphone(iOS or Android) and a desktop/laptop (Windows, Mac or Chromebook)

Search Engine Evaluator/Internet Analyst/Rater Exam Strategy for Appen, Lionbridge, Raterlabs & iSoftstone

As soon as you get the General Guidelines from the evaluator company you apply to, read it like hell. You get ten days to read and complete the test. Click on each and every links given (I mean it) and make your own notes.

Go through every screenshot examples, there are hundreds and hundreds of them.

Take special notice of minute but subtle differences in the examples. Even though the Guidelines is only 166 pages long, due to hundreds of examples with links ( which you have to click and study), it takes a lot of time to soak in that many information.

Here is how you should allocate your time for each part of the rater qualification test :

  • Theoretical Part : Take 1 day aka 24 hours to complete this test. This is an open book Multiple Choice Question type test. Questions are directly taken from the Guidelines. As soon as you hit Submit, you will get an email saying that you have passed the test (or failed the test). Even though it is an open book test, don’t take it lightly. Many people fail at this part of the test.
  • Practical Part I : Allot 2 and half days for this test. This has to be completed in a PC or a Mac or Linux running Chrome or Chromebook. This is a tough exam and hence be very diligent while giving the scores. Your result will be notified immediately as soon as you hit submit button.
  • Practical Part II : Allot 3 and half days for this test. You’ll need a mobile device (not a tablet) and a PC or a Mac or to complete this test. This is the toughest of all the exams (at least it was for me).

Other helpful tips for this test

Don’t leave anything to the last day

We all tend to leave everything to the last day and rush through. I am guilty of it and probably you are too. This is not the kind of exam that you can rush through. Please follow the time management strategy detailed above. Thank me later.

Take Notes

As you read the guidelines, make sure to go through all the examples and take notes. Take note of the minute but extremely important difference between different kinds of rating examples and types.

Refer to the guidelines all the time

Always go through the guidelines before attempting any questions in real test. Remember this is an open book test. If you are stuck in a question, refer back to the guidelines and make sure that you find that page or article or example that are kind of similar to question that you are attempting.

Control +F or Command+F  is your best friend

When referring back to the guidelines, always use “Control+F” in Windows or “Command+F” in Mac to look for the keywords contained in the exam.


While appearing for the test, you don’t have to complete every questions of that part in one sitting. This applies to all parts of the exam.

Finish a question ( which may have multiple sub-questions), then hit “submit and stop”, take a break, and restart the test again.

You have 90 mins to complete each set of question ( for all parts of the exam) before the question expires.

Search Engine Evaluator Exam Free Course

Want a free 5-day course on search engine evaluator exam?

Enter your name and email in the form below

and get instant access !

This Post Has 116 Comments

  1. I am awaiting to take the qualification test to make my account active. I applied to 4 different projects. Should I take the test for the first available? Do I need to take all of the qualification tests to obtain each project? I applied to Truckee, Noyo, Atascadero, and Fonseca. Thanks for your help!

    1. Yes there is qualification test for each project except for Nile & Noyo. You can take the test for whatever project you are approved for.

  2. Hi… I have received an email from Appen Recruitment team through which they have invited me to join a data collection project called Butano on 9th august. Upon applying, my account was created and I was asked to click the qualify button on the homepage. The page states that my account is not yet active and I have to qualify for atleast one project for the account to be active. However, I cannot see the project Butano to click on the qualify button. There are several other projects on the page and it states that there are no projects matching my details. This is strange. They sent me the project name and I cant see that project on the homepage. Also there is no clarity of role. No follow-up email received by them for test or anything.

    I also registered for the five day SEE course and yet to receive the follow up email.

    1. Hi, did you check the list of projects that you are yet to qualify? Check in the “all projects tab” and hit qualify for everything that you see. Once you hit qualify button, they will check your profile, and then invite you for any projects that are hiring. When that is done, you get all the details of the project.

      Regarding the invite that you got, may be the project was full and hence couldn’t accommodate you for now. Keep looking out for other projects and keep hitting that qualify button.

      About the course – did you confirm your subscription? In order to avoid spams, I have set up a two-step subscription. You need to opt in and than click on the confirm email that you get as soon as you sign up for the course. Kindly check your spam, promotions or social inbox for any email from Search Engine Evaluator. If you haven’t got any emails, please let me know.

    2. Hi, did you check the list of projects that you are yet to qualify? Check in the “all projects tab” and hit qualify for everything that you see. Once you hit qualify button, they will check your profile, and then invite you for any projects that are hiring. When that is done, you get all the details of the project.

      Regarding the invite that you got, may be the project was full and hence couldn’t accommodate you for now. Keep looking out for other projects and keep hitting that qualify button.

      About the course – did you confirm your subscription? In order to avoid spams, I have set up a two-step subscription. You need to opt in and than click on the confirm email that you get as soon as you sign up for the course. Kindly check your spam, promotions or social inbox for any email from Search Engine Evaluator. If ypu haven’t got any emails, please let me know.

  3. hi Sam, I was selected for the Arrow and Arrow Photo business. They asked for my information such as name and address. I think Final paperwork is next. I’ve been waiting two weeks. Is there a time limit for them to return?

    1. Hey sometimes they do take time, but it is always best to follow up with them. Please email them in that email from where you got contacted first. It will start will something like [email protected]. Something like that.

  4. Hi Sam,
    I signed up for the five day SEE course, but I haven’t received the follow up email. It’s not in my spam either.

    1. Hey I am so sorry. I will manually add you to the free course. You should get it by tonight.

    2. Could you please tell me the email where I should send the course?

  5. Hi Sam
    Its elt from indonesia, is it the same or different exam for all the project at the LF/appen? There is micro noyo, micro nyasa, truckee, yukon, arrow project in my account. I am afraid of the exam @[email protected]

    1. Search engine evluator is known as Yukon inside Appen and Internet Safety Analyst in Lionbridge. The exams are different for all the projects. What I am talking about is for search engine evaluator project because this is a long term project. Noyo has no test, arrow has a qualification test, so does truckee. Truckee qualfication test is easy, Arrow a bit tough.

      I don’t focus on other projects as most of them are short term project. You want to be doing long term projects that gives you consistent monthly income.

      1. Thank you sam…

  6. I wish I saw your post after I failed the exam. I got the link for the test without the guidelines, and I was afraid of missing the opportunity so I answered the test, I did not know we have 7 days to answer. A day after they send me the email saying I failed, I received the guidelines, I was so frustrated cause I did not even have all the information to pass 🙁 .

    1. Aahhh. So sorry to hear that. But you do get an option to retake the exam. In case you didn’t get an email to retake the exam, email them politely saying why failed the test. They should let you retake the test. Good Luck !

      1. Thanks for answering. I did write them, I explained them that they send me the guidelines after the test so I didn’t even have the chance to pass, but they answered saying that it was shame but the project had just one attempt :(. I guess I missed my chance this time, but I’ll keep trying. Thanks for your advices!

  7. 1. Appen or Lionbridge for an Indian SEE? In terms of work and pay? Can I do both at the same time?
    2. Why has the pay been reduced at Appen? How does it impact an SEE from a year or two ago when it was $7/hour? Any other options for better pay and work?

    1. 1) If you want higher payrate, go with Lionbridge. If you want more projects go with Appen. Some people work in both the companies (at the same time) in different projects. They risk losing their contract. Lionbridge has a strict policy of not allowing their workers work with other companies.
      2. I don’t know that. Lionbridge pays $6.81/hr, but their hours are kept to 20 hours per week and you can’t work in multiple projects at the same time.

      1. Hello Sam, I have worked with appen and one month ago I applied for a SEE job in lionbridge and I have provided our appen experience there, but my application was rejected by lionbridge. Is it because i have provided the details that i am working in other SEE provider company (Appen) and thus they rejected me because of their policy ? or it is something else that’s why my application rejected

        1. Yes. Exact reason why your application was rejected.

  8. Grettings, sam
    I am trying to take the third exam, however i am having difficulties with my iphone. Do you have any suggestions on how to access it the most simple way.
    J ruano

    1. aaahhh.. I am so sorry for the late reply. If you haven’t taken already, you could just use “raterhub.com/url” in your phone browser to access send to device tasks of the last part.

      Let me know how it went.

  9. Hi Sam

    I want a free 5-day course on search engine evaluator exam? I got recently failed in lionbridge rater exam.

  10. Hi sam, i am going to apply for the SEE exam, can you please tell how many times can i apply for that test in case i get negative result in first attempt, one of my known is working as SEE for appen can i apply from that system or do i need a separate computer to apply? I mean does Appen remotely monitor the system on which SEE agent is already working??

    1. You may get two chances?

      One of your who is working sorry?

      1. My cousin is working in appen as SEE

        1. Okay, You are going to be need a different system, different IP address and you shouldn’t be living in the same household.

  11. Hi Sam, I applied for raters job and got an email after two weeks. currently I have two options, namely “ch*rry” and “em*rald”. When I click on qualify it says I’ll have one hour to review some guidelines and give a short test. Is the test really hard? I dont want to fail the test.

    1. There are multiple projects inside Appen. The projects you mentioned is not search engine evaluation project. Search engine evaluator job is called as Yukon inside Appen. Look out for Yukon in your all projects tab and apply for it when that project pops up.

      For now, you can try and pass the test for the projects you have.

  12. very informative blog.saw ur answer in quora i think which led me to your portal.
    Have recently passed lionbridge search engine evaluator exam..how is the job is it worth it?

      1. Hey sam its almost gonna be a month i passed lionbridge exam.I got all the username and password and even bank account setup.
        I am yet to receive a task.I have mailed them many times they just reply thanks for patience keep logging in tasks may drop anytime as my locale is a new one(punjabi).
        i guess i need to keep patience only.lol?

        1. Hey Kapil, I have no idea about Punjabi locale unfortunately. Only their support can tell you.

          I guess tasks might be a bit low, considering it is a Punjabi locale.

          I am wondering what is the pay rate for you guys though?

          1. Yeah i did email them they keep saying keep checking regularly tasks may drop anytym.
            Its been a month not a single task.l
            Can only wait.lol
            6.8$/hour is what they are offering atm.

          2. Okay..I guess you will have to wait and see..

  13. Hello Sam,

    I’m from the Philippines and I applied for the post of Search Engine Evaluator after reading your blog. A few days ago, I received an email from Appen informing me that they will have an upcoming project soon and asked my level of experience in various fields through a survey. Is this related to my application for the SEE post? Also if ever I get selected and pass the exams, can I work remotely even if I’m traveling to the US for vacation?

    1. Hi Mary, I am not too sure what those surveys are for. There shouldn’t be any surveys for SEE.

      At the moment, they are hiring for a lot of projects. Could be some other projects not SEE. Unfortunately, you can’t work these projects from abroad. You will have to inform them that you are going out. Depending on the project, you should be able to take up to 3 weeks off though.

  14. Hey! I had applied to leapforce two years ago. I guess they rejected me but I want to again apply with a better CV which i did but as i have already applied , I guess I cannot apply again. Help on how can I apply again ?

    1. Have you re-applied again like recently? If you did apply again, then don’t re-apply again until its already 6 months.

      You can always apply to Lionbridge though.

      1. Hello Sam ,I’m from India. Are you sure that we can always reply to lionbridge ? With same email id or different email id?. I have terminated from the project before. Thanks

        1. If you have been terminated before, it is very difficult to work with them again. Your name and ID will the same right. So.

          However, you can try for different projects with different email ID, different IP , different devices.

  15. How do you do your taxes? Please make article on it.

    1. We are responsible for our own taxes. Americans get form 1099 from inside the website and they got to do the needful.

Leave a Reply