Overview

The North Kingdom was given the responsibility of creating a fresh 3D avatar for Tengai, a company known for its use of physical avatars in job candidate interviews on a Swedish market. With advancements in technology, Tengai aimed to transform the avatar into a fully digital experience with North Kingdoms’ help. I was assigned the task of conducting testing of the newly designed avatar and assessing users' reactions.

Result

As a result, Tengai successfully launched the new avatar in a short span of time. If you're interested, you can now sign up for a demo of the avatar here.

Role

Researcher

Tools

Figma

Duration
2 months

Company

North Kingdom for Tengai

Responsibilities

Participating in team check-ins, advising on the research topics, scheduling research participants, conducting remote moderate testings, analyzing results, creating a deck with actionable recommendations;

Test Methodology

In order to ensure the participants' job application experiences were recent and relevant, we specifically sought individuals who had been hired within the past year for the testing. That aspect was important considering the avatar task was to interview job seekers as a first contact with a potential hiring company. The participant sample was 20 participants across different industries. To create an authentic user experience, we chose an avatar that closely mimicked real interactions, incorporating natural voice and pauses for potential user responses in the presented demo. The test itself was conducted remotely using the Lookback platform. I was the only Researcher on a team.

Hypothesis

We believe that validating the avatar attributes will allow us to understand if users react positively toward the avatar.

Study Objectives

Understand how users react to the first impression of the avatar.

Understand how users would describe the avatar.

Understand how users would rate those descriptors

Understand if users would like to change something about the avatar

Understand if users would feel comfortable in its presence.

Understand how users would envision the experience

Interview and testing

Study Plan

Interview: Context questions

Understanding past experience and expectations around the first interviews with companies.
Understanding candidates’ preferences and hesitations around an interview with an AI Robot.

Questions

1. When was the last time you interviewed for a new job?

2. What do you appreciate during your first job interview with a new company?

3. How would you react to the fact that an AI Robot would conduct your

4. first interview? Why?

5. If it’s optional, would you turn on your camera if an AI Robot would be interviewing you? Why?

6. Which device would you use for that interview? Desktop or mobile?

7. Why?

8. Is there anything that you find particularly difficult about the interview conducted by an AI robot?

9. What could make you feel more comfortable?

Avatar Demo Testing

Understanding how users would describe and react to the avatar demo.
Understanding which descriptors they would choose and how they would rate it.

In this part of the testing participants were interacting with a prototype that would guide them through the testing experience of the avatar demo.
The prototype contained few parts:

1. Step: Watching the demo.

2. Step: Contextual questions about the demo
   Following questions: What is your first reaction to the avatar? How would you describe it yourself?

3. Step: The participant saw on the screen poll of adjectives that needed to choose from.
   Following question: Which of the presented words would you choose to describe the avatar?

4. Step: Rating selected descriptors.
   

Interview: Additional Areas

Understanding potential areas that users disliked and liked.
Understanding how users would react to the possibility of customization and if they would change anything about the avatar.

Questions

1. Would you feel comfortable speaking to the avatar?

2. What, if anything, did you like the most about the avatar?

3. What, if anything, did you like the least about the avatar?

4. Wouldyouchangesomethingabouttheavatar?Why?

5. What’s your opinion on the gender of the avatar?

6. Did the way of speaking of the avatar look convincing to you?

   (Meaning you are comfortable speaking to avatar)

7. How would you feel about the possibility of customization of the

   avatar before the interview?

8. How would you like to customize it?

9. How likely would you want to participate in a full real-time interview with the avatar?
   1.Very likely 2.Likely 3.Unlikely 4.Very Unlikely

10. How would you describe the avatar's appearance to a friend?

11. Is there anything we haven’t asked you that you want to share with

    us?

Testing material

The North Kingdom was tasked with designing an avatar that was intended to be relatively gender-neutral and easily customizable by the client if needed in the future. Due to technical difficulties and the time required for implementation, it was not feasible to test the avatar in real-time. Instead, we decided to create a video demo that simulated the conversation participants would have with the avatar even though participants would have been informed it’s only a video.

Demo video objectives:

Including a script for the avatar's self-introduction.

Engaging participants in small talk to create a natural conversational atmosphere.

Allowing ample pauses between asked questions to give participants sufficient time to respond to the avatar's inquiries.

Ensuring a neutral background and appearance for the avatar to avoid any potential bias or distractions.

Key Findings

Recommended actions