MirrorVerse
A unique virtual platform for self-interaction and reflection.
Overview
MirrorVerse combines ChatGPT's conversational abilities with MetaHuman Creator's realistic avatars to create a unique virtual platform for self-interaction and reflection. This platform allows for genuine dialogue and emotional feedback with a lifelike virtual version of yourself. Engage in authentic conversations where the virtual "you" offers comfort, advice, and diverse responses, enhancing self-engagement beyond real-life limitations.This is my graduation design project, and I wanted to challenge myself with something entirely new. Inspired by the rise of ChatGPT, I wondered if I could use it for self-healing purposes.
Team
JIAHUI LIU
Role
UX/UI Designer
user research
prototyping
wireframing
interface design
testing
Tools and Method
Figma, UE5, Photoshop, ChatGPT, Metahuman, FaceGenModeller, Azure Speech
I used Lean UX methods to quickly create prototypes, and repeatedly tested prototypes to improve my design
Challenge
The challenges I encountered were primarily about improving ChatGPT's ability to simulate the user's tone more accurately, making the interaction feel more authentic. Additionally, I had to consider whether the uncanny valley effect would cause users to feel uneasy about their 3D digital images.
Background Research
-
Have you ever talked to yourself in a mirror?
Positive self-talk enhances attention, mental state, and self-perception. While traditional mental health support is well-studied, new technologies offer fresh opportunities. Tools like the Portable Interactive and Meditative Mirror (PIMM) bring these practices into the digital realm, collecting emotional data to manage emotions more effectively.
-
The Proteus Effect
Research shows that digital avatars can influence offline behavior through the Proteus effect, where changes in one's online persona impact self-disclosure and negotiating skills.
Technical Study
-
ChatGPT EA ability & Role-playing ability
Conversational agents like ChatGPT hold significant potential for mental health applications. Research is assessing its emotional awareness (EA) capabilities, making it a potential tool for cognitive training and psychiatric diagnostics. Using natural language processing and large text data, ChatGPT generates human-like responses and simulates role-based interactions, creating engaging and immersive experiences.
GOAL
Conversational agents like ChatGPT hold significant potential for mental health applications. Research is assessing its emotional awareness (EA) capabilities, making it a potential tool for cognitive training and psychiatric diagnostics. Using natural language processing and large text data, ChatGPT generates human-like responses and simulates role-based interactions, creating engaging and immersive experiences.
Why Lean UX Method?
Lean UX is a collaborative, iterative way of designing and building products. It helps me refine my projects in a constant cycle of thinking, making, and testing
01 THINK
What are the problems?
I wrote down some Assumptions from both a business perspective and a user’s perspective to help me clearly define the problem I was trying to solve. I list part of the assumptions in the table below
How to resolve problems?
According to those assumptions I did some Brain Writing to help me to think of some solutions. I did two rounds. The first round is a direct idea in my head, and the second round is an improvement based on the ideas of the first round
I then compiled all these ideas into the following Assumptions Mapping table to help me better understand which problems are most important and need to be solved first
Persona
User Journey Map
02 MAKE
After I identify assumptions that need to be tested and validated. I decided to build an MVP prototype that would allow testers to interact. This prototype can interact with their avatars in a text-based or voice-based conversational interface.
Interactive prototype
Step 1: Generate your own digital image
I started by generating my own 3D model from the photos in FaceGenModeller, then imported my assets into Unreal Engine to set up the skeleton and rigging for the model. And customised the skin textures as well as tweaked the look.
Step 2: Accessing ChatGPT and AzSpeech in UE5
Accessing ChatGPT and Azure Speech in UE5 allows me to have real-time interaction with my models; then I let the voice drive my model to speak and redirected my character animation
Wireframes
03 TEST
For the initial test, I recruited two participants and asked them to provide photos that were used to create their digital avatars. In addition, I asked each participant to write a paragraph about themselves. Using this information, I trained ChatGPT to mimic their speaking style and thought patterns. After the test, I conducted interviews to gather their insights.
Due to time limitation, I mainly focus on the test of prototype instead of the test of the entire project process. The test includes whether the digital image meets the user's expectation or whether it arouses the uncanny feeling; And whether ChatGPT can perfectly reproduce a user's tone of voice or thoughts.
Insights -Test Round 1
1. Participants felt the digital image resembled them but lacked detail. Allowing adjustments to their avatar's appearance could increase satisfaction.
2. They found the avatar's movements stiff and its speech expressions unnatural. Improving contextual expressions, like frowning instead of smiling to show sympathy, would enhance the experience.
3. Sound quality needs improvement, such as training the avatar to replicate the user's timbre and accent accurately.
4. Participants noted vague answers due to insufficient data and misinterpretations. Providing templates and guidelines could help users give more detailed information, improving ChatGPT's replication of their thoughts and tone.
5. Participants sometimes struggled to maintain the conversation. Offering chat prompts and having ChatGPT ask questions could make interactions more natural.
Due to time constraints, I focused on training ChatGPT to better represent participants' thoughts rather than perfecting the avatar's look and sound. I provided participants with a template to input richer information and modified prompts to have ChatGPT ask questions instead of just answering or repeating information.
Insights -Test Round 2
1. Alter the instructions being given to ChatGPT so that it does not recycle or repeat the information provided in such a unrealistic way.
2. Provide more specific instructions to ChatGPT to not answer questions in such a robotic way. It should ideally not offer answers in a list. A friend in conversation would not respond to questions in such a logical way. Instead, it can be given directions to answer more freely and conversationally.
3. Provide further instruction to ChatGPT intially to have it overall respond in a more emotional manner. For example, if the user is acting aggressive towards the digital avatar, it can react in turn aggressively, or become offended. Similarly, if the user is sad, the digital avatar can react sympathetically or with a counteracting emotion that could try to improve the mood of the user.
04 Final Design
Final lean UX canvas
UI Design
Create the avatar
Chatting
Community
Future Work
This project is different from the ones I've done before, I'm focusing on testing for MVP. Using Lean UX methods, I achieved an 80% confirmation rate in hypothesis tests, despite having limited resources and a sample size of only three people. The provided prompt templates help train ChatGPT to better mimic a user's personality, but also face challenges such as emotional nuance and handling abstract queries. Future work will address these limitations, aiming to achieve a more natural range of interactions and emotions.