Helping clinical psychologists collect valid audio diaries by creating a private, approachable, and engaging experience for research participants.
Dayrio
Helping clinical psychologists collect valid audio diaries by creating a private, approachable, and engaging study experience for research participants.
Mobile | UX/UI | 0-1
OVERVIEW
I led the app design in a four-member team in this course project. We closely collaborated with the Industry sponsor, AppHatchery, and research clients, lead psychologists at Emory University. Our project is now under development and will be put into use this Spring with a pilot study on real population 🎉
DURATION
08.2022 - 12.2022
(4 months)
ROLE
UX Design
User Research
Competitive Analysis
TEAM
Lina (Research)
Ben Mayo (PM, Design)
Hudson (Research)
TOOLS
Figma
Procreate
Miro
BACKGROUND
Audio Diary Method is a popular method in clinical psychological studies
Audio Diaries is clinical psychologists' preferred method for collecting self-reports of participants about their daily lives, where participants need to verbally respond to prompts pre-set by psychologists.
Compared with written diaries, audio diaries make it easier for participants to give a stream of consciousness. Besides, voice contains rich information for analysis, e.g., emotions. Also, compared with conducting interviews, coupling audio diaries with pre-set prompts can help reduce the time cost and can thus increase the sample size.
PROBLEM
No reliable method to help researchers collect valid audio diaries
After meeting with our stakeholders, Apphatchery Team and lead psychologists, we learned that researchers relied on recording devices, answering machines, or manual uploading from the participants' end to collect audio diaries.
%20(1).png)
Researchers
Participants
SOLUTION
A dedicated mobile app that assists the interaction experience between two ends
We designed a dedicated mobile app that enable participants to record audio diaries that are privately synced with researchers. To ensure the data validity during such longitudinal audio diary study, our app uses Virtual Assistant and Momentary Sidenotes features to increase the approachability during the audio diary study. Also, it uses progress tracking and incentive strategies to increase participants' engagement throughout the study period.
CONCEPT 1
Walkthrough with the Virtual Assistant
Familiarize participants with the study procedure, setup, and the app UI with an inviting virtual assistant to reduce researchers' manual labor and ensure study validity from the early stage
CONCEPT 2
Dialogue-Like Diary Recording
Mimic a private conversational experience with the virtual assistant to make audio diary recording more approachable, natural, and less anxious for new users
CONCEPT 3
Take "Memos" to Help Recall
Enable participants to take memos throughout the day that can be reviewed to ease the remembering difficulties in the end-of-day diary recording
CONCEPT 4
Incentive & Progress Tracking
Set short-term incentives and visualize the current progress of the study keep participants motivated and engaged throughout the study period
Final Outcome
Our project was spoken highly by our clients and sponsors. And our design will be put into development and launched by our sponsors, the AppHatchery Team, in early 2023, with pilot study on helping burn out for frontline healthcare workers for clinical psychologists at Emory University. Here are some feedback we got from the researchers at Emory University and the AppHatchery Team:
Our Approach
01
RESEARCH
Literature Review
Competitive Analysis
Stakeholder Meetings
User Interviews
Expert Interviews
02
ANALYSIS
Affinity Mapping
Personas
User Journey Maps
Success Criteria
03
DESIGN
Design Ideas
Concept Sketches
Wireframes
Feedback Sessions
Interactive Prototype
04
EVALUATION
Think-aloud User Testing
Follow-up Questionnaire
Quantitative Analysis
Expert Evaluation
Feedback Grouping
Design Iterations
Final Design
RESEARCH
Existing Methods and Limitations
Through preliminary research with clients, we learned that there are mainly 3 methods for clinical psychologists to use for the audio diary study.
Voice Recording Device
Answering Machine
General Recording Mobile App
However, none of them are reliable due to 3 Issues
1
Security & Privacy Concerns
General audio journaling apps pose security and privacy concerns. Besides, researchers are NOT allowed to extract voice data from participants’ personal phones.
2
Too Many Steps
Currently, participants need to go to the lab to obtain/return the recording device or manually upload their recordings. The logistics involved add much workload for both sides.
3
Poor Data Validity
Issues like forgetting to record, lack of engagement in the study, low recording quality, failure to answer questions accurately, resulting in invalid data for research use.
INITIAL DESIGN GOAL
A dedicated recording app that supports participant-researcher interaction
While personal mobile apps can effectively minimize the manual labor caused by extra recording devices or manual data transferring, a mobile app specifically for audio diary study use is desired by both researchers and participants to reduce privacy&security concerns, and to provide opportunities for more customized experience in an audio diary study.
Interviews
We first met sponsors and collaborators to understand the problem space and their initial expectations. Then we interviewed clinical psychologists at Emory University and prior research participants to get first-hand information about their pain points, needs, and expectations for the audio diary recording device / application.
2
Stakeholder Interviews with Emory School of Medicine and AppHatchery Team
4
Expert Interviews with
Emory researchers who have audio diary study experience
6
User Interviews with participants who participated in audio diary studies before
Affinity Mapping
To better inform design decisions and derive design implications, we synthesized user quotes and organized them based on their affinities to find the common themes and key findings from the interviews.
.jpg)
TOP KEY FINDINGS:
%20(1).jpg)
Personas
To better guide our design and empathize with our users, we further synthesized the interview results and came up with typical personas for the participants. Despite the diversity of potential participants, we chose these two typical personas since they can cover participants with different motivations for participation and levels of tech literacy that our design should cater to.
Also, the two personas have typical, distinctive pain points in different stages in the study. For example, the most challenging thing for Caroline is to familiarize herself with audio diary recording in the early stage in the study, while for Jake it's to keep engaged and motivated in the study in the later stage in his busy schedule.
KEY TAKEAWAY
As shown in the key findings and user journey maps, participants' experience throughout the entire study period plays a key role for the study validity and success.
DESIGN GOAL
How can we create a private, approachable and engaging audio diary study experience for participants and ensure study validity?
IDEATION
Success Criteria
With the end design goal and previous research findings in mind, we then summarized the success criteria we would like to focus on in the next design stage. It considers the needs and expectations of both participants and researchers.
1.
Learnability
Enable participants to easily learn the study procedure and app UI to reduce researchers' workload as well as ensure data validity from an early stage
2.
Privacy
Properly address participants' privacy and security concerns both technically and perceivably to reduce their anxiety around the recording process
3.
Approachability
Create an approachable and comfortable audio recording experience. Reduce first-time participants' negative feelings, such as nervousness, weirdness, etc
4.
Ease of Recall
Help participants recall their daily activities during end-of-day recording to help collect more complete and accurate self-reports without causing biases
5.
Engagement
Keep participants engaged throughout the study period and motivated to record valid audio diaries in time every day
6.
Accessibility
Within the scope of voice recording, make the app accessible in different contexts for potential participants with varied abilities
Brainstorming
To generate as many interesting ideas as possible in the ideation stage, all of us brainstormed solutions together by sketching concepts on paper independently. Then we came together to discuss our ideas and clustered them in Miro.
Concept Sketches
Two core concepts that best satisfy our design goals emerged in our brainstorming and discussions: 1. Virtual Assistant and 2. Audio Album. To quickly evaluate the concepts, I created concept sketches. Then we invited stakeholders to give us feedback.
.png)
Virtual Assistant ❤️❤️❤️
According to our research, participants feel more natural talking into a real person. Also, they are more responsible when directly engaging with a real person. By using the virtual agent to guide the user and mimic an in-person conversation during diary recording, we try to improve the participants' approachability and engagement.
Record Case ❤️
In using the record and tape metaphors for audio recording, we tie digital recording experience to users' knowledge of how physical audio works to improve the approachability of the study. Also, it makes the whole experience more engaging than plain and sterile digital recording and allows users a fun way to metaphorically build up their record collection.
Concept Sketch Feedback
After discussing with our stakeholders about how the two concepts satisfy our design goals and meet their expectations, we decided to go ahead with the Virtual Assistant idea, since according to expert psychologists, the virtual assistant provides a very novel and effective way to make audio recording more natural and less awkward. Also, it can communicate with participants about the study requirements, recording quality, etc, in a more friendly way.
Still, expert psychologists pointed out that they like the self-reflection aspect in the record case concept to improve participants' engagement in the longitudinal study.
Core Functions
After deciding the core concept of the virtual assistant, we co-designed with participants and expert psychologists to propose other features that can further makes the audio diary study experience more private, approachable, and engaging for research participants. We then discussed about the feasibility, effectiveness, and novelty, we finally settled on the following core functionalities to satisfy our design goals after brainstorming and co-design sessions:
FEATURES FOR APPROACHABILITY
.png)
FEATURES FOR ENGAGEMENT
.png)
Wireframes
We then created the wireframes to show and test the core features and the entire user flow.
.png)
Wireframe Feedback and Key Iterations
To evaluate the features we proposed, we conducted think aloud interviews with 3 researchers to learn the usefulness and possible influence of our concepts. We then collected their feedback and annotated them as "positive", "negative", and "could be improved".
Concept 1: Study Walkthrough by the Virtual Agent
Walkthrough the study procedure and setup before the study starts
-
Can help reduce the researchers' workload and improve the learnability.
-
Love the virtual agent which makes participants feels more connected with the researcher.
-
Still feel uncertain about the real recording process; Want a training
.png)
.png)
Concept 2: "Talking" into the Virtual Agent
Turn the audio diary recording experience into a conversational experience.
-
Makes the voice recording more approachable, less awkward and less nervous.
-
Virtual agent is very neutral and will not influence the recording performance.
-
Status in recording responses can be more clear.
Concept 3: Sidenotes Taking
Take momentary sidenotes throughout the day to ease remembering difficulties
-
Can successfully ease remembering difficulties and can also help researchers collect momentary self-reports.
-
The purpose and usage of sidenotes are not clear for new users.
-
Prefers written-text as an alternative input method when audio recording is not convenient.
.png)
.png){kind=tracking}
Concept 4: Progress Tracking & Incentives
Track the current progress and incentives to keep participants motivated and engaged
-
Feel more motivated when seeing the positive messages and incentives for the next level.
-
Love the calendar view that helps track the current progress.
-
Incentives are not easy to track. Can be more attractive and motivating.
01
"Many a little makes a mickle"
Encourage participants to focus on recent, short-term performance to improve the likelihood of completing the entire study
.png){kind=tracking}
-
Encourage users to focus on short-term performance instead of the performance over the entire study span
-
Increase users’ motivation by showing the next small challenge and incentive
02
Motivation should be more directly triggered, not found by users
Improve users’ awareness of what has been achieved, what can be achieved, and how to arrive there
.png)
-
Enable the users to directly check incentives on the homepage to increase its exposure and thus better improve users' motivation
-
Expose the users to the next challenge and the incentive in the calendar view clearly to keep them engaged in the study
03
"Sidenote" is useful for recall only when users know its purposes and love to use it
Clearly convey the usage of “Sidenote” and improve the UX accessibility of taking notes
.png)
-
Provide user guide to help first-time users grasp the purpose and usage of Sidenotes
-
Enable written-text as an alternative input method to make sidenote taking more accessible
PROTOTYPE
Visual Style Guide
For the visual style, I landed on a few keywords of the kind of user experience that we want to bring to our target users: Inviting, Private, and Rational. Thus, I chose purple, orange, and white as the primary colors. Also, I used flat UI design to create a clean and peaceful diary recording environment.
For the virtual agent, first, to ensure the validity of the study, it should be neutral so that it won't bias the participants' responses. Second, to make participants more engaged in the study, I made it more friendly. Third, considering the diversity of potential users, I try to make it agender. Also, the facial expressions and body movements of the virtual agent are made more universal so they are easier to understand by different users.
.jpg)
High-Fidelity Design - Iteration 1
STUDY WALKTHROUGH
Walkthrough the study procedure, setup, the app UI during onboarding and provide the training session to ensure study validity from the early stage.
Study Intro & Setup
UI User Guide
AUDIO DIARY RECORDING WITH THE VIRTUAL AGENT
Get inviting reminders for diary recording. Mimic a private conversational experience to make audio diary recording more natural and less nervous.
Notification
Dialogue-Like Audio Recording
MOMENTARY SIDENOTES
Get inviting reminders for diary recording. Mimic a private conversational experience to make audio diary recording more natural and less nervous.
Sidenote Taking
Sidenote List
SHORT-TERM INCENTIVES
Set challenge bonuses and provide positive messages and monetary incentives to improve participants’ motivation.
.jpg)
PROGRESS TRACKING
Present the current progress of the study to encourage self-reflection so as to improve participants’ engagement.
%20(1).jpg)
EVALUATION
Study1: User Testing
To evaluate if our design meets the overall design goal and the success criteria we set previously, we first moderated user feedback with 5 users with different experiences in audio diary studies and personal diaries. We conducted think aloud user testing over Zoom where we set tasks for each concept. Here is a brief overview of the process:
-
Introduce the project goal, and put users at ease.
-
Describe the scenario, give the tasks, and let users use the app while thinking aloud.
-
Ask follow up questions and gather feedback.
-
Let users fill out a questionnaire.
-
Thank them for participation.
Measuring Success
To quantitatively evaluate the success of the design, we broke down our design goal into detailed, measurable ones and developed a questionnaire to gather user feedback.
%20(1).jpg)
Study 2: Expert Evaluation
Second, we also conducted semi-structured interviews with 4 clinical psychologists who use daily diaries in their research to gain expert feedback. Here is a brief overview of the process:
-
Introduce the project goal
-
Briefly introduce the key features of the app
-
Describe the scenario and ask the researchers to explore the app while thinking aloud
-
Ask follow-up questions
-
Thank them for participation
Quantitative Results
Here's a summary of the ratings in the questionnaire. On average, user participants rated Moderately Positive (6) for all the questions.
According to the questionnaire, Users like the Virtual Agent, Incentive & Progress Tracking, and the Study Walkthrough the best. User participants' attitudes towards the Sidenote feature varied the most, due to two main reasons: First, users have different levels of remembering difficulties and different habits of sidenotes; Second, some users still felt confused about the usage and purpose of sidenotes. Despite the overall positive feedback, its user guide, UI, etc, should be improved so that potential users can better utilize them to improve their performance in the audio diaries.
%20(1).jpg)
Feedback & Key Iterations
We grouped feedback from the user interviews and expert interviews based on themes and features. I colored different themes based on whether they are positive or negative feedback, or "could be improved".
Study Walkthrough
1. App Walkthrough Can Be More Clear
Both users and researchers agreed that the user guide with the virtual agent is very helpful, friendly, and clear. However, 3/5 users thought the app walkthrough is a bit wordy and easy to get lost. It's important to let users remember the key features of the app, and let them know where they are in the entire user guide process.
.png)
2. Communication with the Researcher Should be Involved in the Study Guide
3/5 users and all researchers pointed out the necessity of communication between the participant and the researcher when participants are in trouble despite the availability of the help guide.
.png)
Audio Diary Recording
1. Virtual Agent Is Very Satisfying
Every interviewees loved the virtual agent and agreed that it was cute, inviting, and made the study more engaging. 3/4 researchers said the virtual agent design was neutral, while 2/4 of them suggested that it would be better if the avatar could be customized according to the nature of the study.
2. Showing Sidenotes Before Diary Recording Is Redundant and Misleading
Even though users and researchers agreed on the usefulness of sidenotes in easing remembering difficulties, 3/5 users doubted about showing sidenotes directly before diary recording. For example, P1 said showing sidenotes before diary recording made him feel repetitive to record. Also, R1 pointed out that showing sidenotes directly before the diary recording might mislead participants that they must remember every detail clearly before recording, which might increase their anxiety.
3. Other Tips Are Wanted to Ensure Data Validity
Also, researchers suggested other tips to help ensure the data validity, such as a microphone test.
.png)
Momentary Sidenote
1. The Purpose and Usage Should be More Distinguishable from Diaries
2/5 users misunderstood the purpose of the sidenotes at first. P5 thought sidenotes are complementary to diaries and pointed out that the sidenote card looks very similar as the diary entry card, making it more confusing.
2. Reviewing the Sidenotes Should be More Efficient
Users mentioned that the time cost for reviewing sidenotes can be lowered so they would use it more often. For the sidenote list, 3/5 participants mentioned that the display of title, duration, and time is not very inefficient for catching the content of the sidenote.
.png)
Incentive & Progress Tracking
More Diverse Positive Messages Help Improve Engagement:
Participants like incentives, and they also like to feel useful
All participants agreed that seeing the monetary incentives and the challenge for the bonus would greatly improve their motivation and engagement in a longitudinal study. Also, 3/5 participants said they loved to see positive messages related to their contribution to the research or benefits in their personal development.
.png)
FINAL DESIGN
Here are the prototypes of my final design based on the changes I have made. You can navigate different user flows through the side menu.
REFLECTION
Real Influence of the Virtual Assistant on Participants' Performance
From our user testing results, the virtual agent can effectively improve the approachability of audio diary recording and improve their engagement in the study. Still, due to the time limit of our project, we didn't have time to evaluate the long-term influence of the virtual assistant in a real clinical psychology study setting. While experts did mention that our virtual assistant design made the study more guided, inviting, while stayed neutral, questions like whether and how the existence of the virtual assistant can really positively affect participants' performance in audio diary studies without affecting the validity of the study need more investigation in a real experiment over a longer time span.
Besides, researchers we interviewed mentioned that different avatars may be suitable for different studies, and that some variations in the avatar itself may better engage the participants. Considering the scope of this project, this field was not explored.
There is a large space for interesting research on the influence of the existence and appearances of the virtual assistant on participants' performance in such self-reports.
Constantly engaging different stakeholders into the design process
I think the biggest challenge in this project was to consider both the needs of the researchers and the participants. Here, even though our customers are the researchers, the real users, however, are the participants. Participants' experience in this study plays a key role and should also be carefully considered - If they feel confused, impatient, or unwilling to engage in the study, they may behave badly, which will in turn impair the validity and accuracy of the study. By taking different stakeholders into the design process, we had a more well-rounded idea of our design requirements. Frequently engaging them to give us feedback also helped us quickly evaluate our ideas and make meaningful iterations.
Sometimes the goals & needs of different stakeholders can have conflicts - For example, while participants want to record the diaries as efficient and easy as possible, researchers want them to encourage them to share more; While participants want to get the incentives earlier, the researchers want to first make sure that they complete the self-reports; etc. We need to carefully balancing needs from different sides in a way that can better satisfy our end goals
Next Step
We will evaluate the avatar design in a real experiment over a longer time span and may use a between-subject experiment that compares between the group with the virtual assistant in audio recording and another group without it.
Secondly, we will design the back stage system for researchers to customize their study requirements. We first focused on the design of the user-facing app because in this way, we have a better idea about what features or information is needed, and what should be customized by the researchers.
Lastly, we will further improve the accessibility of our app considering the large audience it may face. For example, when starting to record, we can use the sound effect to help participants with visual impairments ensure that it's started.