Grove - A gamified mobile App that helps neuro-impaired patients self-monitor upper-limb dysfunction in an accessible, engaging, and fun way
My Roles -
For -
With -
3 Engineers,
1 HCI Professor,
Healthcare Experts
Duration -
Aug. 2023 - Now
Under Implementation
Overview.
iOS App currently under implementation
Patients with upper-limb dysfunction induced by neurological disorders now rely on clinic visits to assess their upper-limb functions, which is inconvenient and hinders a transparent view of their recovery progress. Thus, our team in Georgia Tech Contextual Computing Group designed Grove, a mobile app that enable patients to self-assess upper-limb function conveniently, gain insights into their recovery progress, and stay motivated in long-term self-monitoring.
The iOS App is currently in the exciting implementation phase! 🎉
My role.
I led the user-centered, design & tech bidirectional exploration
As project leader, I led a cross-functional team to research user requirements and explore a suitable technical solution, closely collaborating with healthcare experts and patients. As the sole designer, I conceptualized and designed the mobile app in an iterative process, integrating stakeholder feedback from co-design and testing sessions.
Key abilities:
Design Leadership
Multi-Stakeholders Management
Design for Behavior Change
Game Design
Background.
Patients with neurological upper-limb dysfunction can now self-rehab at home with the VibroTactile Stimulation Gloves
Patients with neurological disorders like Multiple Sclerosis (MS) and Stroke, often suffer from upper-limb dysfunction. To help these patients recover their upper-limb function in a more low-cost and accessible way, the VibroTactile Stimulation (VTS) gloves were developed by our Georgia Tech's Contextual Computing Lab, to enable patients self-rehabilitate their upper-limb function at home, reducing the overhead of clinic visits.
Problem.
While continuous upper-limb function monitoring is valuable, inaccessible assessments and lack of engagement make it challenging for patients
Healthcare experts emphasize the importance of continuous monitoring of upper-limb recovery progress as it enhances their visibility into patients' conditions and improves patients' self-awareness of their recovery journey.
Yet, patients must still visit clinics in person to assess their upper-limb recovery progress. These clinic-centric assessments are inaccessible, especially for underserved communities, and impede timely and transparent tracking of recovery. Moreover, given that the recovery journey spans months or even years, patients often lack engagement in long-term symptom self-monitoring.
Product Goal.
How might we make upper-limb self-monitoring more convenient and accessible for patients while promoting sustained engagement throughout their self-recovery progress?
Final Design.
Grove: Monitor your self-recovery progress with accessible and engaging motion games on your phone
By employing phone-based gamified tests with motion capture, Grove simplifies at-home self-testing, making it accessible and engaging. The gamification component, alongside clear task tracking and insightful self-reflection, fosters lasting motivation and commitment to self-monitoring.
Personalize Self-Monitoring Plan
Generate personalized self-monitoring tasks based on the patient's profile, while enabling them to customize according to their own preferences to help them establish scientific and also feasible self-monitoring routine.
Profile Setting
Generated Tasks Walkthrough
Home: Weekly Tasks
Virtual Farmland Building
Track Self-monitoring Tasks
Clearly visualize the weekly tasks on the homepage to help patients track their current progress and display the virtual farmland to visualize the virtual agriculture rewarded for timely task completion, encouraging patients to stick to the self-monitoring schedule.
Play Motion Games for Upper-Limb Assessments
The patient can self-assess their upper-limb function conveniently by playing motion games adapted from clinic assessments. The game results help patients easily understand their recovery progress. Moreover, the farm-themed games provide a sense of achievement, making patients more motivated.
Motion Games Overview
Arm Selection
All Upper-limb Self-Assessment Games
🙋 Shoulder Extension & Flexion Test
🫱 Shoulder Rotation Test
💪 Elbow Extension & Flexion Test
🫳 Wrist Up & Down Test
👃 🫵 Finger-to-Nose Test
🖐 👊 Finger Extension & Flexion Test
👌 Finger Tapping1 - Coordination Test
👌 Finger Tapping2 - Speed Test
Self-Reports Overview
Self-Report Example
Self-Report Contextual Info
Provide well-designed, personalized self-reports for the patient alongside assessment games, so that the patient, healthcare providers, and clinicians can gain a more comprehensive understanding of the patient's condition.
Reflect on Recovery Progress
Visualize longitudinal self-assessment results and self-reported responses to help patients or doctors quickly gain a clear overview of the patients' recovery journey.
All Self-Test Results
All Self-reported Responses
Stakeholder
Interviews.
Investigate the potential & requirements for transforming clinic-based assessments into more convenient self-assessments
Our secondary research suggested the potential for at home, self-assessments for upper-limb functionality. To further investigate such potential, I designed and co-led multi-stakeholder interviews to gain comprehensive perspectives about:
Requirements, existing practices, and challenges in patients’ self-monitoring of upper-limb dysfunction induced by neurological disorders like MS and Stroke
Ideal technical methods that support feasible and accessible at home, self-assessment.
Potential strategies to encourage patients to stay engaged in long-term self-monitoring.
Key Finding 1.
3 Key challenges in sustained self-monitoring of upper-limb dysfunction
Stakeholder interviews and secondary research suggested 3 major barriers to at home, self-monitoring of upper-limb dysfunction:
Inaccessible clinic-centric assessments
Current assessments for upper-extremity function are clinic-centric, relying on clinicians' experience-based diagnoses. However, patients with limited access to clinical resources face challenges in timely tracking their recovery progress.
No effective tools to precisely self-track symptoms
Due to the subtle and gradual nature of symptom changes, patients find it challenging to accurately track changes in their upper limb functionality at home without effective tools.
Lack motivation & adherence in self-monitoring
Maintaining long-term self-monitoring is crucial. Yet, patients frequently struggle due to a lack of awareness regarding its value or simply forgetting to engage in the practice.
Key Finding 2.
Promising to adapt experience-based clinical assessments into more accessible, objective self-assessments using the mobile device
We learned that current clinical assessments for upper-limb function primarily focus on measuring the range of motion for various upper-limb joints (Fig. 1-5), as well as motor coordination, accuracy, and speed through activities like finger tapping (Fig. 6) and the finger-to-nose test (Fig. 7).
Clinic Assessments for Upper-Limb Dysfunction Induced by Neurological Disorders
While all these assessments necessitate the involvement of a healthcare expert for diagnosis, experts strongly support the concept of converting clinical assessments into phone-based self-assessments using phone cameras and sensors for upper-limb movement detection, as they would be more accessible and quantifiable.
Key Finding 3.
Combining objective self-assessments with self-reporting subjective symptoms provides a more comprehensive view
In addition to assessing upper-limb functionality objectively, domain experts emphasized patients' self-reporting of subjective symptoms like behavior, mental health, and quality of life, for both patients and doctors to gain a comprehensive view.
Technique Exploration.
What techniques would be suitable for patients' upper-limb function self-assessment?
Developers in our research team have developed various techniques for assessing different facets of upper-limb functionality. I designed and led the expert interviews to evaluate these techniques based on their usefulness, feasibility, and accessibility.
Technical solution.
💡 Utilizing Camera-based motion capture technique would be effective, accessible, and feasible!
Expert interviews suggested that the hands-free nature of camera-based computer vision techniques makes them more accessible for patients with upper-limb impairments to complete self-assessments. The sensor-based solutions, however, are either not feasible or not useful for our target users.
To prove the validity of CV-based method for upper-limb self-assessment, we tested with other body movements involved in clinic tests using MediaPipe, which was proved to be feasible and accurate.
Design Implications.
💡 A mobile app using motion capture technique to help patients conveniently self-assess upper-limb function at home while fostering motivation and sustained commitment
After synthesizing our research findings, I proposed the overall product concept and the following design requirements:
1.
Self-Awareness
Help patients gain a clear view of their subtle and gradual changes in symptoms and improve their self-awareness in their journey of recovery.
2.
Accessibility
Enable patients to assess their upper-limb functionality in an accessible and easy way at home, instead of visiting the clinics.
3.
Motivation & Commitment
Motivate patients to stay motivated and committed to symptom self-monitoring, particularly regarding their upper-limb functionality.
brainstorming.
Brainstorming App features to satisfy design requirements in accessibility, visibility, motivation & commitment
To come up with solutions that well satisfy design requirements while taking medical/technical/user experience requirements into consideration, I organized the brainstorming session that involved our multi-functional research team (including healthcare experts, AI experts, designers). We brainstormed product features, categorized and evaluated based on their usefulness, impact, urgency, and technical feasibility.
core features.
💡 We've identified 6 core features to satisfy design requirements, including the innovative gamified assessment feature
Through more literature reiew and discussions, we finally landed the following pivotal features that better align with our design requirements. The gamified assessments idea were highlighted for its potential to not only improve test accessibility, but also quantify subjective evaluation through game scores, and boost patients' motivation and engagement in self-assessment.
Concept Testing.
Illustrating core features with wireframes to solicit stakeholder feedback
I illustrated the core concepts we have, and created the games based on the theme “treasure hunting”. Then, we showed healthcare experts and HCI experts working in relavant fields to seek their feedback (shown in the following graph).
Testing Results.
🥳 Our core features were all highly approved!
Overall speaking, experts agree that all the core concepts we have well satisfy user requirements. Particularly, they highlight that the gamification components and the consistent reward system are great ideas for motivating patients to record their symptoms.
Next step.
💡 Extending the gamified assessment concept to cover comprehensive assessments shows promise!
Domain experts think it's highly promising for us to adapt clinical assessments involving other joint movements, e.g., shoulder, elbow, wrist into such motion capture games, so that patients can rely on the mobile device for comprehensive self-assessments, which made me super excited to further explore the game design in the next design phase.
Design Challenge1.
Q1: What would make an effective, engaging, and cohesive game setting?
Literature review on "gamified design" and "narrative as a motivator" suggested the importance of a cohesive game theme and narrative for effectively promoting users' motivation & engagement, as users may find it meaningless and confusing to play random games. Thus, I proposed key design considerations to ensure that the game narrative/theme is coherent and it meets our design goals:
Game Setting Ideation.
Co-designing the game theme based on design requirements
I led brainstorming sessions to engage our multifunctional team in generating diverse design concepts aligned with our considerations. These sessions employed whiteboards, quick sketches, and iterative discussions to explore, synthesize, and refine ideas.
Game Setting Redesign.
💡 Farming is chosen as the ideal theme due to its diverse array of physical activities and tangible outcomes for real-time feedback
Following evaluation, my concept of farm-themed gamified tests received team approval since farm activities:
Have enough varieties to cover all necessary assessments comprehensively.
Can effectively mirror upper-limb movements
Have tangible and measurable outcome to foster user motivation and engagement.
Additionally, I changed the game layout from horizontal orientation to landscape view, which provides a wider space for motion detection and a more immersive game experience. I sketched out the game concepts based on this theme.
Design Challenge2.
Q2: How to ensure the game meets both medical requirements and user experience needs?
Another challenge I faced was to ensure that the game meet medical needs and satisfy user experience. I asked myself the following questions while establishing the game settings:
Do they intuitively mirror the body movements in upper-limb assessments?
Can they accurately capture measurements needed to reflect patients' symptoms?
Can patients easily understand them and feel engaged in the game?
💡 Build a unique design approach that integrates logical reasoning and emotional expressions
Thus, I created a unique design procedure that integrated multifaceted design considerations. Besides, I facilitated quick feedback sessions across the multifunctional team to arrive at the ultimate game concept for each assessment.
👌 Let's dive into the finger tapping coordination test as an example!
Mid-fi Prototypes.
Refine the design concepts into mid-fi prototypes
Task Personalization & Tracking
Tests for evaluating the range of motion
Tests for evaluating brain and upper limb coordination
Tests for evaluating finger coordination, dexterity, and tapping speed
Longitudinal recovery progress visualization to promote self-awareness
Mid-Fi User Testing.
Qualitative & Quantitative evaluation with multiple stakeholders
I collaborated with another user researcher to design the user testing session. We showed stakeholders the prototypes to seek their feedback. Specifically, our mid-fi user testing has the following goals that match design goals regarding visibility, accessibility, motivation&commitment.
Testing Results.
🥳 Our design received high ratings across all success metrics
I continued to iterate the mid-fi design to high-fi after synthesizing the feedback.
Now, let's jump into the major design iterations I made!
Accessibility.
💡 Enhance the accessibility of self-test games with a more user-friendly guide and improved visual accessibility
Self-Awareness.
💡 Promote patients' self-awareness by refining more intuitive and insightful data visualizations
Motivation & Commitment.
💡 Boost patients' motivation by creating a more transparent and appealing reward system
Hi-Fi Prototypes.
Heyyy! Let's click through the interactive hi-fi prototypes!
Want to revisit my final design? Jump to Final Solution
Current stage.
🎉 iOS App now under implementation!
As the project initiator, I presented the final solution to the research lab, receiving enthusiastic acclaim from the researchers. It's incredibly thrilling that it's slated for development in 2024!
Stakeholder Feedback.
🫶 Warm feedback from stakeholders
Our design was spoken highly by HCI researchers, healthcare experts, and patients that we closely collaborated with throughout this project.
What I learned.
🎮 ➕ 👩⚕️ Value of using gamification in healthcare
Healthcare is a serious field demanding in-depth medical expertise and precision in design details. I was excited and also surprised that my gamification concept, which might sound "unserious", received a warm reception from healthcare experts and patients. Besides being useful in the product itself, making users feel valued and engaged to use the product is equally important for product success.
Another exciting aspect that I discovered about gamification was its potential to convert subjective assessments into measurable tasks with structured game mechanics. It reminded me of advice from my mentor at Alibaba:
“Be rigorous and logical in your design process, while being soft in conveying human touch when delivering your design to your end users.”
💬 ➕ 📖 Combine bottom-up and top-down design methods
Continuously reading literature on narrative motivation, health informatics, gamification in self-monitoring, behavior change, and habit formation significantly contributed to my design ideation and refinement. In addition to leveraging user research (bottom-up approach), designers should adeptly synthesize secondary research (top-down approach) into design decisions. I think it's a good habit to keep ongoing secondary research throughout the design journey, extending beyond those directly relevant sources/products for more inspirations.
Next Step.
🫶 Continue to enhance the accessibility through usability tests
To validate the actual feasibility and accessibility of our proposed solutions, we will conduct usability tests after app implementation and clinic trials. Patients with vision loss, cognitive issues, and other types of disabilities will also be involved in future tests to enhance the accessibility and inclusiveness of our application.