Improving Cooper Hewitt website’s mobile browsing experience for aspiring designers with eye tracking
Overview
The Cooper Hewitt Smithsonian Design Museum website is an entryway for visitors to access the historical and contemporary design collection, resources, and learning opportunities. To further enhance the experience of accessing the museum’s diverse and comprehensive information from mobile devices, a team of 4 UX Researchers conducted an eye-tracking study with retrospective talk aloud method for the Design Topics and Emerging Designers pages of the Cooper Hewitt website. While the website had a readable structure and interesting robust information, the overall site and its Design Topics page can be improved by changing some of the interactive elements, layout, and content.
My Role
UX Research
Data Analysis
Data Visualization
Presentation Design
Highlights Reel Editing
Administrator
Tools
Tobii Eye Tracker
Figma
iMovie
Zoom
Project Type
Advanced UX Research & Evaluation
Team
Olivia Turpin, Shih-Wen Huang, Wuke Zhou, & myself
Period
Mar 2022 - May 2022
The Problem
Efficacy of the visual design/hierarchy for Design Topics & Emerging Designers
The Cooper Hewitt Museum, represented by the lead content strategist, Pam, was interested in understanding how users browse and comprehend the content on the Design Topics and Emerging Designers pages, which were linked to many other pages that commonly researchers and aspiring/enthusiastic designers would use to find resources and learning opportunities across the site. It was especially important to conduct the usability test of the Design Topics page, because it was a recently implemented resource page and it was not clear whether the new way of aggregating the design topics made sense to visitors.
The research team’s objectives for this project were to understand how users are looking at the visual design and hierarchy of the Design Topics and Emerging Designer pages while browsing on a mobile device.
Research Questions to check the consistency and UX experience of the 2 pages
The get insights about the visual design, hierarchy, and overall user experience of the two pages, our team created 4 research questions to guide the research:
Testing & Analysis Process
Methodology: Combine User Interview with Eye Tracking and RTA
Usability testing with tasks inspired from our hypothesis
The team developed the following hypothesis at the same time the research plan for the in-person lab testing was made →
People usually don’t use a mobile device to perform research- oriented tasks.
This hypothesis led the team to focus the tasks around what people would tend to do on a mobile device, which are quick action tasks like purchasing a ticket to an exhibition, or finding out where a documentary was screening, and less about reading through a lot of content for in-depth research purposes.
Supplementing data with Eye-Tracking
A total of 8 participants who were aspiring designers and occasional visitors of museums completed the tasks on an iPhone 11 while their eye gaze was tracked through the Tobii Eye Tracking software. The team asked the participants to silently complete the tasks while their session was recorded in order to not interfere with the eye-tracking data as much as possible.
Eye Tracker setup for mobile
Understanding behaviors with Retrospective Think Aloud (RTA)
After completing the tasks, the participants watched a video replay of their gaze and explain the thoughts and feelings they had at the time. This think aloud session was recorded using Zoom recorder to revisit and make into a highlights reel later.
Participant engaged in a RTA session
Gathering quantitative data about the website usability
After the RTA was completed, participants were asked to answer a System Usability Scale (SUS) questionnaire (consists of 10 questions and 5 response options) that offer a quick and reliable way to measure the usability and user satisfaction with a website and compare it to relevant industry benchmarks. With the SUS scoring system, the usability scale score for the pages we tested across the site was 56, a little below the benchmark score (68) we were aiming for but still ranking higher than 21% of other websites.
Gaze plots were also generated through the Tobii EyeTracking Software. Analyzing the gaze patterns together with the user’s input and the SUS score provided a better understanding of the pages.
The team also kept a record of the participants’ task effectiveness, efficiency, and difficulty rating, which had averages within positive ranges.
Opportunities to Improve the User Experience of the overall website and Design Topics page
Analyzing the data revealed that the website was doing well at following a readable structure, and providing robust and purposeful resources, but there were still a few areas for improvement found. Interestingly, although the team designed the tasks around Design Topics and Emerging Designers, the findings turned out to be similar and more broader challenges that can be observed throughout the website. Therefore, our final key findings and recommendations were applicable to not only the Design Topics and Emerging Designers pages, but also the entire website.
Finding 1: Clickable elements weren’t recognizable
1.1 Page headers didn’t look clickable
A common theme emerging from the participants was that clickable areas on the website did not seem highlighted enough. Page headers, along with its header images were one of the elements that did not look clickable. Unlike the desktop experience where mouse cursors will provide an indication of a clickable element upon hover, there was no way to tell if a heading was clickable until actually tapping on it because headings didn’t have any sign or indication that they link to another page on the mobile experience.
Highlight clip of a participant saying that the headers on the Design Topics page didn’t look clickable.
1.2 Hyperlinks don’t stand out as CTA buttons
“I wouldn’t intuitively click on it, I would just think it was a piece of text and scroll through it.” — Participant 3
“I know it's a different color but I feel like I could just skip over it" — Participant 1
The orange hyperlinks throughout the website were visible, but did not stand out as Call to Action buttons for some participants who thought that they were just another piece of text to scroll or skip through.
1.3 Too many hyperlinks are distracting
There were sometimes too many hyperlinks within the body text that it distracted the user from finding specific information. For one of our tasks where we asked the participant to find the Mud Frontier Film’s screening locations, the gaze plot of the page showed that participants were especially looking at the places where the hyperlinks were at.
"I wish they highlighted the location better. The first thing I'm reading is [what's highlighted in orange] but I don't know where any of these festivals are." — Participant 1
“I'm distracted by those orange texts." — Participant 7
Recommendation: Differentiate what’s clickable
Design Topic page mockup with card design and button links
Differentiating elements that are clickable by using a card design for content that links out to more pages.
Incorporate noticeable buttons instead of a simple text hyperlink.
Finding 2: Mobile isn’t optimized
2.1 Mobile users were stuck in an infinite scroll.
The pages are too long to read through on a mobile device leading to what feels like infinite scrolling.
Highlight clip of a participant scrolling through a long page.
Figure 1: Average Task Completion Time
The usability test tasks that took a long time to complete usually involved pages with a long scroll as well. For example, Task #2 and Task #3 involved looking for information in the Image Description Guidelines and the Health/Design & Healing Exhibition pages respectively, which were pages with the longest scrolling. These tasks took the first and third longest time to complete (see Figure 1).
2.2 Important Call To Action (CTA) buttons are at the bottom of the page
The Call to Action buttons that were linked to an exhibition tickets page or a workshop registration form were located at the bottom of the page instead of the top of the page where most participants expected it to be at.. Good news tasks involved participants to use Call to action button were with 100% completion rate. However participants also responded if it wasn’t performing the task as requested, they would have opt out from the website.
Highlight clip of a participant skipping over most of the text to find the Design Snacks workshop CTA
Recommendation: Optimize for mobile responsiveness
Design and Healing page mockup with a ticket purchasing CTA at the top of the page and a page break after 3 articles
Move CTAs to the top of the page on mobile to avoid infinite scrolling and not make the visitor give up
If there are many articles of information on a page, create a shorter page break on mobile to avoid infinite scrolling (eg. break after 3 articles)
Finding 3: Users were confused by ‘Design Topics’
The participants felt that the Design Topics page should include a broad categories of topics, and therefore thought that topics with names that were less specific belonged in the page more.
Highlight clip of what the participants thought about the ‘Design Topics’ page
Recommendation: Design Topics should be more general
According to the following additional quotes, participants thought that the ‘Design Access’ and ‘Health Design’ fit in the Design Topics page the most. Taking into account the observation from the highlight clip leads to our recommendation of grouping more specific subjects like ‘Mud Frontier’ into broader topics.
"My first thought was which design topic would [Mud Frontier] belong to? Like, for instance, there is Health Design and so I thought is there any design topic about nature or something?" — Participant 7
“I think the ‘Design Access’ and all the ‘Health Design’ pages are most relevant to Design Topic page. Others are not.” —Participant 5
Conclusion
Clients were happy to see that many of their hunches were confirmed
The team presented the research findings and suggestions in a slide deck report/presentation format to the lead content strategist, along with two other workers from the Cooper Hewitt Museum. The clients were willing to use our data to move forward on building the still in-progress Design Topics page, take into account the fact that users are likely to do quick tasks over deeply diving into research on a mobile device, and thinking about the duality between the mobile and desktop more. They were excited to share our report with all of their teams and stakeholders.
Our team presenting to the client via Zoom.
The team was also surprised and happy to see that a day after the presentation, changes were already starting to be made on the Design Topics page. Hyperlinks under each design topic content block were temporarily added to the sections because we mentioned that it was difficult to recognize clickable headers and images. We hope that a more noticeable button format can replace the hyperlink in the future, and our other insights and findings be considered if the site can be redesigned to be more mobile responsive.
Design Topics page with additional hyperlinks (orange text) added
Lessons Learned
Although we chose not to do these analysis because we didn’t think it would provide additional insight, our team could have planned to allot more time on trying out the Tobii Eye Tracking software’s additional analysis features besides gaze and heat maps to at least get a practice of visualizing more data.
Next Steps
Since the Design Topics page was still a work in progress, I would be interested in evaluating the page when it is finished in the future. Besides usability testing, conducting a website analysis would also be helpful to quantitatively analyze the success of the Design Topics page redesign compared to when it was not redesigned yet.