Evaluating and Optimizing the Usability Experience of an Audio Tours Web App: The Jewish Museum

Located on New York City’s Museum Mile, THE JEWISH MUSEUM combines art and Jewish culture, welcoming visitors from all backgrounds. Founded in 1904, it is the first Jewish museum in the U.S. and among the oldest globally. Its collection includes nearly 30,000 pieces, showcasing over 4,000 years of Jewish history and art.

In July 2019, the museum launched a mobile-friendly AUDIO TOURS WEB APP, now part of the Bloomberg Connects app. This app enhances the visitor experience both on-site and remotely by offering: 

• Audio Tours: Narrated guides to exhibitions and key artifacts.
‍
• Search: Quick access to audio clips using item numbers. 
‍
• Museum Guide: Navigation support to explore the museum. 

The app reflects the museum’s commitment to making its content more accessible and engaging. 

Screen recording of the Jewish Museum Web App in its original launch version

CHALLENGE

The Jewish Museum sought to enhance visitor engagement with its newly launched audio tours web app. The challenge was ensuring the app was user-friendly, intuitive for a diverse audience, and met its intended purpose effectively. 

OBJECTIVE

The goal was to assess and enhance the web app’s user experience through usability testing and analysis, identifying areas for improvement and providing actionable recommendations. 

DISCOVERY

Our team of four Usability Experts collaborated with museum representatives to gather key insights, laying the groundwork for our research. 

TARGETED USER GROUPS

• Jewish patrons aged 50 and above with limited tech-savviness. 

• Millennials with minimal experience using audio guides.

KEY QUESTIONS

• How do users engage with the web app’s interface?

• Do patrons find the web app intuitive and easy to use?

• How effectively does the web app enhance the overall museum experience for the visitors?

RESEARCH AND STRATEGY

METHODOLOGY

We conducted on-site user testing to observe how users interacted with the web app and museum artifacts. 

‍User Testing is a process where real users interact with a website, app, product, or service tasks in realistic conditions. Their experiences, feedback, and thoughts are then recorded and analyzed to evaluate the effectiveness of the interface and functionality. 

PRELIMINARY PREPARATIONS

• Moderator Script: For moderators to explain the project and guide participants through the experiment.
‍
• Consent Forms: For participants to grant permission to use the data collected during testing. 
‍
• Pre-test Questionnaire: To gather participants’ demographic information, smartphone experience, and museum familiarity.
‍
• User Tasks: Designed to explore the two primary functions of the web app: Taking a guided audio tour.Using the search function to access specific audio clips.

• Post-test Questionnaire: A survey to capture participants’ feedback, experiences, and thoughts on the app. 

• Incentives: Participants received a $10 Amazon gift card and a free admission ticket to the Jewish Museum as a thank-you for their participation.

• Materials: Items like smartphones, audio devices, pens, and alcohol wipes for cleaning electronics. 

Team conducting usability testing  at The Jewish Museum

EXPERIMENT

• We recruited 8 museum visitors on-site, with an even split between young adults and those aged 50+.

• Participants were asked to perform tasks while thinking aloud. Their interactions with the web app were screen-recorded, and their verbal feedback was audio-recorded for analysis.

TASK A

“You have arrived on the museum’s third floor and would like to tour the exhibits. Find an audio tour for the third floor that is longer than 30 minutes. Complete the first five items in the guided tour. 

Feel free to use the device to explore the museum and audio guide web app. If you do not want to listen to the full audio for an object, feel free to skip to the next object.”

(From left to right): 1. Selecting an audio tour for an artifact on the 3rd floor, 2. Audio tour details page, 3. Playing the audio guide.

TASK B

“Find an object you want to learn more about with a museum label indicating an audio guide available. Search for the corresponding audio guide on the web app.

After you finish listening, return to the guided tour.”

(From left to right) 1. Search page, 2. Artifacts display

ANALYSIS

We examined the recordings and notes from the experiment to identify challenges, which were then graded on a scale of 1 to 4, ranging from cosmetic issues to catastrophic problems.

RESULTS

The overall sentiment from participants towards the web app was positive. One participant noted, “I really like that. It is exactly the information I am looking for in a museum app.”

• 100% of users agreed that the web app enhanced their museum experience.

• 87.5% of users would use it again during their next visit.

• 87.5% of users would use it again during their next visit.

Three key challenges were identified, and actionable recommendations to address them were outlined.

Challenge 1: WAYFINDING

• Artifacts from the same audio tour were frequently dispersed throughout the gallery. This caused wayfinding difficulties for 7 out of 8 participants due to the lack of a clear physical order.

• An image of the artifact is presented on the Directions page, accompanied by 1-2 lines of guiding text. Most users relied on the visual cues rather than reading the text. However, the images sometimes lacked specificity, which led to confusion for some users.

• Users who relied on the text were also confused, as the directions referenced the elevator instead of the previous audio clip's subject.

‍“I have no idea where I am going. It gives me no order, and that makes me not want to do this.“

RECOMMENDED SOLUTION

• Outlining the object within the images on the Directions page will emphasize the artifact and provide helpful contextual cues for locating it.

• Specifying the item’s floor number and/or gallery name can clarify its location.

• Present the text in bullet points to enhance reader comprehension. Guide users using multiple reference points, especially the object of the last audio clip.

(From left to right): 1. Directions for the Passover Seder Plate on the web app, 2. Mockup of the proposed solution.

These modifications would offer all users improved visual and textual cues, regardless of the starting point of their tour or familiarity with the objects. As a long-term solution, consider developing a map with clear indicators.

Challenge 2: THE NAVIGATION BAR BLENDS INTO THE BROWSER

• The web app's navigation bar is positioned at the bottom of the screen, a departure from the typical placement for web-based platforms. Users expected traditional hamburger menus located at the top of the page.

• The dark-colored navigation bar buttons blended into the equally dark background of the homepage, making them indiscernible.

• The Chrome browser's search icon was positioned directly under the web app's search button, leading 5 out of 8 users to inadvertently click on the browser's search icon.

RECOMMENDED SOLUTION

• The app's color palette is primarily dark grey, with blue as the secondary color. This holds across the app, except for the Search page, where blue becomes the primary color and grey is the secondary.

• To maintain brand consistency, the navigation bar could be colored blue without deviating from the brand colors. This change would make it stand out more effectively. Additionally, ensuring the Search page is consistent by using the primary dark grey background would promote overall coherence within the app's design.

(From left to right): 1. Web app Home page, 2. Web app Search page, 3. Recommended Home page mockup, 4. Recommended Search page mockup.

Challenge 3: NON-INTUITIVE PLAY LINK ON THE SEARCH RESULTS PAGE

• On the search results page, the link to the audio file is represented by a red arrow and text. This proved ambiguous for 4 out of 8 users, who instead clicked on the "Learn More" button underneath.

‍“I have no idea where I am going. It gives me no order, and that makes me not want to do this.“

RECOMMENDED SOLUTION

• Adhering to industry standards, encircling the arrow will make it appear as the intended play button, reducing the likelihood of users mistaking it for a bullet point or a menu option.

• Users often mistook underlined headers on the Homepage for clickable links, even though they were plain text. This observation suggests a solution: underlining text in the links may make them more conspicuous to users.

Furthermore, removing underlines from the headers on the Homepage could enhance clarity and prevent confusion about clickable elements.

(From left to right): 1. Web app search results page, 2. Mockup of the proposed solution.

CONCLUSION

Users were enthusiastic about the concept of the web app, its functionality, and its content, describing the information as enlightening. Overall, they expressed high satisfaction with their experience using the web app.

‍“I would like to keep listening to some of the audio tours. I really liked it!“

‍The project revealed key areas for improvement in the web app. The recommendations are practical for immediate implementation, with suggestions for long-term enhancements to further elevate the user experience, such as including a museum map and exploring various evaluation methods to assess the overall web app experience.