Audiobook and "read aloud" feature in its current state (2024).

overview

The app

My team at Pearson (160M global customers) launched a direct-to-learner app, Pearson+ (5M users), in July 2021. Beyond accessing textbooks on any device, P+ empowered students with a suite of features to enhance their learning experience.

The team

Under the guidance of a director who championed neurodiversity and spent 3 years at Audible, my UX teammate and I set out to deliver a stellar audio experience.

problem

Spoken text was not synchronized with the on-screen text

Launching and using the audio player was clunky

Limited discoverability:

Launching the feature was not intuitive, leading to low rates of adoption.

Text-audio misalignment:

The written text on the screen did not align with the spoken narration, creating friction and disrupting the learning flow.

Solution

Add a read along (RA) feature that spotlights the text as its spoken

Establish patterns to access and launch features across the app

Establish logical patterns to access features.

Add a read along (RA) feature that spotlights the text as its spoken.

Goals

Measuring success by number of sessions, length of sessions, and recurring usage.

100%

increase in audio usage (6% to 12%)

50%+

customer awareness

Challenges

Distinguishing spotlighted text from highlighted text

We determined that the text will be "spotlighted" as it's spoken but realized this treatment could easily be mistaken for highlighting, another essential tool.

I used my visual and interface design skills to test, and land on, a creative solution.

Students listen, read, and study differently on mobile vs desktop

Accessibility: Pearson+ is available on all devices so we need to optimize for touch vs. mouse, screen size, and color modes.

Context: Learners have different goals depending on where they are and how they're using the app. Audiobook on mobile is primarily for on-the-go listening and optimizing non-traditional study times. Neurotypical students are more likely to read along with their audiobook using the desktop web app for their most challenging subjects and topics. They're intentionally sitting down to study and comprehend as much information as possible.

Customer demographic: The majority of our users are digitally native and fluent. They expect top-notch interactive experiences.

gathering feedback

Unlocking early insights with ratings and reviews

An initial research method was gathering and analyzing reviews from the App Store and Google Play.

1. No cost

2. Insight from early adopters and power users

3. Feedback is ongoing

Wireframes

Determining which access points to test

Content selector (location A): Opportunity

Seamlessly connect text and audio.

Audio drawer (location B): Most obvious

Add a line item to the existing audio settings

Analysis: wireframes of viable locations to access the feature

user testing

Make it easier for students to listen to their textbook with intuitive access points

I completed this first round of testing with my design lead.

who

Unmoderated testing with 10 English speaking higher ed students in the US.

how

Tested prototypes on usertesting.com, recorded the sessions, and assessed pain points.

hypotheses

RA will be easily discovered in the Audiobook drawer
Content selector discovery and usage may be confusing
RA will not be easily in display settings

Key results

✅ Audiobook drawer settings (Location B)

10/10 students easily found how to turn the read along feature on and off.

⚠️ Content selector dropdown (Location A)

Only 3 participants interacted instinctively.

Once prompted, 9/10 students said it made sense.

user testing

Determine how to synchronize text and audio by testing different visual treatments

hypotheses

Students will:

prefer the sentence and word spotlight option
strongly prefer the "fade" option of text spotlighting

Key results

✅ Sentence and word spotlighting style

13/20 students. Many found word only spotlighting too abrupt.

❌ The fade effect was not well received

9/20 strongly preferred "no fade" and 9/20 had no preference

Distracting in its current state, but saw potential if we can more closely match the fade to the cadence of the speaker.

approach

Proactively anticipating challenges

The initial audiobook feature introduced an alternative learning format for students. However, it was deliberately launched with a limited scope. We recognized the potential for shortcomings in user experience and functionality related to text-audio integration and access to the audio player.

Gathering feedback for improvement

Testing key elements

Text Highlighting: Evaluating various highlighting styles and functionalities to identify the most effective approach for aiding users in following the spoken narration visually.

Accessibility: Testing different access points for the audio player within the app to ensure easy and intuitive access from any screen.

ui & visual design

Launching audio and

I was responsible for delivering the final UI. All screens were designed in different color modes for desktop web and native app (iOS & Android). Examples below.

Color contrast: ensuring highlighting and spotlighting styles are accessible on all devices and color modes.

Spotlighted text visuals on default color mode

Spotlighted text visuals on dark color mode

Listening and Learning with Audiobook

Pearson+ Audio: From Misaligned to Synchronized