A multimodal interactive database that includes all monosyllabic sounds in the Mandarin Chinese language is providing educators, students, and researchers with a unique resource in which to help teach and study the language and has earned its creator, Catherine Ryu, Associate Professor of Japanese Literature and Culture, a prestigious national award.

Tone Perfect, a web-based teaching and research tool consisting of nearly 10,000 audio files, was developed by Ryu, a Department of Linguistics and Germanic, Slavic, Asian, and African Languages faculty member, in collaboration with the MSU Libraries digital humanities team.

Officially added to the MSU Libraries’ database earlier this month, Tone Perfect is an exhaustive collection of monosyllabic sounds in four tones, including a tone-sound combination that does not carry any lexical meaning. Mandarin Chinese is a tonal language in which each sound comes with four different tones where the same sound can hold entirely different meanings depending on the tone used. For example, “ma” can mean mother, hemp, horse, or scold depending on the tone you use.

Tone Perfect offers a full catalog of the Mandarin Chinese monosyllabic sounds (410 in total) in all four tones with each spoken by six native Mandarin speakers (three female and three male), resulting in a total of 9,860 audio files that can be accessed through the MSU Libraries website.

These audio recordings were first created to support the Picky Birds application, which is a language learning game that teaches students the four main Mandarin tones by helping them associate each tone with a corresponding colored bird. Picky Birds, which will be released at a later date, was developed by an MSU research team led by Ryu, who has a U.S. patent for the technology on which the game is based.

We would have purchased the audio files for the Picky Bird app if they were available, but nothing like this existed so we had to make them.

“We would have purchased the audio files for the Picky Bird app if they were available, but nothing like this existed so we had to make them,” Ryu said.

It took about a year to produce the nearly 10,000 audio files in what Ryu described as a “very arduous, rigorous, and prolonged process.”

Before even beginning the audio recordings, Ryu and her research team created a whole chart of all the sounds and came up with naming conventions to identify each sound. In addition, they had to select and train the six voice actors.

The actual recordings were done in the Center for Language Teaching Advancement (CeLTA) audio recording room. Working 6-7 hours per day, each voice actor produced 1,640 files. A three-person team then rated each recording for its naturalness and accuracy, and if a consensus wasn’t reached, the sounds were recorded again. Once this step was complete, a team of undergraduate students processed the files by segmenting and labeling them according to the naming convention and removed any distracting noises such as dental clicks. 

 “We really paid lots of attention to quality control and consistency,” Ryu said. “To that end, we had to create lots of ways to standardize things because there was so many people involved. In a way, we paved the way for any kind of audio project, how to run it, and the process that it involves.”

Then after creating the Picky Birds app, Ryu was left with all the audio files.

“Considering how much effort that was put into creating them, I really wanted to find a way to repurpose and archive the files,” Ryu said. “I wanted to create a repository where these resources could be utilized by other people.”

Ryu contacted the MSU Libraries digital humanities team for help in building an online database.

“The library saw merit in this because these kind of files did not exist before,’” Ryu said. “So we repurposed the audio files used for Picky Birds to create a completely different scholarly tool.”

Tone Perfect’s open-access files are available to the general public for free, are searchable, and can be downloaded directly from the website. The database also includes wave forms in color designed to reinforce tone-color associations while hearing the sound so as to visually reinforce what tone the listener is hearing.

There are many ways for researchers to use this, such as for experiments or to build games,” Ryu said. “The database also can be used by teachers to help teach students the language.

“There are many ways for researchers to use this, such as for experiments or to build games,” Ryu said. “The database also can be used by teachers to help teach students the language.”

Both the Picky Birds app and Tone Perfect database are a result of Ryu’s 2015 Tone Perception and Efficacy Study (ToPES) where Ryu and her research team studied the efficacy of different ways of training tone perception.

Tone Perfect recently received the 2018 Esperanto “Access to Language Education” Award presented by CALICO (Computer-Assisted Language Instruction Consortium) , the Esperantic Studies Foundation, and Lernu.net. The award selection committee found the goals, structure, and realization of the Tone Perfect website to be “very impressive and exemplary of the aims for the award.” 

Ryu attended the 2018 Digital Humanities Conference in Mexico City, Mexico, June 26-29, where she did a poster presentation on Tone Perfect. She also led a session on Tone Perfect and Picky Birds with hands-on demonstrations at the Chinese Language Teachers Workshop offered by the Confucius Institute at Michigan State University on August 14.

The Tone Perfect project is supported by the College of Arts & Letters, MSU Asian Studies Center, MSU Technologies, and the Office of the Vice President for Research and Graduate Studies.

For more information, visit the Tone Perfect website or see the Tone Perfect video.