Toward a Society in which Deaf and Hard-of-Hearing People Can Maximize Their Abilities with Assistive Smart Glasses
─A Collaboration with Pixie Dust Technologies, Inc.

A Communication Barrier That Has Always Been Felt as It Is by Deaf and Hard-of-Hearing People Since Their Infancy

In Japan, the number of deaf and hard-of-hearing people is estimated to be 14.60 million¹⁾. On the other hand, only 14.4% of them are said to own hearing aids²⁾. Hence, 12.24 million deaf and hard-of-hearing people have no hearing aids. In addition, the user satisfaction with their hearing aids is low at 38%²⁾; Sumitomo Dainippon Pharma therefore considers the issues imposed on deaf and hard-of-hearing people has not been completely resolved with the use of conventional hearing aids alone.

Major reasons for the low use rates and low satisfaction with hearing aids include the poor usability (painstaking handling) and difficult communication with more than one person. “Painstaking handling” includes “uncomfortably,” “tightness,” “painfulness,” “overly large size,” “still no need for use in daily activities,” and other reasons²⁾.

Target Issue and Ultimate Goal

Issues likely to be felt by deaf and hard-of-hearing people during communication with more than one person include “communication barriers” such as the inability to hear well in cases where more than one person speaks at one time, and difficulty to know who is speaking.

In fact, users of hearing aids are dissatisfied. Their complaints include “Since I become fatigued when wearing the device continuously, I remove it except in necessary situations,” “While conversing with more than one person, I don’t know who is speaking,” “I can’t hear clearly, because the voices are mixed together,” “I can’t differentiate between the Japanese “a” and “ga” pronunciations,” and “I can’t sense the direction of the voices.”

In interviews with persons engaged in supporting the employment of deaf and hard-of-hearing people, one interviewee voiced an opinion that the inability to hear was not the only issue with hearing impairment. For example, persons who have had hearing impairment since their infancy reportedly have fewer opportunities for communication than hearing people. This reduces the opportunity to build vocabulary and hampers the understanding of nuance, which in turn can cause painstaking communication for deaf and hard-of-hearing people, as they interpret the same speech differently from hearing people.

A Society in which Communication Barriers between Deaf and Hard-of-Hearing People and Hearing People are Broken,
to Allow Deaf and Hard-of-Hearing People to Maximize Their Abilities

PxDT and Sumitomo Dainippon Pharma are committed to fighting these issues, aiming to create a world in which deaf and hard-of-hearing people can maximize their abilities, by providing Assistive Smart Glasses for Deaf and Hard-of-Hearing People.

Assistive Smart Glasses for Deaf and Hard-of-Hearing People first targets deaf and hard-of-hearing people of working age. If deaf and hard-of-hearing people begin using this device in their childhood, the communication barrier between deaf and hard-of-hearing people who have been disordered since their infancy and hearing people might be broken. The device is expected to help deaf and hard-of-hearing people expand their potentials.

Assistive Smart Glasses for Deaf and Hard-of-Hearing People Targets the “Working Generations”

When a person wears Assistive Smart Glasses for Deaf and Hard-of-Hearing People, subtitle text is displayed to allow the wearer to understand who has spoken what. The subtitles appear along the direction of the speaker, even among multiple speakers.

A noteworthy feature is that the “direction of sound” is provided. Not only bilaterally deaf and hard-of-hearing people, but also unilaterally deaf and hard-of-hearing people sometimes encounter difficulty perceiving the direction of sound, because sound is not clearly hearable when the person is spoken to in the direction from the affected side, and information on sound differences between the left and right ears is difficult to utilize. Even so, this device clearly notifies the wearer as to from which direction the wearer is spoken to.

In addition, Assistive Smart Glasses for Deaf and Hard-of-Hearing People particularly targets deaf and hard-of-hearing people in the working generations. This is because they are considered to represent a population in large need of it, as they face many troubles during work hours. The developmental project is ongoing, with an aim for the device to be fully effective during meetings and communication with colleagues.

Furthermore, we suppose that our device is likely to spread among the current working generations because they are familiar with IT tools such as computers and smartphones, and are hence face a lower psychological hurdle in the use of the smart glasses. To determine whether the solution could support deaf and hard-of-hearing people in their workplaces, Sumitomo Dainippon Pharma asked some 20 persons, including deaf and hard-of-hearing people and persons from companies employing or supporting the employment of deaf and hard-of-hearing people, to actually use the prototype, and heard their opinions. The following voices were heard:

As a result, the hearing showed that all the 20 subjects who used the prototype had positive responses, such as “This solution is helpful” and “I want this”; Sumitomo Dainippon Pharma became confident that the device would help resolve the issues.

Social Implementation of Research and Technologies by PxDT

The aforementioned Assistive Smart Glasses for Deaf and Hard-of-Hearing People is a device that has been produced through a joint R&D project of PxDT and Sumitomo Dainippon Pharma.

Beneath this joint R&D project was the “Pixie Nest™” program, run by PxDT. Pixie Nest™ is a consortium in which PxDT serves as a bridge between social needs and advanced technology to promote the resolution of issues and creation of value, and to create new businesses.

Pixie Nest is a New Business Project/Forum for Social Issue Solutions

The two companies found through discussions at Pixie Nest™ that “hearing aids represent the major solution for deaf and hard-of-hearing people at present, but are used at extremely low frequencies.” They then started this project to approach the communication barriers for deaf and hard-of-hearing people by developing and providing the Assistive Smart Glasses for Deaf and Hard-of-Hearing People, an eyeglass-type device to present “subtitle“ text that allows them to understand what is spoken by others.

To date, the two companies have been working together to develop elemental technologies and prototypes of Assistive Smart Glasses for Deaf and Hard-of-Hearing People, thus confirming social needs for the smart glasses through the aforementioned interviews with deaf and hard-of-hearing people.

In September 2021, the two companies concluded a new agreement for joint R&D activity to upgrade the user interface of Assistive Smart Glasses for Deaf and Hard-of-Hearing People, and to improve its overall usability.

What Is Pixie Dust Technologies?

PxDT is a company engaged in building structures for quickly implementing university-based research and technology for solutions to social issues. Its business encompasses a broad range of technical developments, mainly audio and visual user interfaces, sensing and data processing for image and spatial recognition, and their applications.

The core business fields are digital transformation in workplaces and healthcare/diversity. Ongoing projects in the healthcare/diversity fields include xMove™, an automatic wheelchair as a solution to issues in long-term care facilities, and SOUND HUG™, a device for persons with hearing impairment to enjoy sound through light and vibrations.

Example Projects of Pixie Dust Technologies

The collaborative business of Sumitomo Dainippon Pharma and PxDT aims to create a society in which deaf and hard-of-hearing people can maximize their abilities, by breaking the communication barrier faced by deaf and hard-of-hearing people through joint R&D for highly usable Assistive Smart Glasses for Deaf and Hard-of-Hearing People.

Further Upgrades for Market Launch in 2023

This product is scheduled to be launched in fiscal 2023. The project plans include development of a product with worker-friendly specifications that initially targets deaf and hard-of-hearing people in working generations with high needs.

Sumitomo Dainippon Pharma and PxDT are planning to continue upgrading the product for higher comfort of use and better design. The results of the latest user hearing showed that they favorably evaluated the product’s concept to display what is spoken visually in a way that indicates who is the speaker. They found, however, that the current smart glass hardware is still poorly evaluated; the product remains to be improved. It is necessary to improve the hardware to have an eyeglass-like shape suitable for daily use, and to reduce weight to reduce fatigue with long-term use.

With these usability survey results in mind, an upgraded device will be developed in the joint project Sumitomo Dainippon Pharma wants to upgrade the product to have better usability to expand the markets to include children and older adults with hearing impairment.

Furthermore, overseas marketing outside Japan will be considered in the future. While a Japanese language version of the device will be developed first, to be launched into the Japanese market, Sumitomo Dainippon Pharma plans to develop global markets to enable deaf and hard-of-hearing people outside Japan to maximize their abilities in the future.

In addition to Assistive Smart Glasses for Deaf and Hard-of-Hearing People, we at the Frontier Business Office of Sumitomo Dainippon Pharma Co., Ltd. are endeavoring to conduct R&D for new values and business by combining our own knowledge that has been compiled through R&D activities for pharmaceuticals for neuropsychiatric diseases and unique technologies, findings, and patents of our business partners. Please contact us to discuss your wishes and potential for collaboration as our business partner.