The World Health Organization estimates that over 5% of the world's population has disabling hearing loss, a figure that is expected to grow to 10% by 2050, a percentage already found today in the most disadvantaged countries. Hearing problems, including those related to aging, lead to communication difficulties that are often underestimated. These difficulties represent real barriers to communication, obstacles to the full social participation of the person, comparable to architectural barriers for individuals with motor disabilities. In face-to-face communication, both people with hearing loss and the hearing people who interact with them can encounter difficulties. The former often struggle to understand due to perceptual or linguistic deficits, sometimes aggravated by unsuitable environments; the latter can experience a sense of frustration or discomfort in the face of a lack of understanding, with the risk of withdrawing from the conversation. In this context, assistive technologies that convert speech into text or facilitate listening are called to play a crucial role, promoting more effective mutual understanding and offering support to all people with complex communication needs.
DETAILED DESCRIPTION OF PLANNED ACTIONS AND INTERVENTIONS
Activity 1:Selection and evaluation of applications for speech transcription
The main objective of this activity is to identify the best applications available for transcribing speech into text and for communication support through a testing phase with hearing people. The applications will be subjected to an evaluation process divided into two main phases:
-The subjects will participate in reading sessions of pre-processed and coded texts containing sentences and words of different types and frequency of use (high and low). The applications will be used to transcribe the contents read by different subjects, with male and female voices, also with regional inflections, in silent and noisy contexts.
-The performance of the apps will be compared with functional and non-functional criteria. For the former, the following will be evaluated: transcription accuracy, processing speed, best signal-to-noise ratio, multilingual support, option to save communications in the cloud and speaker recognition capability. For the latter: intuitive and user-friendly interface, simple and well-structured navigation, short learning times for new users, guarantee of continuity even in offline mode, minimum loading time for the application, ease of updating without service interruptions, compatibility with different operating systems (Android, iOS), demo mode, quick installation, cost and possible free version with basic mode. The weights of the various evaluation items will be agreed with the client.
2.Interaction phase and conversation tests:
-The subjects will use the applications to interact with an operator via the text displayed on the application and, if applicable, via integrated chatbots.
-Different environmental conditions (silent and non-silent) and distances from the microphone will be tested to evaluate the robustness of the applications.
-During some sessions, subjects will use noise-cancelling headphones to reduce the impact of external noise, simulating specific usage conditions.
Activity 2:Experimentation on subjects with hearing problems
In this activity, the applications selected in the previous phase will be tested on a sample of subjects with hearing disabilities to evaluate their effectiveness and usability in experimental and real contexts. In particular, the following will be involved: CeDisMa collaborators with cochlear implants; students with hearing loss or deafness attending the inclusion services of the Università Cattolica in Milan, and/or Brescia and Piacenza and the Politecnico di Milano; deaf people participating in events promoted by industry associations already collaborating with CeDisMa for other projects always on the theme of overcoming barriers to communication; patients with hearing problems attending the CUP of the Audiology department of the Policlinico hospital in Milan, a context within which CeDisMa is already operating for another project.
The experimentation will be divided into different phases:
1.Reading and transcription test:
-The sample of people with hearing problems will be divided according to clinical-rehabilitative parameters and with respect to the digital skills possessed.
-The subjects will read pre-processed texts using the selected applications.
-The performance of the apps will be compared with functional and non-functional criteria. For the former, the following will be evaluated: transcription accuracy, processing speed, best signal-to-noise ratio, multilingual support, option to save communications in the cloud and speaker recognition capability. For the latter: intuitive and user-friendly interface, simple and well-structured navigation, short learning times for new users, guarantee of continuity even in offline mode, minimum loading time for the application, ease of updating without service interruptions, compatibility with different operating systems (Android, iOS), demo mode, quick installation, cost and possible free version with basic mode. The weights of the various evaluation items will be agreed with the client.
-Only for this phase, congenitally deaf people, cochlear implant trials, with dysarthria or dyslalia that make speech difficult to intelligible will be excluded from reading.
2.Assisted conversation test:
-Subjects will interact with operators using applications to transcribe the operator's speech and provide responses based on the displayed text.
-Assisted conversation scenarios will also be tested using integrated chatbots, if available.
3.Evaluation of environmental conditions:
-The experimentation will include tests in indoor environments (e.g., offices, conference rooms) and outdoor environments (e.g., public areas) with different noise and lighting conditions.
-The performance of the applications will be analyzed at different distances from the microphone.
4.Collection of subjective feedback:
-Participants will complete questionnaires to provide feedback on ease of use, perceived effectiveness and any cognitive fatigue.
Activity 3:Testing on subjects with hearing aids or cochlear implants of devices capable of transmitting speech directly to the device (Listen WiFi and Sennheiser)
This activity aims to evaluate the effectiveness of the Listen WiFi and Sennheiser devices in improving listening and verbal comprehension of deaf subjects with hearing aids. The testing will focus on the analysis of hearing performance through the use of the device in real contexts and specific test procedures. The activity will be divided into different phases:
1.Interface and ease of use
-Analysis of the device manuals highlighting their compatibility with hearing aids and cochlear implants from different manufacturers and different versions.
-Evaluation of ease of installation
2.Listening and repetition tests:
-The sample of people with hearing problems will be divided based on clinical-rehabilitation parameters, personal education level and speech intelligibility.
-Subjects will listen to a pre-recorded text using the Listen WiFi and Sennheiser device, designed to optimize hearing in subjects with hearing aids/cochlear implants.
-After listening, participants will repeat the text out loud.
-The repetitions will be automatically transcribed using the app selected in the previous steps and compared with the original text to evaluate:
-Repetition accuracy (correct words compared to the original).
-Response time (interval between the end of listening and the start of repetition).
3.Evaluation of hearing quality and ease of use:
-Subjects will provide subjective feedback, through questionnaires, on the quality of the sound perceived through the Listen WiFi and Sennheiser devices and on their intuitive interface.
4.Analysis of environmental conditions:
-The test will be carried out in different noise conditions (silence, moderate noise, intense noise).
-The effectiveness of the device and the degree of text comprehension will be evaluated in indoor (e.g., offices) and outdoor (e.g., public areas) environments.
5.Collecting subjective feedback:
-Participants will complete structured questionnaires to express their degree of satisfaction with the Listen WiFi/Sennheiser device
-The questionnaires will include questions on ease of use, perceived cognitive fatigue and usefulness of the device in daily life situations.
Specific objective and expected result
The main objective of this phase is to evaluate the effectiveness of the Listen WiFi and Sennheiser devices in supporting listening and verbal repetition of subjects with cochlear implants.
The expected results include:
•Quantitative data on the accuracy of text repetition and response times.
•Qualitative data on the perception of audio quality and the overall experience of the participants.
•Identification of possible improvements for the integration of the Listen WiFi or Sennheiser devices in real-life contexts.
Inclusion criteria
•Subjects with congenital hearing problems, preverbal or postverbal, even with possible phonation difficulties
Exclusion criteria
•Subjects with hearing problems in comorbidity with cognitive and/or personality disabilities
Activity 4:Drafting of Report and dedicated conference
Drafting of a report with the results of the different phases of the project carried out with a report in a specific conference at the UCSC to make the results of the action research carried out public. The possibility of disseminating them in any different accessible formats (leaflets, brochures, flyers, Open Access materials, tutorials, etc.) will also be evaluated, in order to disseminate the results in a widespread manner, contributing to the promotion of an increasingly inclusive perspective.
DETAILED DESCRIPTION OF PLANNED ACTIONS AND INTERVENTIONS
Activity 1:Selection and evaluation of applications for speech transcription
The main objective of this activity is to identify the best applications available for transcribing speech into text and for communication support through a testing phase with hearing people. The applications will be subjected to an evaluation process divided into two main phases:
-The subjects will participate in reading sessions of pre-processed and coded texts containing sentences and words of different types and frequency of use (high and low). The applications will be used to transcribe the contents read by different subjects, with male and female voices, also with regional inflections, in silent and noisy contexts.
-The performance of the apps will be compared with functional and non-functional criteria. For the former, the following will be evaluated: transcription accuracy, processing speed, best signal-to-noise ratio, multilingual support, option to save communications in the cloud and speaker recognition capability. For the latter: intuitive and user-friendly interface, simple and well-structured navigation, short learning times for new users, guarantee of continuity even in offline mode, minimum loading time for the application, ease of updating without service interruptions, compatibility with different operating systems (Android, iOS), demo mode, quick installation, cost and possible free version with basic mode. The weights of the various evaluation items will be agreed with the client.
2.Interaction phase and conversation tests:
-The subjects will use the applications to interact with an operator via the text displayed on the application and, if applicable, via integrated chatbots.
-Different environmental conditions (silent and non-silent) and distances from the microphone will be tested to evaluate the robustness of the applications.
-During some sessions, subjects will use noise-cancelling headphones to reduce the impact of external noise, simulating specific usage conditions.
Activity 2:Experimentation on subjects with hearing problems
In this activity, the applications selected in the previous phase will be tested on a sample of subjects with hearing disabilities to evaluate their effectiveness and usability in experimental and real contexts. In particular, the following will be involved: CeDisMa collaborators with cochlear implants; students with hearing loss or deafness attending the inclusion services of the Università Cattolica in Milan, and/or Brescia and Piacenza and the Politecnico di Milano; deaf people participating in events promoted by industry associations already collaborating with CeDisMa for other projects always on the theme of overcoming barriers to communication; patients with hearing problems attending the CUP of the Audiology department of the Policlinico hospital in Milan, a context within which CeDisMa is already operating for another project.
The experimentation will be divided into different phases:
1.Reading and transcription test:
-The sample of people with hearing problems will be divided according to clinical-rehabilitative parameters and with respect to the digital skills possessed.
-The subjects will read pre-processed texts using the selected applications.
-The performance of the apps will be compared with functional and non-functional criteria. For the former, the following will be evaluated: transcription accuracy, processing speed, best signal-to-noise ratio, multilingual support, option to save communications in the cloud and speaker recognition capability. For the latter: intuitive and user-friendly interface, simple and well-structured navigation, short learning times for new users, guarantee of continuity even in offline mode, minimum loading time for the application, ease of updating without service interruptions, compatibility with different operating systems (Android, iOS), demo mode, quick installation, cost and possible free version with basic mode. The weights of the various evaluation items will be agreed with the client.
-Only for this phase, congenitally deaf people, cochlear implant trials, with dysarthria or dyslalia that make speech difficult to intelligible will be excluded from reading.
2.Assisted conversation test:
-Subjects will interact with operators using applications to transcribe the operator's speech and provide responses based on the displayed text.
-Assisted conversation scenarios will also be tested using integrated chatbots, if available.
3.Evaluation of environmental conditions:
-The experimentation will include tests in indoor environments (e.g., offices, conference rooms) and outdoor environments (e.g., public areas) with different noise and lighting conditions.
-The performance of the applications will be analyzed at different distances from the microphone.
4.Collection of subjective feedback:
-Participants will complete questionnaires to provide feedback on ease of use, perceived effectiveness and any cognitive fatigue.
Activity 3:Testing on subjects with hearing aids or cochlear implants of devices capable of transmitting speech directly to the device (Listen WiFi and Sennheiser)
This activity aims to evaluate the effectiveness of the Listen WiFi and Sennheiser devices in improving listening and verbal comprehension of deaf subjects with hearing aids. The testing will focus on the analysis of hearing performance through the use of the device in real contexts and specific test procedures. The activity will be divided into different phases:
1.Interface and ease of use
-Analysis of the device manuals highlighting their compatibility with hearing aids and cochlear implants from different manufacturers and different versions.
-Evaluation of ease of installation
2.Listening and repetition tests:
-The sample of people with hearing problems will be divided based on clinical-rehabilitation parameters, personal education level and speech intelligibility.
-Subjects will listen to a pre-recorded text using the Listen WiFi and Sennheiser device, designed to optimize hearing in subjects with hearing aids/cochlear implants.
-After listening, participants will repeat the text out loud.
-The repetitions will be automatically transcribed using the app selected in the previous steps and compared with the original text to evaluate:
-Repetition accuracy (correct words compared to the original).
-Response time (interval between the end of listening and the start of repetition).
3.Evaluation of hearing quality and ease of use:
-Subjects will provide subjective feedback, through questionnaires, on the quality of the sound perceived through the Listen WiFi and Sennheiser devices and on their intuitive interface.
4.Analysis of environmental conditions:
-The test will be carried out in different noise conditions (silence, moderate noise, intense noise).
-The effectiveness of the device and the degree of text comprehension will be evaluated in indoor (e.g., offices) and outdoor (e.g., public areas) environments.
5.Collecting subjective feedback:
-Participants will complete structured questionnaires to express their degree of satisfaction with the Listen WiFi/Sennheiser device
-The questionnaires will include questions on ease of use, perceived cognitive fatigue and usefulness of the device in daily life situations.
Specific objective and expected result
The main objective of this phase is to evaluate the effectiveness of the Listen WiFi and Sennheiser devices in supporting listening and verbal repetition of subjects with cochlear implants.
The expected results include:
•Quantitative data on the accuracy of text repetition and response times.
•Qualitative data on the perception of audio quality and the overall experience of the participants.
•Identification of possible improvements for the integration of the Listen WiFi or Sennheiser devices in real-life contexts.
Inclusion criteria
•Subjects with congenital hearing problems, preverbal or postverbal, even with possible phonation difficulties
Exclusion criteria
•Subjects with hearing problems in comorbidity with cognitive and/or personality disabilities
Activity 4:Drafting of Report and dedicated conference
Drafting of a report with the results of the different phases of the project carried out with a report in a specific conference at the UCSC to make the results of the action research carried out public. The possibility of disseminating them in any different accessible formats (leaflets, brochures, flyers, Open Access materials, tutorials, etc.) will also be evaluated, in order to disseminate the results in a widespread manner, contributing to the promotion of an increasingly inclusive perspective.