In a clinical setting, especially in the intensive care unit, a deficit in verbal communication causes a significant limitation of the patient’s autonomy. The temporary or long-term loss of speech ability caused by tracheostomy, neurological pathologies, or oncological diseases deprives a person of the fundamental opportunity to express their own needs. In such a situation, non-verbal methods, such as sign language, are often ineffective and ambiguous, while written communication, due to the patient’s general weakness, requires great physical resources. Precisely this alienation often becomes the cause of the patient’s anxiety and complications in the treatment process.
To overcome this communication barrier, the company Liopa turned 15 years of academic research from Queen’s University Belfast into a practical innovation. The visual speech recognition technology (VSR) developed by them relies on artificial intelligence and does not require a sound signal at all for functioning.
The SRAVI application performs the decoding of phrases only by analyzing the movement of the lips. The patient “pronounces” words in front of a smartphone camera only with the movement of the lips, and the system selects the most accurate versions in seconds. This helps a patient in a critical condition to escape from social isolation and to establish an effective, two-way connection with medical personnel.
Study Design
A multicenter study conducted in Northern Ireland observed the practical effectiveness of SRAVI for 18 months. During this period, researchers systematically collected data regarding the age, diagnosis, and delirium indicators of the patients.
The process proceeded as follows: patients selected desired phrases from a special list, and the application analyzed their lip movements and displayed three probable answers on the screen.
In the study, two different models of artificial intelligence were used:
DTW (Dynamic Time Warping): This algorithm is based on an individual approach. It studies the manner of lip movement of a specific patient. Since all people pronounce words with a different tempo and articulation, DTW fits the application to the specific characteristics of exactly one user.
DNN (Deep Neural Network): This model relies on massive “experience.” It has processed speech samples of more than half a million different people. The DNN uses these global data in order to recognize what any new patient has said without preliminary preparation.
The study in Northern Ireland clearly showed how difficult it is for technology to work in a real, critical environment. Despite the fact that the average age of the patients was 61 years, the main challenge still became their general state of health (87.5% of participants exhibited delirium). Naturally, such confusion of consciousness and general weakness had a direct influence on the accuracy of the primary version of the application, due to which the system showed only 21.8% accuracy.
However, the second version of the application, oriented toward free speech, improved the result to 34.6% compared to its predecessor. This dynamics confirms that artificial intelligence is gradually “learning” the specific articulation of the patients more and more.
According to the Lancashire pilot project, in optimal conditions, the system can show significantly higher results. Here, the accuracy of the DNN reached 75%, while the probability of recognizing the correct answer among three probable versions increased to 86%. These indicators point toward the great potential of the technology, although the physical condition of the patient still stands between theoretical success and practical use.
In a real environment, the specific position of the body became the main hindering factor; for example, while lying down, the tilt of the head often completely limited the perception of lip movement for the camera. Also hindering were environmental factors: sharp shadows, insufficient lighting, bandages. The application reaches maximum accuracy only when the patient sits in a vertical position, which is a fairly rare occurrence in the intensive care unit.
The SRAVI study makes it clear that the technology requires further refinement. Despite the existing challenges, the potential for its development in medical practice is quite large. However, digital innovation becomes an effective medical tool only when it merges with clinical experience. Speech therapist Ekaterine Chavchavadze speaks to us about the barriers caused by tracheostomy, social isolation, and the real therapeutic potential of the SRAVI application.
From a clinical point of view, what type of barriers are created by a tracheostomy or post-intubation tracheal stenosis, and how is it reflected on the quality of the patient’s verbal communication?
From a clinical point of view, a tracheostomy and post-intubation tracheal stenosis, along with anatomical-functional barriers, also create quite serious psychological discomfort for patients, which is directly reflected on their verbal communication.
A tracheostomy is a hole created by a surgical path in the windpipe, through which a person breathes by bypassing the upper respiratory tract, meaning the air does not reach the voice-producing apparatus—the larynx and the vocal cords; accordingly, the production of voice is disturbed, the person develops a weak, airy voice timbre, and the sensory stimulation of the larynx decreases, which is necessary for the implementation of swallowing, phonation, and their coordinated actions.
The total or partial loss of voice significantly reduces the strength and endurance of the voice, a person gets tired quite easily and quickly while talking, speech is deprived of intonations, emotion, and stresses, and sometimes the need for the use of alternative communication—gestures, written communication, augmentative means—is outlined.
Post-intubation tracheal stenosis is a scarring narrowing produced as a result of damage to the tracheal mucosa and cartilage after long-term intubation, which first of all creates clinical barriers: it increases airflow resistance, causes inspiratory and expiratory stridor, the pressure of the airflow necessary for voice production is weakened, the voice is strained and changed, and a person, due to the imbalance of breathing-phonation, is forced to speak in short phrases with a hoarse, strained voice timbre and frequent pauses, and shortness of breath develops. Verbal communication is so tiring that they often impose social isolation and communicative restrictions on themselves, which causes acute anxiety, panic reactions, withdrawnness, and pain caused by the loss of dignity.
How would you evaluate the therapeutic potential of the SRAVI application, and what should be taken into account when integrating it into clinical practice?
The therapeutic potential of the SRAVI application is quite large, taking into account the reality in which patients in emergency and intensive care units find themselves; first of all, this is a means of self-realization for the patients themselves, which is vitally important for them, the application makes the actions of medical personnel much easier, it makes their actions more purposeful and rational through feedback, and finally, the application is the best means for avoiding stress and experiences for the patient’s family members and close people.
In your opinion, who should manage the use of this technology in the ward so that the communication process is consistent and effective?
In the beginning, it is better for a specialist to start using the technology—a speech therapist, who with the help of speech tests and trials, will study in detail the speech portrait of the patient, their respiratory, articulatory, and phonatory resource, will be able, taking into account the patient’s personal-social and psychological individualism, to add necessary phrases and sentences to the application, will maximally fit the technology to specific speech capabilities and medical needs, and subsequently will help in the use and mastery both for the patient themselves and for their caregiver and family member, if the patient is unable to manage the use of the application independently.
How would you evaluate the influence of the restoration of communication on the patient’s psycho-emotional state, and how much can the timely overcoming of isolation accelerate the rehabilitation process?
Communication returns the possibility of self-control to the patient—I can be me again! They manage to express pain, joy, longing, to ask for help, to ask necessary questions; the reduction of the level of stress hormones increases their sense of security, reduces anxiety, depression, and post-traumatic symptoms; with immediate communication with the environment, fear disappears, and identity returns, because language is an important component of identity.
During communicative isolation, motivation is decreased, they exhibit therapeutic passivity, and upon the restoration of communication, therapeutic involvement increases, the patient follows instructions better, actively participates in the rehabilitation process, and feedback with the doctor is more intense. Cognitive skills also improve: attention, memory, thinking. Communication with family members and social support cause hope and strengthen the rehabilitation effect.
Studies confirm that upon the restoration of communicative skills, the need for the use of sedative medications in severe patients decreases. So the timely overcoming of isolation accelerates the rehabilitation process.
What type of risks might accompany communication based on artificial intelligence, and where does the line lie between technological capabilities and clinical safety?
The first thing that we might consider as a danger during AI-based communication is the automatic recognition of speech by similar technologies, the generation of speech from text, the generation of articulatory movements into text, which does not exclude the erroneous interpretation of information; for example, it might not distinguish dysarthric problems from dyslalic ones, it might not properly decode the linguistic lapses of an aphasic patient, it might not perform the differential diagnostics of symptoms, and as a result, it might set an erroneous diagnosis and outline ineffective therapeutic strategies.
Verbal communication is not only a linguistic process, it is an echo of a person’s biological, psychological, and social individualism, while AI technologies seriously limp when recognizing contextual features, cultural, and emotional nuances.
As for the safety line, any specialist should remember that although communication based on artificial intelligence is a strong supporting instrument, it can never replace the live relationship and empathy between a doctor and a patient; therefore, its integration must always be based on clinical supervision, ethical standards, and the principle of patient safety.
We should not grant clinical autonomy to AI, we must re-verify every one of its recommendations, we must perform an individual interpretation of results; along with this, the patient must be fully informed as to what level artificial intelligence participated in the process of their diagnostics and treatment; otherwise, the technological benefit might turn into irreparable clinical harm.
Source: Critical Care
