Hello, I am a Digital Sign Language Avatar.
I was created to help remove the communication barriers and information gaps for the Deaf community.
Wow, exciting!
While many Australians will have noticed more Auslan interpreters popping up in the corners of their TV screens, the reality is, we still have a vast undersupply of interpreters: just 306 at last count for a community of over 30,000 signing Deaf people. Now, imagine how much greater the need becomes for specialised interpreters who can deliver complex medical advice, culturally-sensitive community information, or emergency warnings during floods or bushfires.
This last, lifesaving category is the focus of a unique partnership between Humanitech Lab and the New Zealand-based software startup, Kara Technologies, which is refining a family of hyper-realistic ‘digital humans’ with the capacity to save real human lives.
Image: a trio of Humanitech Lab and Kara Technologies' signing avatars.
Signing avatars are nothing new, but what sets Kara’s apart is an ‘animation blending’ technology that has already captivated Deaf users with its ability to deliver clear, highly nuanced messages with uncanny realism. Kara believes that its tech offers a global first opportunity to give Deaf users instant access to emergency broadcasts in situations where interpreters are simply not available.
“Natural disasters don’t care what time of day they happen,” says Grace Covey, Kara’s passionate Communications Manager, who is profoundly Deaf and has two Deaf children. “If a fire breaks out at 3am, your local SES will send off an emergency SMS and most people will get the information fairly promptly from their radio or television. But can they 'magic up' an interpreter at that time? No way.”
Far from replacing interpreters, Kara’s tech offers a stop-gap solution that emergency services can use to generate an instant video with an avatar delivering specific messages on a user’s phone. The service manager writes an SMS in the Kara portal and the software uses a series of templates for the type of emergency (whether it's a flood, fire, cyclone or earthquake), to deliver a specific message about the event, its location and severity, and instructions from emergency providers.
“Right now, we’re looking at about one minute to create a video message and get the information out there,” says Grace. “With the current process, when an SMS text alert is sent off, an interpreter has to be contacted, an announcement written, a politician chosen to deliver it… it takes several hours at best.
“Any Australian who has lived through the fires, floods and lockdowns of the past few years knows that these hours can literally mean the difference between life and death.”
Over the pilot phase of Humanitech’s 2023 program, Kara will be testing its tech in partnership with relevant Australian emergency organisations and Deaf people, enabling users to report back on the practicalities of its application.
The team has come a long way since it began translating children’s books into sign language in 2017. Since then, Kara’s hardworking animators and signers have developed a large family of ‘digital humans’ from multiple races and genders, who can sit at the bottom of your computer or phone screen and translate audio, video or text content into signed language, virtually in real time.
Kara was set up by digital entrepreneur Arash Tayebi, who had his own ‘eureka moment’ when he contracted Ménière's disease and lost some of his hearing while studying for a PhD in Auckland. A quick survey of Deaf students showed him how marginalising hearing loss can be and Arash resolved to make a positive change for the Deaf community. He contracted more than 20 Deaf technologists and signing experts in the creation of his technology.
Under the guidance of co-founder Farmehr Farhour, Kara has rigorously tested and refined its avatar tech, using sophisticated motion-capture technology and cameras that pick up the tiny movements that accompany signed words and phrases. “It took a long time to capture the precise hand gestures and subtle changes in facial expressions,” explains Grace. “But those minute changes can alter the entire meaning of a sign and the message it’s conveying.”
Kara’s animation blending melds signs together seamlessly and immediately, says Grace, “stripping the jerkiness out of the movements and making them less robotic, so it looks just like a normal person signing”. Validation of the project by Humanitech last year provided funds and support for the company to approach emergency providers, as well as advocacy groups such as Deaf Connect, which has offered to support Kara with research and promotion. Now with Humanitech’s pilot project enabled by a second grant from the Telstra Foundation, they’re testing the technology to prove the need is there and potential users will use it.
“This project is very close to our hearts,” says Grace Covey. “I have two kids who are both Deaf and I’m always worrying about them being safe. When they’re older and our technology is more sophisticated, I now know that they’ll be safe wherever they go.”
