How to teach a chatbot to speak Samoan

NZ-based engineering consulting firm Beca looks to natural language processing to improve community engagement.

Author: Sarah Putt – New Zealand Editor, CIO

NZ-based engineering consulting firm Beca looks to natural language processing to improve community engagement.

When your everyday business is changing the physical environment that people live and work in, community consultation is key—but how do you find out what they really want? How do you reach the ‘silent majority’ who don’t speak up, especially those for whom English is a second language?

The answer for Matt Ensor, business director at Beca, is AI user interfaces that use natural language processing (NLP). Beca is an independent advisory, design and engineering consultancy headquartered in Auckland with more than 3,300 staff and 21 offices around the world, with its primary focus on Asia-Pacific. “AI is something that can really help communities, but you’ve got to do it in the right way. You’ve got have that respect for the diversity in the community.”

The genesis of Tala, the Samoan-speaking chatbot
Using IBM’s virtual agent platform Watson, Beca first created ‘Frankly’—an English-speaking conversational agent, or chatbot, that enables people to provide text-based feedback on a web browser through phone, tablet or PC. Ensor says they chose Watson technology because it is a “low-code” AI—which means Beca staff don’t need a developer background to use it. “They can train and teach Frankly, without having any knowledge of coding,” he says.

This low-code approach also widens the diversity of disciplines they were able recruit for a pilot known as the Talanoa Project, which was designed to introduce another language by which people could use Beca’s AI tool for community consultation. Beca employed a group of eight interns over the summer break as part of a scheme funded by Callaghan Innovation’s programme for undergraduates.

Diversity in what the interns are studying, as well as their own backgrounds, was an important factor in ensuring they could collectively create an AI tool that could engage effectively. Ensor notes that the subjects they were studying included engineering, law and psychology, and just about all of them are bilingual or multilingual.

One of the interns, Luke Fitzpatrick, a fourth-year Engineering Science student at the University of Auckland, is a fluent Samoan speaker, and so they decided to focus on that language for what became known as the Talanoa Project. Talanoa is Samoan for ‘talk’, although it soon became shortened to just ‘Tala’.

“When I first started, I didn’t know what a chatbot was, so that’s how easy it is to learn through Watson. In that first week I went from knowing nothing to learning a lot, and progressing that and the NLP, and developing a fully functional conversational agent after just less than ten weeks,” Fitzpatrick says.

Fitzpatrick says a challenge in creating Tala was training it to understand the differences between formal and casual Samoan, so that it was respectful for elders using the tool, while at the same time easy and natural for young people to communicate with.

Tala was trialled with 20 Samoan speakers, and Fitzpatrick says that many users were new to using AI tools, and while sceptical at first, “kind of relished the opportunity to converse through text in their mother tongue.”

Ensor says they had plans to extend the use of Tala “but then COVID hit.” Even so, they expect to pick the project up again and drive it beyond the pilot stage, taking what has been learnt and applying it to other languages. “Beca has a South Pacific, Australasian and Asian focus, so those are the languages we’re going to focus on,” he says.

Beca plans more AI-based projects
“One of my roles is to make sure that NLP gets embedded wherever its useful across the business, but also to keep that real client focus on it rather than people creating AI for AI’s sake,” Ensor says.

But conversational agents are just one of the AI projects that Beca is undertaking. In the past year the business has embraced the technology and is looking to embed it’s use throughout the company. “The benefit of AI is that you can leave it on forever; consultation doesn’t end. Once most of our clients do projects, they never go back and ask people what their thoughts are because it’s too expensive,” he says.

An AI steering group was set up a year ago and has evolved into a fortnightly call-in session where about 120 staff discuss their AI projects. For example, Beca is working with Metservice and several universities on using machine learning to improve meteorological data.

Beca currently uses IBM and Microsoft technology because that is what most of its clients are familiar with, but Ensor expects that in the future they will be using a whole range of different vendors’ products.

“What we are trying to do is encourage our staff to upskill in Watson and other AI, so that when they see something that can be accelerated or improved for our clients they have an understanding how they can do that and they might even be able to do that themselves,” Ensor says.

Click here for the CIO article.