AI puts 11 African languages ​​in front of neural network model

Last update April 21, 2022 at 06:46 AM

The researchers developed a model ofIA to help computers work more efficiently with a wider variety of languages ​​– extending natural language processing (NLP) capabilities to languages African who are heavily underrepresented in AI.

 

African languages ​​have received little attention from computer scientists, so little NLP capacity has been available for large swathes of the continent. But a new linguistic model, developed by researchers at the University of Waterloo in Canada, fills this gap by allowing computers to analyze text in African languages ​​for many useful tasks.

 

Lo-Fi Player - Become a Lo-Fi Music Composer Using Artificial Intelligence

 

AI puts 11 African languages ​​in front of neural network model Click to tweet

 

 

AI at the service of the African language.

 

The new neural network model, which the researchers dubbed AfriBERTa, uses deep learning techniques to achieve "state-of-the-art" results for low-resource languages, according to the team.

 

It works specifically with 11 African languages, including Amharic, Hausa, and Swahili, which are spoken collectively by over 400 million people, and achieves output quality comparable to the best existing models despite learning from it. a single gigabyte of text, while other models require thousands of times as much data, the researchers said.

« Pre-trained language models have transformed the way computers process and analyze text data for tasks ranging from machine translation to answering questions Said Kelechi Ogueji, master's student in computer science at Waterloo. " Unfortunately, African languages ​​have received little attention from the research community.s.

“One of the challenges is that neural networks are incredibly heavy on text and data processing. And unlike English, which has enormous amounts of text available, most of the roughly 7 languages ​​spoken around the world can be characterized as low-resource, in the sense that there is a dearth of data available to fuel it. neural networks hungry for data.

 

Artificial intelligence can diagnose cancer in less than 5 minutes

 

According to the researchers, most of these models work with a technique known as pre-workout. To do this, the researchers presented the model with text where certain words had been covered or masked.

The model then had to guess the masked words. By repeating this process several billion times, the model learns statistical associations between words, which mimic human knowledge of language.

"There are many advantages to being able to pre-train models that are just as accurate for certain downstream tasks, but using much smaller amounts of data.“Said Jimmy Lin, president of the Cheriton School of Computer Science.

« Needing less data to train the language model means less computations are required and, therefore, less carbon emissions associated with operating massive data centers. ", he added. “Smaller data sets also make data retention more convenient, which is one approach to reduce bias in models."

Lin thinks the research and the model make a " not small but important To bring natural language processing capabilities to more than 1,3 billion people on the African continent.

 

source: eandt.theiet.org