Augmented Human Communication

Research Staff

  • Prof. Satoshi Nakamura

    Prof.
    Satoshi Nakamura

  • Assoc.Prof. Yu Suzuki

    Assoc.Prof.
    Yu Suzuki

  • Assist.Prof. Sakriani Sakti

    Assist.Prof.
    Sakriani Sakti

  • Assist.Prof. Koichiro Yoshino

    Assist.Prof.
    Koichiro Yoshino

  • Assist.Prof. Hiroki Tanaka

    Assist.Prof.
    Hiroki Tanaka

E-mail { s-nakamura, ysuzuki, neubig, ssakti, koichiro, hiroki-tan }[at] is.naist.jp

"Go Beyond the Communication Barrier" and "Next Generation Big Data Analytics"

The AHC Laboratory performs research and education on a wide variety of technologies that support human-to-human and human-to-computer communication with the final goal of enhancing human communication abilities. Our research areas include multilingual speech translation, human-machine dialog system, communication quality of life (CQoL), voice conversion, silent speech interfaces, user-adaptive speech recognition/synthesis, and brain measurement and analysis.

NAIST launched its big data analytics project in April 2014. Big data including multi-dimensional social, biological, and material information are targets for data analytics and mining. The project also encourages close collaboration with industry. The AHC-lab plays a central role in the project. (for details, please see http://bigdata.naist.jp/ )

Research Area 1

Speech-to-speech translation

Speech-to-speech translation has been a long-standing dream, allowing for the possibility of seamless communication between people that speak different languages. Speech-to-speech translation recognizes the user's speech, translates it, and synthesizes a voice in the target language. Our current research project focuses on simultaneous speech translation of news and lectures. [Fig.1]

Spoken dialogue systems with verbal and non-verbal information

Our spoken dialogue system research aims to develop a computer avatar/agent that can communicate with humans intelligently and naturally. We focus on new statistical dialogue models for natural dialogue using intonation, emotion, personality, face and gesture information as well as verbal information. Individuality modeling is a study of what makes each person different. We study the individuality present in the human voice, face, expression, and dialogue modalities.[Fig.2]

Multi-lingual statistical speech processing

Speech recognition and synthesis are fundamental technologies for realizing natural human-computer interaction. We study statistical methodologies such as hidden Markov models, Gaussian mixture models, deep neural networks, and recurrent neural networks. We are extending these models for emotional speech, conversational spontaneous speech and multilingual speech.

Cognitive communication/brain analysis

Our research on cognitive communication analyzes brain activity to detect real-time communication difficulty using Electroencephalograms (EEG). We also perform research on education and support those with communication disabilities such as Autism. [Fig.3]

Natural language processing and understanding

Natural language processing aims to process human language (such as English or Japanese) using computers. Our research into natural language processing focuses on creating natural language interfaces between humans and computers, thus allowing computers to understand natural language queries and commands so that they may answer questions or follow directions.

Multimedia web information analysis

Huge amounts of multimedia information have been accumulated on the web. We conduct research on technology to analyze multilanguage and multimodal information and utilize it to enhance communication. [Fig.4]

Multimodal concept learning

Computers need not only understand language, but also understand objects, motions, and their connection with the words in language. Our research covers the concept of making computers study speech, language, image and motion linked together.

Research Area 2

Big data analytics

Big data analytics is one of the hottest topics in the area of information science. A variety of information, sensors, social network services, and lifelogging have become available through the development of telecommunications, and techniques of big data analytics are expected to add new values to such information in the real world. Our project addresses the problems of big data analytics by using real data provided by several research projects collaborated with research laboratories and schools inside NAIST, government offices, and private companies. We tackle real-world problems using techniques of data engineering and machine learning. The overall goals of this project are the extraction of knowledge from data and the development of data analysts and data scientists. [Fig.5]

Key Features

As a Super Research Group, SRG, we collaborate with other research laboratories within NAIST and other international research laboratories. We participate in the InterACT consortium with 8 research universities including CMU/KIT.
The AHC-lab provides an international research environment for students where all students can experience interaction and collaboration with students and faculty from all over the world.

Fig.1  Speech-to-speech translation

Fig.1 Speech-to-speech translation

Fig.2  Spoken dialogue system

Fig.2 A spoken dialogue system

Fig.3 EEG measurement system

Fig.3 A EEG measurement system

Fig.4   Web information analysis

Fig.4 Web information analysis

Fig.5 Big Data Analytics

Fig.5 Big Data Analytics