| 坂井 優介 | D, 中間発表 | 自然言語処理学 | 渡辺 太郎, 中村 哲, 上垣外 英剛 |
|
title: Investigation of the Inference Capabilities and Memorization of Pre-trained Language Models
abstract: Pre-trained Language Models (PLMs) can answer known problems using acquired knowledge and natural language understanding capability from pre-training, while unknown problems require pure inference capabilities to answer. To evaluate pure inference capabilities, we need to separately consider memorization capability, which is difficult with existing datasets due to its known information in PLMs. This study targets Knowledge Graph Completion (KGC), predicting unknown relations (links) from known ones in the knowledge graphs. Traditional embedding-based KGC methods predict missing links from pure inference capability, while recent PLM-based KGC methods also utilize knowledge obtained in pre-training. Therefore, KGC is suitable for evaluating the effect of memorization capability and inference capability. We propose a method to construct datasets for measuring performance of memorized knowledge and inference capability in KGC. We discuss whether PLMs make inferences based on memorized knowledge about entities and its conclusion suggests that PLMs also learn inference capabilities for unknown problems. language of the presentation: Japanese 発表題目: 未知の知識に対する事前学習済み言語モデルが持つ推論能力の調査 発表概要: 事前学習済み言語モデル (PLM) は事前学習時に獲得した言語理解能力や知識によって，既知の事象に対して推論を行うことができる一方，未知の事象に対してはPLMの推論能力のみで解を導き出す必要がある．しかし言語モデルの推論能力のみを評価するには，PLMが事前学習時に記憶した知識と獲得した推論能力を完全に切り分けた分析が必要となり，既存のデータセットで測定するのは，事前学習時の記憶が作用してしまうため困難である．本研究ではPLMの推論能力の分析に，知識グラフ上の既知の関係から欠損している未知の関係を予測するタスクである知識グラフ補完 (KGC) を対象とする．KGCにおいて埋め込みに基づく従来手法は推論のみから欠損箇所を予測する一方，近年利用されているPLMを用いた手法では事前学習時に記憶したエンティティに関する知識も利用している．そのためKGCは記憶した知識の利用と推論による解決との両側面を有することから，PLMが記憶する知識の影響を測るのに適したタスクである．我々はKGCに対し知識と推論による性能向上を切り分けて測定するための評価方法及びそのためのデータ構築手法を提案する．本研究ではPLMが事前学習時にエンティティに関する知識の記憶により推論を行っている箇所を明らかにし，PLMに備わっている未知の事象に対する推論能力も同時に学習していることを示唆する結果が得られた． | |||
| VASSELLI JUSTIN RAY | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, 中村 哲, 上垣外 英剛 |
|
title: A Closer Look at k Nearest Neighbors Grammatical Error
Correction
abstract: In various natural language processing tasks, such as named entity recognition and machine translation, example-based approaches have been used to improve performance by leveraging existing knowledge. However, the effectiveness of this approach for Grammatical Error Correction (GEC) is unclear. In this work, we explore how an example-based approach affects the accuracy and interpretability of the output of GEC systems and the trade-offs involved. The approach we investigate has shown great promise in machine translation by using the k nearest translation examples to improve the results of a pretrained Transformer model. We find that using this technique increases precision by reducing the number of false positives, but recall suffers as the model becomes more conservative overall. Increasing the number of example sentences in the datastore does lead to better performing systems, but with diminishing returns and a high decoding cost. Synthetic data can be used as examples, but the effectiveness varies depending on the base model. Finally, we find that finetuning on a set of data may be more effective than using that data during decoding as examples. language of the presentation: English | |||
| 四條 光 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, 中村 哲, 進藤 裕之 |
|
title: Investigation of LayoutLMv3 pre-training task to improve the accuracy of layout analysis
abstract: Layout analysis is the task of extracting the layout of a document, and we focused on the LayoutLMv3 model, which is the basic model for layout analysis. We showed the challenges of LayoutLMv3 in layout analysis, proposed a pre-training task to improve its accuracy, and evaluated the proposed task. language of the presentation: ***Japanese*** 発表題目: レイアウト解析の精度向上のためのLayoutLMv3の事前学習タスクの検討 発表概要: レイアウト解析とは,文書中のレイアウトを抜き出すタスクであるが、そのレイアウト解析の基盤モデルであるLayoutLMv3というモデルに着目した。レイアウト解析におけるLayoutLMv3の課題を示し,精度向上を目指して事前学習タスクの提案を行い, 評価を行った。 | |||
| MARTINEZ PEGUERO ARTURO | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, 中村 哲, 進藤 裕之 |
|
title: Reframed text generation
abstract: A glass that is half empty can also be seen as being half full. Carefully-rephrased wording can adjust our frame of reference and shift our point of view of the same fact. In other words, through meaning-preserving text style transfer, a change of perspective can be conveyed. My research seeks to develop a large language model-based text generation system that takes text as input, identifies relevant existing frames, and generates a re-framing of the input with a persuasive, appropriate and context-sensitive rephrasing. language of the presentation: English | |||