| 栗生 紗希帆 | M, 2回目発表 | ソーシャル・コンピューティング | 荒牧 英治 | 渡辺 太郎 | 若宮 翔子 | 矢田 竣太郎 | |
|
title: Detecting signs of eating disorders using blog text abstract: It is estimated that approximately 220,000 people are affected by eating disorders in Japan, with a mortality rate of approximately 5%, the highest rate among mental disorders. Furthermore, the number of facilities offering highly specialised treatment for eating disorders in Japan is currently limited. Early detection of eating disorders is therefore an important issue. In this study, we considered that excessive dieting can lead to eating disorders and worked on the classification of dieting and eating disorder blogs to detect the signs of eating disorders. Specifically, diet blogs (number of data: 407,668, number of people: 826) and eating disorder blogs (number of data: 81,340, number of people: 380) were collected from Ameba blogs and a dataset was created. A ‘blog text unit’ model was constructed to classify each blog, and a ‘user unit’ model to classify the combined texts of the same user. We used 110,000 blog texts for the training data and 27,500 blog texts for the test data for the per-blog-text model, and 576 merged texts for the training data and 144 merged texts for the test data for the per-user model, to evaluate the results. The classification results showed that the Accuracy and F1 values for the blog text unit were 0.772 and 0.768 respectively, while the Accuracy and F1 values for the user unit were 0.671 and 0.655 respectively. Error analysis was also used to investigate whether ‘signs’ were associated with blogs that could not be classified correctly. language of the presentation: Japanese 発表題目: ブログテキストを用いた摂食障害の兆候検出 発表概要: 日本国内における摂食障害の罹患者数は，約22万人と推定されており，その死亡率は約5%と精神疾患の中では最も高率となっている．さらに，国内では，摂食障害に関する専門性の高い治療を受ける施設が限られているのが現状である．そのため，摂食障害の早期発見は重要な課題である．本研究では，過度なダイエットが摂食障害をもたらすと考え，その兆候を検出できるようダイエットと摂食障害ブログの分類に取り組んだ．具体的には，アメーバブログからダイエットブログ（データ数：407,668件，人数：826人），闘病記から摂食障害ブログ（データ数：81,340件，人数：380人）を収集し，データセットを作成した．ブログごとに分類する「ブログテキスト単位」モデルと同一ユーザごとに結合したテキストを分類する「ユーザ単位」モデルを構築した．ブログテキスト単位の訓練データには110,000件，テストデータには27,500件のブログテキストを用い，ユーザ単位モデルの訓練データには576件，テストデータには144件の統合テキストを用いて，評価を行った．分類結果は，ブログテキスト単位のAccuracyは0.772，F1値は0.768であり，ユーザ単位のAccuracyは0.671，F1値は0.655であった．また，エラー分析により，正しく分類できなかったブログと「兆候」と関連があるかを調査した． | |||||||
| 鈴木 刀磨 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎 | 荒牧 英治 | 上垣外 英剛 | ||
|
title: Investigation of Vulnerabilities in Large Language Models Caused by Task-Specific Surface Expressions
abstract: Large language models (LLMs) achieve high task generalization performance through pre-training on unlabeled data. However, their capabilities can be further enhanced by applying instruction-tuning, where various tasks are learned using instruction templates. In instruction-tuning, it is essential to ensure the diversity of instruction templates used during training to avoid overfitting. Accordingly, existing instruction datasets, such as the FLAN dataset, provide multiple templates for each task to ensure such diversity. However, these templates often contain task-specific surface expressions, such as words closely related to the target task. Biases in these instruction templates can be reflected in LLMs through training, potentially causing performance degradation when encountering certain surface expressions. In this study, we investigate vulnerabilities in LLMs that arise from task-specific surface expressions included in instruction templates. For this investigation, we propose a method that inserts target words into the instruction sentence while maintaining the content from a task perspective. Using instruction templates created from the FLAN dataset, we validated our method on benchmark datasets, including MMLU and BBH. The results revealed that the presence of task-related words in instruction sentences can significantly alter the output, regardless of the sentence’s meaning. These findings suggest that surface expressions in instruction templates may cause vulnerabilities in LLMs, providing important insights for making instruction-tuning more robust. language of the presentation:Japanese | |||||||
| 中谷 響 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎 | 荒牧 英治 | 大内 啓樹 | 東山 翔平 | 寺西 裕紀 |
|
title: A Context and Attribute-Aware Geocoding Model
abstract: Geocoding is a fundamental technology that associates language expressions related to specific locations, namely, location reference expressions (LRE), with geographical information, such as latitude and longitude. A representative geocoding approach links each expression to an appropriate entry with geographical information in a map database (DB). A main challenge in this approach is how to resolve the ambiguity between entries with similar names. To address this challenge, we propose a model that captures (i) contextual information of each LRE in the input sentence and (ii) attribute information of each entry in the map DB. In this presentation, we discuss the details of our model, experimental results, and future directions. language of the presentation: Japanese | |||||||
| 吉田 快 | D, 中間発表 | 自然言語処理学(ロボット対話知能) | 渡辺 太郎☆ | 吉野 幸一郎 | 河野 誠也 | ||
|
title: Towards a dialogue system with personal preference-based responses
abstract: With the recent advancement in large language models (LLMs), the responses generated by dialogue systems have become more natural. However, many LLMs focus on correctly answering questions, there is room for improvement in terms of developing communication with users. This study aims to realize a dialogue system that focuses on users' preferences, providing personalized responses tailored to each user. In this presentation, we explain the current progress and outline the future prospects of our work. language of the presentation: Japanese 発表題目: 個人的選好を考慮した応答を行う対話システムに向けて 発表概要: 近年の大規模言語モデル(LLM)の発達により、それを活用した対話システムの応答は自然なものになっている。 その一方で、多くのLLMは質問に正しく回答することに焦点が当てられており、ユーザとのコミュニケーションの展開という点では改善の余地がある。 そこで、本研究ではユーザの趣向に焦点を当て、ユーザにパーソナライズした応答を行う対話システムの実現を目標とした取り組みを行う。 本発表では現在までの取り組みと、今後の見通しについて発表を行う。 | |||||||