| 帖佐 宗浩 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, | 荒牧 英治, | 上垣外 英剛, | 坂井 優介 |
|
title:Early Word Segmentation in Neural Language Models
abstract:Neural language models (NLMs) are increasingly compared with human language development, but most such comparisons assume a pre-specified word or subword vocabulary. This assumption abstracts away early word segmentation, an important aspect of language acquisition. Usage-based theories suggest that children may initially treat multiword expressions, such as \textit{holophrases}, as unanalyzed communicative units before developing more adult-like lexical boundaries. In this work, we explore the early segmentation dynamics of character-level NLMs trained without pre-specified lexical units. We train models on an English corpus designed to approximate infant linguistic input, remove explicit boundary cues, and infer hypothesized segmentation boundaries from the models' probabilistic uncertainty throughout training. The models rapidly acquire boundary-like behavior, but this segmentation is not organized around a holophrase-like decomposition of longer multiword chunks into smaller units. Comparisons with n-gram baselines further suggest that this boundary formation is largely frequency-driven, although not reducible to simple n-gram statistics. These results reveal an early developmental regime in NLMs that is obscured when lexical units are assumed in advance. language of the presentation:Japanese 発表題目:ニューラル言語モデルにおける学習初期の分節化 発表概要:ニューラル言語モデル（NLM）は、人間の言語発達との比較対象としてますます注目されているが、その多くは、事前に定義された単語またはサブワード語彙の存在を前提としている。この前提は、言語獲得における重要な側面である初期の語分割過程を捨象している。 用法基盤理論では、子どもは、成人に近い語彙境界を獲得する以前に、ホロフレーズのような複数語表現を、内部構造を分析されていない一まとまりの伝達単位として扱う可能性が示唆されている。 本研究では、事前に定義された語彙単位を用いずに学習された文字レベルNLMを対象として、その初期の分割ダイナミクスを検討する。具体的には、乳児が受ける言語入力を近似するよう設計された英語コーパスを用いてモデルを学習し、明示的な境界手掛かりを除去した上で、学習過程を通じて観測されるモデルの確率的不確実性から、モデルが想定している分割境界を推定する。 その結果、モデルは境界に類似した振る舞いを速やかに獲得する一方で、この分割は、長い複数語チャンクをより小さな単位へと分解するホロフレーズ的な過程として組織化されてはいないことが明らかとなった。さらに、n-gramベースラインとの比較から、この境界形成は主として頻度に駆動されるものの、単純なn-gram統計のみに還元できるものではないことが示唆された。 これらの結果は、語彙単位をあらかじめ仮定することで見えにくくなる、NLMにおける初期発達段階の存在を明らかにするものである。 | ||||||
| 平川 稜真 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, | 荒牧 英治, | 上垣外 英剛, | 坂井 優介 |
|
title: Analyzing Prompt-Induced Bias and Their Internal Representations in LLMs for Reliable Preference Estimation
abstract: Reliable preference estimation is becoming increasingly important in applications such as LLM-as-a-Judge and preference optimization. Among the prompt-induced factors affecting its reliability, Position Bias—where model decisions are influenced by the presentation order of candidate answers—has emerged as a critical challenge. However, it remains unclear how this bias is formed inside Transformer models and how it relates to model reliability. In this work, we investigate Position Bias as a representative prompt-induced bias by analyzing the Residual Stream, Layer Delta, Multi-Head Attention (MHA), and MLP modules throughout Transformer models. Our analysis reveals that Position Bias is not confined to the final output but is broadly distributed across internal Transformer modules. We further show that reducing internal Position Bias is accompanied by improvements in both Accuracy and Calibration. Based on these findings, we propose a layer-wise post-hoc internal intervention method for Transformer modules and demonstrate that suppressing internal Position Bias can improve both model performance and reliability. These results provide new insights into the relationship between prompt-induced biases, internal Transformer representations, and reliable preference estimation. language of the presentation: Japanese 発表題目: 信頼性の高い選好推定に向けたPrompt-induced BiasとTransformer内部表現の分析 発表概要: 人間の選好推定（Human Preference Estimation）は、LLM-as-a-JudgeやPreference Optimizationなどの基盤技術であり、その信頼性は生成AIシステムの性能を左右する重要な要素である。その中でも、回答の提示順序によって判断が変化するPosition Biasは、推定結果の信頼性を損なうPrompt-induced Biasとして知られている。しかし、このPosition BiasがTransformer内部でどのように形成され、モデルの信頼性とどのように関係しているかは十分に明らかになっていない。 本研究では、Transformer内部のResidual Stream、Layer Delta、Multi-Head Attention（MHA）、MLPに着目し、Position Biasの形成過程および内部Position Biasとモデルの信頼性との関係を解析した。 解析の結果、Position Biasは最終出力だけでなくTransformer内部の各モジュールに広く存在することを確認し、さらに内部Position Biasを低減することで、AccuracyおよびCalibrationが改善することを確認した。 さらに、これらの解析結果に基づき、Transformer内部への層ごとのpost-hoc介入手法を提案し、内部Position Biasの低減がモデルの性能および信頼性の向上につながることを示した。 本発表では、Transformer内部におけるPosition Biasの形成と信頼性との関係を報告するとともに、解析結果に基づく内部介入の有効性について議論する。 | ||||||
| 藤田 剛 | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, | 荒牧 英治, | 上垣外 英剛, | 坂井 優介 |
|
title: Automatic Identification of Delegation Relationships in Japanese Legal Texts
abstract: The Japanese legal system has a hierarchical structure, and legal texts frequently employ a technique in which a higher-level law defers the stipulation of more detailed provisions to a lower-level law that has lesser formal authority, which is called delegation relationships. When interpreting legal texts that contain delegation relationships, readers must repeatedly refer to the lower-level laws that set out the detailed provisions, which imposes a significant burden. Therefore, a mechanism that supports readers by enabling immediate identification of delegation relationships is needed. However, manually annotating delegation relationships requires extensive legal expertise, reading a large number of laws and articles that may serve as delegation sources or targets, and continuously handling newly enacted legislation, which makes the task highly challenging. To address this issue, this study attempts to reduce the burden of manual annotation by automatically identifying delegation relationships. This paper constructs a two-stage pipeline system that automatically extracts candidate delegation relationships by (1) extracting keywords that indicate delegation, and (2) identifying the corresponding target provisions for each keyword. Specifically, we develop a model that extracts delegation keywords as a named entity recognition task, and another model that identifies the delegation targets as an entity disambiguation task. Experimental results show that the system achieves a level of performance sufficient to support manual annotation. language of the presentation: Japanese | ||||||
| SHUM KIT MAY | M, 2回目発表 | 自然言語処理学 | 渡辺 太郎, | 荒牧 英治, | 上垣外 英剛, | 坂井 優介 |
|
title: Leveraging Big Five Personality Traits for Suicide Severity Level Detection
abstract: Suicide risk assessment from text is typically framed as a direct classification task, yet psychological research has long linked suicide risk to personality structure. This project focuses specifically on Neuroticism, one of the Big Five traits, and tests whether explicitly injecting its signal into a Large Language Model can causally improve the model's ability to classify suicide-risk severity, beyond what the model already infers from text alone. We augment a frozen, pretrained LLM with lightweight trainable components that inject Neuroticism signals derived from the model's own internal representations, then compare performance with and without this injection to isolate the causal contribution of explicit personality information from the implicit cues the model already captures. language of the presentation: English | ||||||
| 植村 奏 | M, 2回目発表 | 数理情報学 | 池田 和司, | 清川 清, | 久保 孝富, | 日永田 智絵 |
|
title: Computational Modeling of Emotion Concept Formation and Sharing in Response to Auditory Stimuli
abstract: Emotion concepts have been proposed as a framework for explaining our emotional experiences. They are thought to be constructed through the integration of interoceptive and exteroceptive information with prior experiences and language and to be updated and shared through social interactions. However, little is known about how emotion concepts are formed and shared in response to auditory stimuli. To investigate the formation and sharing of emotion concepts in response to auditory stimuli, we conducted experiments with human participants and analyzed the collected data using computational models. Participants listened to auditory stimuli and reported impression words and subjective emotion ratings while physiological signals were continuously recorded. The experiment also included a text-mediated interaction condition. The collected data were used to develop a multimodal probabilistic generative model that integrates physiological signals, auditory features, and linguistic information to model emotion concept formation and to analyze the effects of interaction on subjective emotion ratings and latent affective representations. The proposed model achieved agreement between inferred latent topics and subjective emotion ratings above the chance level, and the integration of linguistic information significantly improved the agreement. Furthermore, text-mediated interaction significantly reduced interpersonal differences in valence ratings and increased the Rand index of latent affective topics. These findings provide a foundation for computational modeling of emotion concept formation and sharing in response to auditory stimuli. language of the presentation: Japanese | ||||||
| 尾﨑 翠 | M, 2回目発表 | 数理情報学 | 池田 和司, | 清川 清, | 久保 孝富, | LI YUZHE |
|
title: Gait Disorder Screening for Animals Using DeepLabCut and CEBRA
abstract: Gait disorders in cats can significantly impair their quality of life (QOL), and early detection is important for improving prognosis. However, early-stage gait abnormalities are often difficult for owners to recognize, and natural gait is hard to capture in clinical settings. In addition, conventional video-based analysis requires extensive pose annotation, and its high cost has hindered practical implementation. In this study, we aimed to develop an annotation-free screening approach for detecting cat gait abnormalities by combining DeepLabCut (DLC), a markerless pose estimation method, with CEBRA, a self-supervised learning framework. Using a pretrained DLC model for quadrupeds, we estimated poses from video recordings and obtained pose sequences for each individual. These sequences were then input into CEBRA, which embedded them into a three-dimensional latent space using individual identity and temporal information as auxiliary variables to visualize latent structure in gait patterns. As a result, DLC stably estimated major body points across diverse cases, including both normal and abnormal gait, without the need for manual annotation. In the CEBRA latent space, normal and abnormal gait patterns occupied distinct regions, although some overlap remained near the boundaries. These findings suggest that cat gait features can be extracted without pose annotation and that the latent structure may enable discrimination between normal and abnormal gait. language of the presentation: Japanese 発表題目: DeepLabCutとCEBRAを用いた動物の歩行障害スクリーニング 発表概要: ネコの歩行障害はQOLを損なうため、早期発見は予後改善に重要である。 しかし軽度段階では発見が難しく、診察室では自然な歩行が捉えにくい。また、従来の動画解析手法は膨大な姿勢アノテーションを必要とし、そのコストの高さから実用化が進んでいない。本研究では、マーカーレス姿勢推定手法の DeepLabCut（DLC）と自己教師あり学習手法の CEBRA を組み合わせ、姿勢アノテーションを用いないネコの歩行異常スクリーニング手法の構築を目指した。まず、DLCの四足動物用事前学習済みモデルを用いて動画から姿勢を推定し、姿勢系列を取得した。次に、この姿勢系列をCEBRAに入力し、個体IDと時間情報を補助変数として3次元潜在空間に埋め込むことで、歩行パターンの潜在的構造を可視化した。その結果、DLCはアノテーションなしでも正常例・異常例を含む多様な症例において主要部位を安定して推定し、CEBRAの潜在空間では正常歩行と異常歩行が異なる領域に偏って分布することが確認された。これらの結果から、姿勢アノテーションを行わずにネコの歩行特徴を抽出し、歩行の正常・異常を区別できる可能性が示された。 | ||||||
| 長尾 宙昭 | M, 2回目発表 | 数理情報学 | 池田 和司, | 清川 清, | 田中 沙織, | 久保 孝富 |
|
title:Exploring the Conditions for the Perception of Extremely Close Sound Sources through Measurements of Near-Field Head-Related Transfer Functions
abstract:Head-related transfer functions (HRTFs) describe the frequency characteristics introduced by the shape of the human head and pinnae as transfer functions. Extensive measurements of HRTFs have been conducted in the far and intermediate sound field, typically at source distances of 1 m or greater, and their results have been widely applied in fields such as gaming and virtual reality. In contrast, studies measuring HRTFs in the extremely near sound field, where the sound source is only several tens of centimeters from the listener, are scarce. This is primarily because even slight misalignment between the sound source and the measurement dummy head can introduce substantial measurement errors, making accurate measurements difficult with conventional acoustic measurement systems. In this study, we construct a high-precision measurement system using an optical rail, which is commonly employed in precision physical experiments, to achieve more accurate positioning of both the sound source and the dummy head. Based on the measured near-field HRTFs, we investigate the acoustic conditions that give rise to the perception that "a sound is extremely close." We also aim to contribute to a better understanding of the auditory and psychological mechanisms underlying phenomena characteristic of near-field listening, such as those experienced in ASMR.* language of the presentation:Japanese 発表題目:超近距離頭部伝達関数の測定による「音がとても近い」と感じる条件の探求 発表概要: 頭部伝達関数(HRTF)は、ヒトの耳介や頭部の形状によって生じる周波数特性変化を伝達関数として表現するものである。これまで、音源から頭部までの距離が１m以上の中・遠距離音場におけるHRTF測定は盛んに行われ、その結果はすでにゲームシーンやバーチャルリアリティ分野等に広く応用されている。一方で、距離数十cm程度の超近距離音場でのHRTF測定を行った研究はほぼ存在しない。音源や測定用ダミーヘッドの配置のわずかなずれによって測定結果に大きな誤差が生じることから、通常の音響実験系での測定は困難だという背景があるためである。そこで本研究では、精密な物理実験を行う際に用いる光学レールを実験系構築に用い、音源とダミーヘッドをより正確に配置した条件下での測定を試みる。また、この結果をもとに「音がとても近い」と知覚される条件を考察し、ASMR等の近距離音場特有の聴覚・心理的な現象の発生原理の解明への寄与も目指す。 | ||||||
| 馬場 珠希 | M, 2回目発表 | 脳・行動モデリング | 田中 沙織, | 清川 清, | CAI LIN, | 荻島 大凱 |
|
title: Analysis of Eye Movement Characteristics of Individuals with ADHD During Gameplay
abstract: Attention Deficit Hyperactivity Disorder (ADHD) is known to be associated with characteristic eye movement patterns, particularly in saccadic behavior. This study analyzes eye-tracking data collected during Flappy Bird gameplay to investigate eye movement characteristics related to ADHD. Initial analyses of the collected data revealed trends that could not be fully explained by existing assumptions. Based on these findings and previous studies on reward-related eye movement learning, a new hypothesis was formulated that reward-related learning may influence eye movements during gameplay. Additional analyses are currently being conducted to examine whether ADHD-related differences in eye movement behavior can be identified. The current research aims to clarify the relationship between ADHD characteristics, eye movement behavior, and game performance, providing a basis for future ADHD assessment using gameplay and eye-tracking data. language of the presentation: Japanese 発表題目：ゲームプレイ中のADHD者の眼球運動特性の解析 発表概要： 注意欠如・多動症（ADHD）は、特にサッカード運動において特徴的な眼球運動パターンを示すことが知られている。 本研究では、Flappy Birdプレイ中に取得したアイトラッキングデータを解析し、ADHDに関連する眼球運動特性を調査することを目的とする。 収集したデータの初期解析を行った結果、従来の仮定だけでは十分に説明できない傾向が確認された。 これらの結果と、報酬が眼球運動学習を促進することを示した先行研究に基づき、報酬による学習がゲームプレイ中の眼球運動にも影響を及ぼす可能性があるという新たな仮説を立てた。 現在は、この仮説に基づき、ADHD者に特徴的な眼球運動がゲームプレイ中にも観察されるかを検証するため、追加解析を進めている。 最終的には、ADHD特性、眼球運動、およびゲームパフォーマンスの関係を明らかにし、ゲームプレイ中のアイトラッキングデータを用いたADHD特性推定の可能性を検討することを目指す。 | ||||||