| ASADI NASTARAN | D, 中間発表 | コンピューティング・アーキテクチャ | 中島 康彦, | 林 優一, | 張 任遠, | KAN Yirong, | PHAM HOAI LUAN, | Le Vu Trung Duong |
|
title: Lightweight Transformer via Optimizing Attention Mechanism for Language Processing on Edge Devices
abstract: Neural machine translation (NMT) models, especially Transformers, have high translation quality but suffer from scalability issues due to the quadratic complexity of the attention mechanism. This drawback leads to high memory usage and computational overhead, making large-scale translation inefficient. In addition, maintaining translation quality while reducing computational costs remains a key challenge. To alleviate these issues, this work presents dynamic sparse masks for Transformers, which fine-grained control sparsity based on percentile thresholds. Our method greatly reduces computation amount in Transformer inference process while preserving translation accuracy by selectively retaining the most significant attention scores. Furthermore, learning the curriculum is incorporated to improve model accuracy by gradually structuring the training process. Experiments are conducted on the WMT2014 English-French dataset to verify the effectiveness of our model on NMT tasks. Compared to the standard Transformer model, our model achieves an improvement of 19.38 in the Bilingual Evaluation Understudy (BLEU) score and reduces 31.4% multiply-accumulate (MAC) operations in Transformer inference process, which is crucial to reducing power consumption in hardware. language of the presentation: English | ||||||||
| XU CHENG | D, 中間発表 | コンピューティング・アーキテクチャ | 中島 康彦, | 林 優一, | 張 任遠, | KAN Yirong, | PHAM HOAI LUAN, | Le Vu Trung Duong |
|
title:An FPGA Accelerator for Vision Transformer with Quantization
and LUT-Based Operations
abstract: An Field-Programmable Gate Array (FPGA) accelerator was proposed for Vision Transformers (ViTs) with quantization and look-up-table (LUT) based operations. First, two improved quantization methods are proposed, achieving comparable performance at lower bit-widths. Furthermore, linear and nonlinear units’ designs are proposed to support diverse operations in ViTs models. Finally, the LUT-based acceler- ator design is implemented and evaluated. Experimental results on the ImageNet dataset demonstrate that our proposed quantization method achieves the accuracy of 80.74% at 2-bit width, outperforming state-of-the-art Vision Transformer quantization methods by 0.1% to 0.5%. The performance of the proposed FPGA accelerator demonstrates a higher energy efficiency, achieving a peak energy efficiency of 7.06 FPS/W and 246 GOPS/W. language of the presentation: English | ||||||||
| TSERENKHUU MANLAIBAATAR | D, 中間発表 | サイバーレジリエンス構成学 | 門林 雄基, | 笠原 正治, | 林 優一, | 妙中 雄三 | ||
|
title: Intrusion Detection System framework for SDN-based IoT networks: A comparative study of eXplainable Artificial Intelligence-based feature selection techniques
abstract: The proliferation of Internet of Things (IoT) applications impacts every aspect of life. However, it also brings significant cybersecurity threats. To mitigate these threats, artificial intelligence (AI) methods, especially deep learning (DL) techniques, have been implemented in intrusion detection systems (IDS). The design of deep learning models and the quality of datasets are two key factors in creating effective IDSs. Randomly selecting hyperparameters and using datasets with irrelevant features can negatively affect model performance and computational complexity. This study proposes an IDS framework to detect various cyberattacks in SDN-based IoT networks utilizing three deep learning algorithms that incorporate hyperparameter tuning and the feature selection process based on explainable artificial intelligence (XAI) techniques. We conducted an extensive set of experiments using the subsets of features derived by XAI-based feature selection techniques and compared their performance against each other, the baseline, and state-of-the-art models. The experimental results reveal that Shapley Additive Explanations and Random Forest feature importance are the reliable feature selection techniques, as they yield consistent results across all deep learning models and different feature subsets. Furthermore, the convolutional neural network model performs best, achieving accuracies of 99.9% and 98.1% for multiclass classification in the InSDN and the X-IIoTID datasets, respectively. language of the presentation: English | ||||||||