Donghwan Rho

As the Large Language Models (LLMs) such as ChatGPT work successfully and are offered as services, language model has been at the center of the latest developments in AI. LLMs are trained on an internet-scale data, demonstrating outstanding knowledge and fluent conversational capacity. Besides, LLMs can provide customized answers to each person’s questions. However, personal data often have to be sent to the LLM provider’s server to receive these personalized answers. Therefore, people who do not want to share their personal information, companies with proprietary technology, and institutions such as banks holding sensitive data are hesitant to use LLMs due to privacy concerns. Because of this privacy issue, privacy-preserving machine learning (PPML) has attracted attention along with the development of LLMs. Homomorphic Encryption (HE) offers a promising solution to PPML. It supports operations in an encrypted data. If we use it, data are sent to a server in encrypted form from the beginning. Therefore, even if the data is leaked from the server, private information is protected. Among various HE schemes, we use CKKS (Cheon-Kim-Kim-Song), which supports real number operations, so it is appropriate for implementing machine learning models. CKKS is based on RLWE (Ring Learning with Errors), and since RLWE is equivalent to an NP-hard problem, the security of CKKS is guaranteed. Even if CKKS has many strengths, it is prohibitively hard to use it practically because of the computational overhead. LLMs are based on the Transformer, which involves many non-polynomial operations such as Softmax; however CKKS only supports addition and multiplication. In this thesis, we study the practical implementation of LLMs under HE by addressing these problems, building on the author’s earlier work [99, 100]. In Chapter 3, we replace Softmax with a Gaussian kernel and reduce matrix multiplications between ciphertexts to reduce computational overhead. In Chapter 4, we propose a novel method for stable text generation when probabilistic sampling is performed under CKKS, which has not yet been developed.

Reinforcement learning with verifiable rewards (RLVR) is a promising approach for training large language models (LLMs) with stronger reasoning abilities. It has also been applied to a variety of logic puzzles. In this work, we study the Korean word-chain game using RLVR. We show that rule-derived rewards can naturally conflict, and demonstrate through experiments that a curriculum-learning scheme mitigates these conflicts. Our findings motivate further studies of puzzle tasks in diverse languages.

As users increasingly interact with large language models (LLMs) using private information, secure and encrypted communication becomes essential. Homomorphic encryption (HE) provides a principled solution by enabling computation directly on encrypted data. Although prior work has explored aspects of running LLMs under HE, the challenge of text generation, particularly next-token prediction, has received limited attention and remains a key obstacle to practical encrypted interaction. In this work, we propose a TSP-based token reordering strategy to address the difficulties of encrypted text generation, together with a post-processing step that further reduces approximation error. Theoretical analysis and experimental results demonstrate that our method prevents collapse, improves coherence in generated text, and preserves data privacy throughout. Overall, our contributions advance the feasibility of practical and privacy-preserving LLM inference.

Large language models (LLMs) offer personalized responses based on user interactions, but this use case raises serious privacy concerns. Homomorphic encryption (HE) is a cryptographic protocol supporting arithmetic computations in encrypted states and provides a potential solution for privacy-preserving machine learning (PPML). However, the computational intensity of transformers poses challenges for applying HE to LLMs. In this work, we propose a modified HE-friendly transformer architecture with an emphasis on inference following personalized (private) fine-tuning. Utilizing LoRA fine-tuning and Gaussian kernels, we achieve significant computational speedups—6.94x for fine-tuning and 2.3x for inference—while maintaining performance comparable to plaintext models. Our findings provide a viable proof of concept for offering privacy-preserving LLM services in areas where data protection is crucial.