Publications

Work in Progress

1. The Oversmoothing Fallacy: A Misguided Narrative in GNN Research

   MoonJeong Park, Sunghyun Choi, Jaeseung Heo, Eunhyeok Park,  and Dongwoo Kim

   arXiv preprint, 2025

   Abstract Paper Bib

   Oversmoothing has been recognized as a main obstacle to building deep Graph Neural Networks (GNNs), limiting the performance. This position paper argues that the influence of oversmoothing has been overstated and advocates for a further exploration of deep GNN architectures. Given the three core operations of GNNs, aggregation, linear transformation, and non-linear activation, we show that prior studies have mistakenly confused oversmoothing with the vanishing gradient, caused by transformation and activation rather than aggregation. Our finding challenges prior beliefs about oversmoothing being unique to GNNs. Furthermore, we demonstrate that classical solutions such as skip connections and normalization enable the successful stacking of deep GNN layers without performance degradation. Our results clarify misconceptions about oversmoothing and shed new light on the potential of deep GNNs.

   @misc{park2025fallacy,
     title = {The Oversmoothing Fallacy: A Misguided Narrative in GNN Research},
     author = {Park, MoonJeong and Choi, Sunghyun and Heo, Jaeseung and Park, Eunhyeok and Kim, Dongwoo},
     booktitle = {arXiv preprint},
     year = {2025},
     eprint = {2506.04653},
     archiveprefix = {arXiv}
   }

2. Taming Gradient Oversmoothing and Expansion in Graph Neural Networks

   MoonJeong Park,  and Dongwoo Kim

   arXiv preprint, 2024

   Abstract Paper Bib

   Oversmoothing has been claimed as a primary bottleneck for multi-layered graph neural networks (GNNs). Multiple analyses have examined how and why oversmoothing occurs. However, none of the prior work addressed how optimization is performed under the oversmoothing regime. In this work, we show the presence of \textitgradient oversmoothing preventing optimization during training. We further analyze that GNNs with residual connections, a well-known solution to help gradient flow in deep architecture, introduce \textitgradient expansion, a phenomenon of the gradient explosion in diverse directions. Therefore, adding residual connections cannot be a solution for making a GNN deep. Our analysis reveals that constraining the Lipschitz bound of each layer can neutralize the gradient expansion. To this end, we provide a simple yet effective normalization method to prevent the gradient expansion. An empirical study shows that the residual GNNs with hundreds of layers can be efficiently trained with the proposed normalization without compromising performance. Additional studies show that the empirical observations corroborate our theoretical analysis.

   @misc{park2024taming,
     title = {Taming Gradient Oversmoothing and Expansion in Graph Neural Networks},
     author = {Park, MoonJeong and Kim, Dongwoo},
     booktitle = {arXiv preprint},
     year = {2024},
     eprint = {2410.04824},
     archiveprefix = {arXiv}
   }

2026

1. Transductive Generalization for GNNs via Optimal Transport

   MoonJeong Park*, Seungbeom Lee*, Kyungmin Kim, Jaeseung Heo, Seunghyuk Cho, Shouheng Li, Sangdon Park,  and Dongwoo Kim

   Conference on Neural Information Processing Systems (NeurIPS), 2026

   Abstract Paper Bib Code

   Analyzing the generalization of graph neural networks (GNNs) on node classification is challenging: message passing induces dependencies that break the i.i.d. assumption underlying standard inductive generalization theory. While distribution-free transductive theory resolves this dependency issue, existing classical bounds are either intractable or correlate poorly with empirical generalization. To fill this gap, we propose a transductive generalization bound based on optimal transport, connecting generalization to intra-class concentration and inter-class separation in the feature space. Empirically, our bound aligns strongly with the generalization gap. Building on this representation-based approach, we analyze how feature geometry evolves under repeated message passing. This explains the non-monotonic relationship between GNN depth and generalization, providing new insights into oversmoothing research. Practically, our bound serves as a geometry-aware regularizer that corrects the harmful behavior of message passing, yielding consistent performance improvements.

   @inproceedings{park2026trans,
     title = {Transductive Generalization for GNNs via Optimal Transport},
     author = {Park*, MoonJeong and Lee*, Seungbeom and Kim, Kyungmin and Heo, Jaeseung and Cho, Seunghyuk and Li, Shouheng and Park, Sangdon and Kim, Dongwoo},
     booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
     year = {2026},
     eprint = {2603.09257},
     archiveprefix = {arXiv}
   }

2. Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems

   Seunghyuk Cho, Sunghyun Choi, Jaeseung Heo, Youngbin Choi, Saemi Moon,  MoonJeong Park,  and Dongwoo Kim

   Empirical Methods in Natural Language Processing (EMNLP), 2026

   Oral

   Abstract Paper Bib Code

   Multi-agent systems (MAS) have substantially advanced autonomous software engineering (SWE), but their growing inference energy demands raise sustainability concerns. In this paper, we demonstrate that this cost is concentrated in an overlooked source: redundant output tokens generated across agents. Two empirical findings ground this claim. First, our per-token energy attribution for MAS reveals a sharp asymmetry: an output token consumes 30 to 1,000 times more energy than an input or cached token. Second, MAS inflate per-episode output because agents repeatedly re-explore overlapping repository regions. To address this inefficiency, we propose Librarian, a persistent search sub-agent that tracks repository-search history and suppresses redundant exploration actions across agents. By returning short references to file regions instead of full file excerpts, Librarian further reduces output-token volume. On SWE-Bench Verified and SWE-Bench-Live, Librarian reduces per-episode GPU energy consumption of existing multi-agent SWE systems by 11 to 30% while preserving task performance. Our code is available at https://github.com/ml-postech/Librarian.

   @inproceedings{cho2026long,
     title = {Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems},
     author = {Cho, Seunghyuk and Choi, Sunghyun and Heo, Jaeseung and Choi, Youngbin and Moon, Saemi and Park, MoonJeong and Kim, Dongwoo},
     booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
     year = {2026},
     additional_info = {Oral},
     eprint = {2605.27787},
     archiveprefix = {arXiv}
   }

3. In-Place Feedback: A New Paradigm for Guiding LLMs in Multi-Turn Reasoning

   Youngbin Choi*, Minjong Lee*, Saemi Moon, Seunghyuk Cho, Chaehyeon Chung,  MoonJeong Park,  and Dongwoo Kim

   Empirical Methods in Natural Language Processing (EMNLP), 2026

   Oral

   Abstract Paper Bib

   Large language models (LLMs) are increasingly studied in the context of multi-turn reasoning, where models iteratively refine their outputs based on user-provided feedback. Such settings are crucial for tasks that require complex reasoning, yet existing feedback paradigms often rely on issuing new messages. LLMs struggle to integrate these reliably, leading to inconsistent improvements. In this work, we introduce in-place feedback, a novel interaction paradigm in which users directly edit an LLM’s previous response, and the model conditions on this modified response to generate its revision. Empirical evaluations on diverse reasoning-intensive benchmarks reveal that in-place feedback achieves better performance than conventional multi-turn feedback while using 79.1% fewer tokens. Complementary analyses on controlled environments further demonstrate that in-place feedback resolves a core limitation of multi-turn feedback: models often fail to apply feedback precisely to erroneous parts of the response, leaving errors uncorrected and sometimes introducing new mistakes into previously correct content. These findings suggest that in-place feedback offers a more natural and effective mechanism for guiding LLMs in reasoning-intensive tasks.

   @inproceedings{choi2026inplace,
     title = {In-Place Feedback: A New Paradigm for Guiding LLMs in Multi-Turn Reasoning},
     author = {Choi*, Youngbin and Lee*, Minjong and Moon, Saemi and Cho, Seunghyuk and Chung, Chaehyeon and Park, MoonJeong and Kim, Dongwoo},
     booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
     year = {2026},
     additional_info = {Oral},
     eprint = {2510.00777},
     archiveprefix = {arXiv}
   }

4. Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

   Seokwon Yoon, Youngbin Choi, Seunghyuk Cho, Seungbeom Lee,  MoonJeong Park,  and Dongwoo Kim

   International Conference on Machine Learning (ICML), 2026

   Abstract Paper Bib Code

   Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest as real-world applications often involve multiple, conflicting objectives. However, existing approaches require joint training for each combination of objectives, meaning that any change in the objective set necessitates retraining from scratch. We propose a framework that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without fine-tuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex nonlinear operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization, and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule generation tasks demonstrate that our approach achieves performance comparable to baselines. The code is available at https://github.com/ml-postech/gflownet-composition.

   @inproceedings{yoon2026gflow,
     title = {Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation},
     author = {Yoon, Seokwon and Choi, Youngbin and Cho, Seunghyuk and Lee, Seungbeom and Park, MoonJeong and Kim, Dongwoo},
     booktitle = {International Conference on Machine Learning (ICML)},
     year = {2026},
     eprint = {2602.21565},
     archiveprefix = {arXiv}
   }

5. Posterior Label Smoothing for Node Classification

   Jaeseung Heo,  MoonJeong Park,  and Dongwoo Kim

   AAAI Conference on Artificial Intelligence (AAAI), 2026

   Oral

   Abstract Paper Bib Code

   Soft labels can improve the generalization of a neural network classifier in many domains, such as image classification. Despite its success, the current literature has overlooked the efficiency of label smoothing in node classification with graph-structured data. In this work, we propose a simple yet effective label smoothing for the transductive node classification task. We design the soft label to encapsulate the local context of the target node through the neighborhood label distribution. We apply the smoothing method for seven baseline models to show its effectiveness. The label smoothing methods improve the classification accuracy in 10 node classification datasets in most cases. In the following analysis, we find that incorporating global label statistics in posterior computation is the key to the success of label smoothing. Further investigation reveals that the soft labels mitigate overfitting during training, leading to better generalization performance.

     title = {Posterior Label Smoothing for Node Classification},
     author = {Heo, Jaeseung and Park, MoonJeong and Kim, Dongwoo},
     booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
     year = {2026},
     additional_info = {Oral},
     eprint = {2406.00410},
     archiveprefix = {arXiv}
   }

2025

1. Influence Functions for Edge Edits in Non-Convex Graph Neural Networks

   Jaeseung Heo, Kyeongheung Yun, Seokwon Yoon,  MoonJeong Park, Jungseul Ok,  and Dongwoo Kim

   Conference on Neural Information Processing Systems (NeurIPS), 2025

   Abstract Paper Bib Code

   Understanding how individual edges influence the behavior of graph neural networks (GNNs) is essential for improving their interpretability and robustness. Graph influence functions have emerged as promising tools to efficiently estimate the effects of edge deletions without retraining. However, existing influence prediction methods rely on strict convexity assumptions, exclusively consider the influence of edge deletions while disregarding edge insertions, and fail to capture changes in message propagation caused by these modifications. In this work, we propose a proximal Bregman response function specifically tailored for GNNs, relaxing the convexity requirement and enabling accurate influence prediction for standard neural network architectures. Furthermore, our method explicitly accounts for message propagation effects and extends influence prediction to both edge deletions and insertions in a principled way. Experiments with real-world datasets demonstrate accurate influence predictions for different characteristics of GNNs. We further demonstrate that the influence function is versatile in applications such as graph rewiring and adversarial attacks.

   @inproceedings{heo2025influence,
     title = {Influence Functions for Edge Edits in Non-Convex Graph Neural Networks},
     author = {Heo, Jaeseung and Yun, Kyeongheung and Yoon, Seokwon and Park, MoonJeong and Ok, Jungseul and Kim, Dongwoo},
     booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
     year = {2025},
     eprint = {2506.04694},
     archiveprefix = {arXiv}
   }

2. CoPL: Collaborative Preference Learning for Personalizing LLMs

   Youngbin Choi, Seunghyuk Cho, Minjong Lee,  MoonJeong Park, Yesong Ko, Jungseul Ok,  and Dongwoo Kim

   Empirical Methods in Natural Language Processing (EMNLP), 2025

   Abstract Paper Bib Code

   Personalizing large language models (LLMs) is important for aligning outputs with diverse user preferences, yet existing methods struggle with flexibility and generalization. We propose CoPL (Collaborative Preference Learning), a graph-based collaborative filtering framework that models user-response relationships to enhance preference estimation, particularly in sparse annotation settings. By integrating a mixture of LoRA experts, CoPL efficiently fine-tunes LLMs while dynamically balancing shared and user-specific preferences. Additionally, an optimization-free adaptation strategy enables generalization to unseen users without fine-tuning. Experiments on UltraFeedback-P demonstrate that CoPL outperforms existing personalized reward models, effectively capturing both common and controversial preferences, making it a scalable solution for personalized LLM alignment.

   @inproceedings{choi2025copl,
     title = {CoPL: Collaborative Preference Learning for Personalizing LLMs},
     author = {Choi, Youngbin and Cho, Seunghyuk and Lee, Minjong and Park, MoonJeong and Ko, Yesong and Ok, Jungseul and Kim, Dongwoo},
     booktitle = {Empirical Methods in Natural Language Processing (EMNLP)},
     year = {2025},
     eprint = {2503.01658},
     archiveprefix = {arXiv}
   }

2024

1. Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs

   MoonJeong Park, Jaeseung Heo,  and Dongwoo Kim

   International Conference on Machine Learning (ICML), 2024

   Excellent Paper Award at BK21 Paper Award

   Abstract Paper Bib Video Code

   Graph Neural Network (GNN) resembles the diffusion process, leading to the over-smoothing of learned representations when stacking many layers. Hence, the reverse process of message passing can produce the distinguishable node representations by inverting the forward message propagation. The distinguishable representations can help us to better classify neighboring nodes with different labels, such as in heterophilic graphs. In this work, we apply the design principle of the reverse process to the three variants of the GNNs. Through the experiments on heterophilic graph data, where adjacent nodes need to have different representations for successful classification, we show that the reverse process significantly improves the prediction performance in many cases. Additional analysis reveals that the reverse mechanism can mitigate the over-smoothing over hundreds of layers. Our code is available at https://github.com/ml-postech/reverse-gnn.

   @inproceedings{park2024mitigating,
     title = {Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs},
     author = {Park, MoonJeong and Heo, Jaeseung and Kim, Dongwoo},
     booktitle = {International Conference on Machine Learning (ICML)},
     year = {2024},
     video = {https://youtu.be/JKN68PDYSSA?si=usk-U8KvA5oX2Mh2},
     additional_info = {Excellent Paper Award at BK21 Paper Award},
     eprint = {2403.10543},
     archiveprefix = {arXiv}
   }

2023

1. SpReME: Sparse Regression for Multi-Environment Dynamic Systems

   MoonJeong Park*, Youngbin Choi*,  and Dongwoo Kim

   AAAI Conference on Artificial Intelligence, Workshop on When Machine Learning meets Dynamical Systems: Theory and Applications (AAAIw), 2023

   Abstract Paper Bib Code

   This paper introduces a new weakly supervised learning approach for instance segmentation using extreme points, i.e., the topmost, leftmost, bottommost, and rightmost points of an object. Although these points are readily available in the modern bounding box annotation process and offer strong clues for precise segmentation, they have received less attention in the literature. Motivated by this, our study explores extreme point supervised instance segmentation to further enhance performance at the same annotation cost with box-supervised methods. Our work considers extreme points as a part of the true instance mask and propagates them to identify potential foreground and background points, which are all together used for training a pseudo label generator. Then pseudo labels given by the generator are in turn used for supervised learning of our final model. Our model generates high-quality masks, particularly when the target object is separated into multiple parts, where previous box-supervised methods often fail. On three public benchmarks, our method significantly outperforms existing box-supervised methods, further narrowing the gap with its fully supervised counterpart.

   @inproceedings{park2023spreme,
     title = {SpReME: Sparse Regression for Multi-Environment Dynamic Systems},
     author = {Park*, MoonJeong and Choi*, Youngbin and Kim, Dongwoo},
     booktitle = {AAAI Conference on Artificial Intelligence, Workshop on When Machine Learning meets Dynamical Systems: Theory and Applications (AAAIw)},
     year = {2023},
     equal_contrib = {true},
     eprint = {2302.05942},
     archiveprefix = {arXiv}
   }

2022

1. MetaSSD: Meta-Learned Self-Supervised Detection

   MoonJeong Park, Jungseul Ok, Yo-Seb Jeon,  and Dongwoo Kim

   IEEE International Symposium on Information Theory (ISIT), 2022

   Abstract Paper Bib Code

   Deep learning-based symbol detector gains increasing attention due to the simple algorithm design than the traditional model-based algorithms such as Viterbi and BCJR. The supervised learning framework is often employed to train a model, where true symbols are necessary. There are two major limitations in the supervised approaches: a) a model needs to be retrained from scratch when new train symbols come to adapt to a new channel status, and b) the length of the training symbols needs to be longer than a certain threshold to make the model generalize well on unseen symbols. To overcome these challenges, we propose a meta-learning-based self-supervised symbol detector named MetaSSD. Our contribution is two-fold: a) meta-learning helps the model adapt to a new channel environment based on experience with various meta-training environments, and b) self-supervised learning helps the model to use relatively less supervision than the previously suggested learning-based detectors. In experiments, MetaSSD outperforms OFDM-MMSE with noisy channel information and shows comparable results with BCJR. Further ablation studies show the necessity of each component in our framework.

   @inproceedings{park2022meta,
     title = {MetaSSD: Meta-Learned Self-Supervised Detection},
     author = {Park, MoonJeong and Ok, Jungseul and Jeon, Yo-Seb and Kim, Dongwoo},
     booktitle = {IEEE International Symposium on Information Theory (ISIT)},
     year = {2022},
     eprint = {2205.15271},
     archiveprefix = {arXiv}
   }

1. Large-scale tucker Tensor factorization for sparse and accurate decomposition

   Jun-Gi Jang*,  MoonJeong Park*, Jongwuk Lee,  and Lee Sael

   The Journal of Supercomputing, 2022

   Extended version of the conference paper "VeST: Very Sparse Tucker Factorization of Large-Scale Tensors"

   Abstract Bib Paper

   How can we generate sparse tensor decomposition results for better interpretability? Typical tensor decomposition results are dense. Dense results require additional postprocessing for data interpretation, especially when the data are large. Thus, we present a large-scale Tucker factorization method for sparse and accurate tensor decomposition, which we call the Very Sparse Tucker factorization (VeST) method. The proposed VeST outputs highly sparse decomposition results from a large-scale partially observable tensor data. The approach starts by decomposing the input tensor data, then iteratively determining unimportant elements, removing them, and updating the remaining elements until a terminal state is reached. We define ‘responsibility’ of each element on the reconstruction error to determine unimportant elements in the decomposition results. The decomposition results are updated iteratively in parallel using carefully constructed coordinate descent rules for scalable computation. Furthermore, the suggested method automatically looks for the optimal sparsity ratio, resulting in a balanced sparsity-accuracy trade-off. Extensive experiments using real-world datasets showed that our method produces more accurate results than that of the competitors. Experiments further showed that the proposed method is scalable in terms of the input dimensionality, the number of observable entries, and the thread count.

   @article{park2022vest2,
     title = {Large-scale tucker Tensor factorization for sparse and accurate decomposition},
     author = {Jang*, Jun-Gi and Park*, MoonJeong and Lee, Jongwuk and Sael, Lee},
     journal = {The Journal of Supercomputing},
     year = {2022},
     equal_contrib = {true},
     additional_info = {Extended version of the conference paper "VeST: Very Sparse Tucker Factorization of Large-Scale Tensors"},
   }

2021

1. VeST: Very Sparse Tucker Factorization of Large-Scale Tensors

   MoonJeong Park*, Jun-Gi Jang*,  and Lee Sael

   IEEE International Conference on Big Data and Smart Computing (BigComp), 2021

   Best Paper Award, 1st Place

   Abstract Paper Bib Code

   Given a large tensor, how can we decompose it to sparse core tensor and factor matrices without reducing the accuracy? Existing approaches either output dense results or have scalability issues. In this paper, we propose VEST, a tensor factorization method for large partially observable data to output a very sparse core tensor and factor matrices. VEST performs initial decomposition and iteratively determines unimportant entries in the decomposition results, removes the unimportant entries, and updates the remaining entries. To determine unimportant entries of factor matrices and core tensor, we define and use the entry-wise ‘responsibility’ of the current decomposition. For scalable computation, the entries are updated iteratively using a carefully derived coordinate descent rule in parallel. Also, VEST automatically searches for the best sparsity ratio that results in a balanced trade-off between sparsity and accuracy. Extensive experiments show that our method VEST produces more accurate results compared to the best performing competitors for all tested real-life datasets.

   @inproceedings{park2021vest,
     title = {VeST: Very Sparse Tucker Factorization of Large-Scale Tensors},
     author = {Park*, MoonJeong and Jang*, Jun-Gi and Sael, Lee},
     booktitle = {IEEE International Conference on Big Data and Smart Computing (BigComp)},
     year = {2021},
     equal_contrib = {true},
     additional_info = {Best Paper Award, 1st Place},
     eprint = {1904.02603},
     archiveprefix = {arXiv}
   }