Po-Yi Lu (呂栢頤)

Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs

Oscar Chew*, Po-Yi Lu*, Yi-Jie Cheng, Kunal Jain, Hsuan-Tien Lin, Zhengzhong Tu, and Kuan-Hao Huang
* Equal contribution
arXiv preprint, 2026.

We introduce GEMMU, a benchmark for multimodal machine unlearning that evaluates real-world pretrained knowledge using held-out forget and retain sets and per-method hyperparameter search. Our re-evaluation reveals generalization gaps and changes in method rankings overlooked by existing evaluations.

Paper

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

Zihan Xue, Po-Yi Lu, Serhii Honcharenko, Zih-Ching Chen, Hsuan-Tien Lin, Nanyun Peng, I-Hung Hsu, and Kuan-Hao Huang
arXiv preprint, 2026.

We introduce a benchmark of paired text and image tasks for controlled comparisons of multimodal in-context learning, and diagnose the modality gap through interventions on visual access, task framing, and reasoning.

Paper  /  Code & data

PEPPER: Perception-Guided Perturbation for Robust Backdoor Defense in Text-to-Image Diffusion Models

Oscar Chew*, Po-Yi Lu*, Jayden Lin, Kuan-Hao Huang, and Hsuan-Tien Lin
* Equal contribution
Findings of the Association for Computational Linguistics: EMNLP 2026.

We defend text-to-image diffusion models against backdoor attacks by rewriting prompts into semantically distant but visually similar descriptions, without additional training or access to model weights.

Paper  /  arXiv  /  Code

Improving the Efficacy of Test-Time Steering in Masked Diffusion Models with Parallel Tempering

Po-Yi Lu, Hsuan-Tien Lin, and Shih-Hsin Wang
In the Generative and Agentic AI for Biology (GenBio) Workshop @ ICML, 2026. Spotlight

We propose Parallel Tempering for Masked Diffusion Models (PT-MDM), a training-free test-time steering method that couples the reward temperature to the remasking fraction to balance global exploration and local exploitation, improving inverse protein folding and regulatory DNA design over training-free baselines.

Tackling Dimensional Collapse toward Comprehensive Universal Domain Adaptation

Hung-Chieh Fang, Po-Yi Lu, Hsuan-Tien Lin
In Proceedings of the 42nd International Conference on Machine Learning 2025

We investigate the self-supervised loss to tackle dimensional collapse in the target representation and step toward more comprehensive Universal Domain Adaptation.

The Unexplored Potential of Vision-Language Models for Generating Large-Scale Complementary-Label Learning Data

Tan-Ha Mai, Nai-Xuan Ye, Yu-Wei Kuan, Po-Yi Lu, and Hsuan-Tien Lin
In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD) 2025.

We make Complementary Label Learning practical with Vision-Language Model-based auto-labeling for large-scale data.

Defending Text-to-image Diffusion Models: Surprising Efficacy of Textual Perturbations Against Backdoor Attacks

Oscar Chew, Po-Yi Lu, Jayden Lin, Hsuan-Tien Lin
In the Dark Side of Generative AIs and Beyond Workshop @ ECCV, 2024.

We propose straightforward textual perturbations on prompts to defend against backdoor attacks for text-to-image tasks.

A More Robust Baseline for Active Learning by Injecting Randomness to Uncertainty Sampling

Po-Yi Lu, Chun-Liang Li, and Hsuan-Tien Lin
In the Artificial Intelligence & Human Computer Interaction @ ICML 2023.

We investigate the injecting of slight randomness into uncertainty sampling to balance the bias in the pure uncertainty sampling with a small variance.

An Expanded Benchmark that Rediscovers and Affirms the Edge of Uncertainty Sampling for Active Learning in Tabular Datasets

Po-Yi Lu, Yi-Jie Cheng, Chun-Liang Li, and Hsuan-Tien Lin
Transactions on Machine Learning Research (TMLR), 2025. Reproducibility Certification   J2C

We re-benchmark active learning in tabular datasets and affirm the effectiveness of uncertainty sampling.

Eliciting Multimodal Large Language Model In-Context Learning by Instruction Induction

Po-Yi Lu, Oscar Chew, Hsuan-Tien Lin, and Kuan-Hao Huang
Technical report at the Texas NLP Symposium 2026.

We induce task instructions from demonstrations to elicit the in-context learning ability of multimodal large language models.