Research
I am broadly interested in deep learning and natural language processing, specifically the emergent
capabilities, applications, and risks of language models.
Thanks to my amazing mentors and collaborators! ☺
* denotes equal contribution
|
Challenges in Context-Aware Neural Machine Translation
Linghao Jin*, Jacqueline He*, Jonathan May, Xuezhe Ma
EMNLP 2023
abstract /
paper /
code
Context-aware neural machine translation involves leveraging information beyond sentence-level context
to resolve inter-sentential discourse dependencies and improve document-level translation quality, and
has given rise to a number of recent techniques. However, despite well-reasoned intuitions, most
context-aware translation models show only modest improvements over sentence-level systems. In this
work, we investigate several challenges that impede progress within this field, relating to discourse
phenomena, context usage, model architectures, and document-level evaluation. To address these
problems, we propose a more realistic setting for document-level translation, called
paragraph-to-paragraph (para2para) translation, and collect a new dataset of Chinese-English novels to
promote future research.
|
MABEL: Attenuating Gender Bias using Textual Entailment Data
Jacqueline He, Mengzhou Xia, Christiane Fellbaum, Danqi Chen
EMNLP 2022
abstract /
paper /
code
Pre-trained language models encode undesirable social biases, which are further exacerbated in
downstream use. To this end, we propose MABEL (a Method for Attenuating Gender Bias using
Entailment Labels), an intermediate pre-training approach for mitigating gender bias in
contextualized representations. Key to our approach is the use of a contrastive learning objective
on counterfactually augmented, gender-balanced entailment pairs from natural language inference
(NLI) datasets. We also introduce an alignment regularizer that pulls identical entailment pairs
along opposite gender directions closer. We extensively evaluate our approach on intrinsic and
extrinsic metrics, and show that MABEL outperforms previous task-agnostic debiasing approaches in
terms of fairness. It also preserves task performance after fine-tuning on downstream tasks.
Together, these findings demonstrate the suitability of NLI data as an effective means of bias
mitigation, as opposed to only using unlabeled sentences in the literature. Finally, we identify
that existing approaches often use evaluation settings that are insufficient or inconsistent. We
make an effort to reproduce and compare previous methods, and call for unifying the evaluation
settings across gender debiasing methods for better future comparison.
|
Can Rationalization Improve Robustness?
Howard Chen, Jacqueline He, Karthik Narasimhan, Danqi Chen
NAACL 2022
abstract /
paper /
code
A growing line of work has investigated the development of neural NLP models that can produce
rationales--subsets of input that can explain their model predictions. In this paper, we ask
whether such rationale models can also provide robustness to adversarial attacks in addition to
their interpretable nature. Since these models need to first generate rationales
("rationalizer") before making predictions ("predictor"), they have the potential to ignore
noise or adversarially added text by simply masking it out of the generated rationale. To this
end, we systematically generate various types of 'AddText' attacks for both token and
sentence-level rationalization tasks, and perform an extensive empirical evaluation of
state-of-the-art rationale models across five different tasks. Our experiments reveal that the
rationale models show the promise to improve robustness, while they struggle in certain
scenarios--when the rationalizer is sensitive to positional bias or lexical choices of attack
text. Further, leveraging human rationale as supervision does not always translate to better
performance. Our study is a first step towards exploring the interplay between interpretability
and robustness in the rationalize-then-predict framework.
|
|