Excited to share that I will be joining
@UCBerkeley
as a PhD student this Fall. I wish to continue working on the efficiency and interpretability of deep learning models, hoping to deepen our understanding of these systems.
🚨 Excited to share our
#ICML2023
work on the Feature Sieve, by which we automatically identify and suppress irrelevant or spurious features in deep networks, hence mitigating simplicity bias.
Paper:
What a way to end my
#WACV2024
. Got a chance to chat (turned to short AMA 😅) with amazing and super humble
@CSProfKGD
. Hope you get a chance to tick off visiting India from your bucket list soon.
@AditayTripathi
@sourabhgothe
A very interesting paper by my good friend
@rach_it_
w/
@GoogleAI
and
@GoogleDeepMind
. Shows the strategy to compose different LLMs to unlock new capabilities. Seems like a critical step in the right direction towards practical usability of LLMs.
Extending an LLM for new knowledge sources is tedious—fine-tuning is expensive/causes forgetting, LoRA is restrictive.
Excited to share our work where we show that an LLM can be efficiently *composed* with specialized (L)LMs to enable new tasks!
🧵(1/8)
(2) Problem:
Simplicity bias: Models depend on weak, easily computed predictive features even if stronger alternatives exist.
Spurious correlations: DNNs latch onto apparent feature-label correlations present in biased training data distributions.
(7) Optimization: We use accuracy on validation set for setting hparams. We require no knowledge of potential spurious attribute dimensions, nor even a curated / balanced validation set. Of course, well-chosen validation data can be exploited to guide the resulting classifier.
(5) Hypothesis: Simple features are learned early, lower in the network, and proliferate through the rest of the network. What if we can automatically identify and suppress such features?
(3) Results (quant): SIFER gives around 65.7% accuracy without access to test distribution outperforming other baselines. Using a validation set drawn from test distribution further improves accuracy by 6.3%. SIFER beats SOTA by significant margins across debiasing benchmarks.
(6) Workflow: We alternate between identifying simple features (i.e., reading them out from intermediate network layers), and erasing them in lower layers. This simple yet powerful scheme mediates between (“sieves through”) competing features, picking only those that generalize.
(4) Results (qual): SIFER automatically identifies and suppresses spurious features without access to attributes or other distributional information regarding features. Examples shown below for CelebA-Hair (spurious gender feature) and BAR (spurious background features).