Yifan Zhang
@yifan_zhang_
Followers
256
Following
71
Statuses
42
Graduate student at IIIS, Tsinghua, visiting graduate student at UCLA.
Los Angeles, CA
Joined October 2022
1/ Introducing “Tensor Product Attention Is All You Need” (TPA) and Tensor ProducT ATTenTion Transformer (T6)! 🚀 Ever wondered if there’s a more memory-efficient way to handle long contexts in LLMs? Homepage:
5
66
314
RT @KMohan2006: 1/n 'Tensor Product Attention is all you need' paper Key Points -> 1. KV size reduction by using contextual tensor decom…
0
15
0
RT @QuanquanGu: MHA-->GQA-->MLA--->TPA🚀🚀🚀 Introducing Tensor Product Attention (TPA). To reduce KV cache size, various Multi-Head Attenti…
0
55
0
RT @iScienceLuvr: Tensor Product Attention Is All You Need Proposes Tensor Product Attention (TPA), a mechanism that factorizes Q, K, and…
0
88
0
RT @gm8xx8: Tensor Product Attention Is All You Need Tensor Product Attention reduces memory overhead by compressing KV cache using tensor…
0
39
0
@Grad62304977 @QuanquanGu @YIFENGLIU_AI @HuizhuoY Forthcoming works will discuss about it (Flash TPA).
0
0
6
RT @yifan_zhang_: 1/ Introducing “Tensor Product Attention Is All You Need” (TPA) and Tensor ProducT ATTenTion Transformer (T6)! 🚀 Ever wo…
0
66
0
12/ Joint work with @yifan_zhang_, @YIFENGLIU_AI, @HuizhuoY, Zhen Qin, Yang Yuan, @QuanquanGu, and Andrew Chi-Chih Yao. Incredible work by an outstanding team!
0
1
12