Computer Vision , AI

[One-page summary] MetaFormer Is Actually What You Need for Vision (CVPR 2022) by Yu et al. 본문

Paper_review[short]

[One-page summary] MetaFormer Is Actually What You Need for Vision (CVPR 2022) by Yu et al.

Elune001 2024. 1. 16. 00:58

● Summary: The performance of a transformer comes from its architecture, not the attention module

 

● Approach highlight

  • MetaFormer: The structure of the transformer plays a bigger role in performance than the type of token mixer

The entire architecture of the transformer is the key to performance, not the token mixer(red box).

  • PoolFormer: Prove that the structure of the MetaFormer has a greater impact on the performance of the transformer by replacing the token mixer with a pooling layer to validate performance.

● Main Results:

 

● Discussion

  • The reason why the proposed method(PoolFormer) doesn't work NLP tasks.

'Paper_review[short]' 카테고리의 다른 글

[One-page summary] From Motor Control to Team Play in Simulated Humanoid Football (Science Robotic 2022) by Liu et al.  (0) 2024.01.16
[One-page summary] Make A Video: Text to Video Generation Without Text Video Data (ICLR 2023) by Singer et al.  (0) 2024.01.16
[One-page summary] A Simple MultiModality Transfer Learning Baseline for Sign Language Translation (CVPR 2022) y Chen et al.  (0) 2024.01.16
[One-page summary] Emergence of Maps in the Memories of Blind NavigationAgents(ICLR 2023) by Wijmans et al.  (0) 2024.01.16
[One-page summary] See, Hear, and Feel: Smart Sensory Fusion for RoboticManipulation( CoRL 2022) by Li et al.  (0) 2024.01.16
'Paper_review[short]' Related Articles more