Momentum Contrast (MoCo) for Unsupervised Visual Representation Learning

1 Introduction

1.1 Instance discrimination (样本判别)

制定了一种划分正样本和负样本的规则

1.2 InfoNCE Loss

1.3 Momentum

动量在数学上可以理解为是一种指数移动平均(Exponential Moving Average)
\(m\)为动量系数，目的是为了 \(Y_t\) 不仅仅依赖于当前时刻的输入 \(X_t\)

$Y_t = mY_{t-1} + (1-m)X_t$

• \(Y_t\): 当前时刻的输出
• \(Y_{t-1}\): 上一时刻的输出
• \(X_t\): 当前时刻的输入

\(m\)越大，对当前时刻的输入 \(X_t\) 的依赖越小

1.4 Momentum Contrast (MoCo)

2 Related Work

the relationships between Supervised Learning and Unsupervised Learning/Self-supervised Learning

2.1 Loss Function(目标函数)

2.2 Pretext tasks(代理任务)