# 负采样

## 负采样

* [返回顶层目录](/machine-learning-notes/summary.md)
* [返回上层目录](/machine-learning-notes/tips.md)
* [两种负采样](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#两种负采样)
  * [第一种负采样](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#第一种负采样)
  * [第二种负采样](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#第二种负采样)
* [两种负采样的本质区别](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#两种负采样的本质区别)
  * [第一种负采样求梯度](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#第一种负采样求梯度)
  * [第二种负采样求梯度](https://luweikxy.gitbook.io/machine-learning-notes/tips/pages/-M7qWQvxMH1Dsr5WM9kZ#第二种负采样求梯度)

## 两种负采样

### 第一种负采样

采样方式为：

$$
\begin{aligned}
\<u, i\_0>,\\
\<u, i\_1>,\\
\<u, i\_2>,\\
\<u, i\_3>,\\
\<u, i\_4>,\\
\<u, i\_5>
\end{aligned}
$$

其中，$$i\_0$$为正样本，$$i\_1,\ i\_2, ... , i\_5$$为负样本。

损失计算方式为

$$
\begin{aligned}
\text{loss}&=-p\_0\mathop{\Pi}*{i=1}^5(1-p\_i)\\
\Rightarrow\text{loss}&=-log(p\_0)-\sum*{i=1}^5log(1-p\_i)\\
&=-log\left(\frac{1}{1+\text{exp}(-s\_0)}\right)-\sum\_{i=1}^5log\left(1-\frac{1}{1+\text{exp}(-s\_i)}\right)\\
\end{aligned}
$$

### 第二种负采样

采样方式为：

$$
\<u, i\_0, i\_1, i\_2, i\_3, i\_4, i\_5>
$$

其中，$$i\_0$$为正样本，$$i\_1,\ i\_2, ... , i\_5$$为负样本。

损失计算方式为

$$
\begin{aligned}
&\text{loss}=-\frac{\text{exp}(s\_0)}{\sum\_{i=0}^5\text{exp}(s\_i)}\\
\Rightarrow &\text{loss}=-(s\_0-\text{log}\sum\_{i=0}^5\text{exp}(s\_i))=-s\_0+\text{log}\sum\_{i=0}^5\text{exp}(s\_i)
\end{aligned}
$$

## 两种负采样的本质区别

对两种负采样的损失函数分别求梯度。

### 第一种负采样求梯度

$$
\begin{aligned}
\frac{\partial \text{loss}}{\partial s\_0}&=\frac{\partial\left\[-log\left(\frac{1}{1+\text{exp}(-s\_0)}\right)-\sum\_{i=1}^5log\left(1-\frac{1}{1+\text{exp}(-s\_i)}\right)\right]}{\partial s\_0}\\
&=-\frac{p\_0(1-p\_0)}{p\_0}\\
&=-1+p\_0\\

\frac{\partial \text{loss}}{\partial s\_i}|*{i=1,2,...,5}&=\frac{\partial\left\[-log\left(\frac{1}{1+\text{exp}(-s\_0)}\right)-\sum*{i=1}^5log\left(1-\frac{1}{1+\text{exp}(-s\_i)}\right)\right]}{\partial s\_i}\\
&=-\frac{p\_i(1-p\_i)}{1-p\_i}\\
&=p\_i\\
\end{aligned}
$$

其中，上式的推导用到了 sigmoid导数的特点：$$f'(z)=f(z)(1-f(z))$$。

注意，这里的

$$
\begin{aligned}
\sum\_{i=0}^5\frac{\partial \text{loss}}{\partial s\_i}=-1+\sum\_{i=0}^5p\_i\neq0
\end{aligned}
$$

即**所有的变量**$$s\_i$$**的梯度的和不是1**

### 第二种负采样求梯度

$$
\begin{aligned}
\frac{\partial \text{loss}}{\partial s\_0}&=\frac{\partial\[-(s\_0-\text{log}\sum\_{i=0}^5\text{exp}(s\_i))]}{\partial s\_0}\\
&=-1+\frac{\text{exp}(s\_0)}{\sum\_{i=0}^5\text{exp}(s\_i)}\\
&=-1+p\_0\\

\frac{\partial \text{loss}}{\partial s\_i}|*{i=1,2,...,5}&=\frac{\partial\[-(s\_0-\text{log}\sum*{i=0}^5\text{exp}(s\_i))]}{\partial s\_i}\\
&=\frac{\text{exp}(s\_i)}{\sum\_{i=0}^5\text{exp}(s\_i)}\\
&=p\_i\\
\end{aligned}
$$

注意：所有变量的梯度值相加等于1:

$$
\begin{aligned}
\sum\_{i=0}^5\frac{\partial \text{loss}}{\partial s\_i}=-1+\sum\_{i=0}^5p\_i=-1+1=0
\end{aligned}
$$

所以，**每个变量**$$s\_i$$**的梯度更新值是相互制约的，总和等于1**。

**这就是两种负采样的本质区别，即所有的变量**$$s\_i$$**的梯度的和是否等于1**。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://luweikxy.gitbook.io/machine-learning-notes/tips/negtive-sampling.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.