pytorch-handbook/chapter2/2.1.2-pytorch-basics-autograd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch 基础 : 自动求导\n",
    "深度学习的算法本质上是通过反向传播求导数，而PyTorch的autograd模块则实现了此功能。在Tensor上的所有操作，autograd都能为它们自动提供微分，避免了手动计算导数的复杂过程。\n",
    "\n",
    "***从0.4起, Variable 正式合并入Tensor, Variable 本来实现的自动微分功能，Tensor就能支持。读者还是可以使用Variable(tensor), 但是这个操作其实什么都没做。***\n",
    "\n",
    "所以，以后的代码建议直接使用Tensor，因为官方文档中已经将Variable设置成过期模块\n",
    "\n",
    "要想使得Tensor使用autograd功能，只需要设置tensor.requries_grad=True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4.1'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 首先要引入相关的包\n",
    "import torch\n",
    "#打印一下版本\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在张量创建时，通过设置 requires_grad 标识为Ture来告诉Pytorch需要对该张量进行自动的求导，PyTorch回记录该张量的每一步操作历史并自动计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.7966, 0.6401, 0.1673, 0.4988, 0.3223],\n",
       "        [0.8881, 0.4756, 0.4391, 0.8762, 0.5714],\n",
       "        [0.9581, 0.3330, 0.0885, 0.5103, 0.7473],\n",
       "        [0.9493, 0.4851, 0.8115, 0.0083, 0.2560],\n",
       "        [0.3152, 0.0132, 0.0858, 0.0406, 0.1677]], requires_grad=True)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(5, 5, requires_grad=True)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.7349, 0.7580, 0.2178, 0.1207, 0.9278],\n",
       "        [0.8113, 0.3937, 0.9049, 0.0681, 0.0810],\n",
       "        [0.5836, 0.2981, 0.7601, 0.6309, 0.3913],\n",
       "        [0.4454, 0.2274, 0.4062, 0.5707, 0.7378],\n",
       "        [0.4096, 0.7709, 0.7267, 0.4186, 0.1841]], requires_grad=True)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = torch.rand(5, 5, requires_grad=True)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看到 该张量的grad_fn已经被赋予了一个新的函数，下面我们来调用反向传播函数，计算其梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(24.0248, grad_fn=<SumBackward0>)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z=torch.sum(x+y)\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 简单的自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]]) tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "#看一下x和y的梯度\n",
    "print(x.grad,y.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 复杂的自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.7252, 1.7343, 0.0371, 0.3972, 0.7656],\n",
       "        [0.1452, 0.6768, 0.7937, 0.4231, 0.5193],\n",
       "        [1.6275, 0.2478, 0.0672, 0.8888, 0.5644],\n",
       "        [1.3385, 0.1239, 1.0143, 0.0052, 1.0832],\n",
       "        [0.9570, 0.2122, 0.5613, 0.1223, 0.3774]], grad_fn=<ThAddBackward>)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(5, 5, requires_grad=True)\n",
    "y = torch.rand(5, 5, requires_grad=True)\n",
    "z= x**2+y**3\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.2330, 1.9195, 0.2359, 1.2472, 1.7473],\n",
      "        [0.7365, 0.5059, 1.7816, 1.0653, 1.4377],\n",
      "        [1.8439, 0.7431, 0.5183, 1.5588, 1.4149],\n",
      "        [1.4076, 0.3347, 1.6417, 0.1141, 0.7594],\n",
      "        [1.8638, 0.5341, 0.3941, 0.6932, 0.6859]])\n"
     ]
    }
   ],
   "source": [
    "#我们的返回值不是一个scalar，所以需要输入一个大小相同的张量作为参数，这里我们用ones_like函数根据x生成一个张量\n",
    "z.backward(torch.ones_like(x))\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用`with torch.no_grad()`禁止已经设置requires_grad=True的向量进行自动求导，这个方法在测试集测试准确率的时候回经常用到，例如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "with torch.no_grad():\n",
    "    print((x +y*2).requires_grad)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Pytorch for Deeplearning",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}