jupyter-《PyTorch基础教程》01 搭建环境 基于Docker搭建ubuntu22+Python3.10+Pytorch2+cuda11+jupyter的开发环境

01 环境搭建

《PyTorch基础教程》01 搭建环境 基于Docker搭建ubuntu22+Python3.10+Pytorch2+cuda11+jupyter的开发环境

Docker部署PyTorch

拉取cnstark/pytorch镜像

拉取镜像：

docker pull cnstark/pytorch:2.0.1-py3.10.11-cuda11.8.0-ubuntu22.04

导出镜像：

docker save -o pytorch2_python310_cuda11_ubuntu22.tar cnstark/pytorch:2.0.1-py3.10.11-cuda11.8.0-ubuntu22.04

导入镜像：

docker load -i pytorch2_python310_ubuntu22.tar

运行镜像：

mkdir -p /docker/pytorch/project
mkdir -p /docker/pytorch/dataset
docker run --name pytorch -itd -v /docker/pytorch/project:/workspace -v /docker/pytorch/dataset:/workspace/dataset cnstark/pytorch:2.0.1-py3.10.11-cuda11.8.0-ubuntu22.04

# 开启GPU
docker run --name pytorch --gpus all -itd -v /docker/pytorch/project:/workspace -v /docker/pytorch/dataset:/workspace/dataset cnstark/pytorch:2.0.1-py3.10.11-cuda11.8.0-ubuntu22.04

测试PyTorch脚本

main.py

# -*- coding: utf-8 -*-

import torch

dtype = torch.FloatTensor
# dtype = torch.cuda.FloatTensor # Uncomment this to run on GPU

# N is batch size; D_in is input dimension;
# H is hidden dimension; D_out is output dimension.
N, D_in, H, D_out = 64, 1000, 100, 10

# Create random input and output data
x = torch.randn(N, D_in).type(dtype)
y = torch.randn(N, D_out).type(dtype)

# Randomly initialize weights
w1 = torch.randn(D_in, H).type(dtype)
w2 = torch.randn(H, D_out).type(dtype)

learning_rate = 1e-6
for t in range(500):
    # Forward pass: compute predicted y
    h = x.mm(w1)
    h_relu = h.clamp(min=0)
    y_pred = h_relu.mm(w2)

    # Compute and print loss
    loss = (y_pred - y).pow(2).sum()
    print(t, loss)

    # Backprop to compute gradients of w1 and w2 with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_w2 = h_relu.t().mm(grad_y_pred)
    grad_h_relu = grad_y_pred.mm(w2.t())
    grad_h = grad_h_relu.clone()
    grad_h[h < 0] = 0
    grad_w1 = x.t().mm(grad_h)

    # Update weights using gradient descent
    w1 -= learning_rate * grad_w1
    w2 -= learning_rate * grad_w2

运行Python脚本：

docker cp main.py pytorch:/workspace/
docker exec -it pytorch python main.py

其他版本

docker pull cnstark/pytorch:2.0.1-py3.10.11-cuda11.8.0-ubuntu22.04
docker pull cnstark/pytorch:2.0.1-py3.9.17-cuda11.8.0-ubuntu20.04
docker pull cnstark/pytorch:2.0.1-py3.9.17-cuda11.8.0-devel-ubuntu20.04
docker pull cnstark/pytorch:2.0.1-py3.10.11-ubuntu22.04
docker pull cnstark/pytorch:2.0.1-py3.9.17-ubuntu20.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.8.0-ubuntu22.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.8.0-ubuntu20.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.8.0-devel-ubuntu22.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.8.0-devel-ubuntu20.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.7.1-ubuntu22.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-cuda11.7.1-devel-ubuntu22.04
docker pull cnstark/pytorch:2.0.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.16-cuda11.7.1-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.12-cuda11.7.1-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.8.16-cuda11.7.1-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.16-cuda11.7.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.12-cuda11.7.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.8.16-cuda11.7.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.16-ubuntu20.04
docker pull cnstark/pytorch:1.13.1-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.13.0-py3.9.12-cuda11.7.1-ubuntu20.04
docker pull cnstark/pytorch:1.13.0-py3.9.12-cuda11.7.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.13.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.12.1-py3.9.12-cuda11.6.2-ubuntu20.04
docker pull cnstark/pytorch:1.12.1-py3.9.12-cuda11.6.2-devel-ubuntu20.04
docker pull cnstark/pytorch:1.12.1-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.12.0-py3.9.12-cuda11.6.2-ubuntu20.04
docker pull cnstark/pytorch:1.12.0-py3.9.12-cuda11.6.2-devel-ubuntu20.04
docker pull cnstark/pytorch:1.12.0-py3.9.12-cuda11.3.1-ubuntu20.04
docker pull cnstark/pytorch:1.12.0-py3.9.12-cuda11.3.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.12.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.11.0-py3.9.12-cuda11.3.1-ubuntu20.04
docker pull cnstark/pytorch:1.11.0-py3.9.12-cuda11.3.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.11.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.10.2-py3.9.12-cuda11.3.1-ubuntu20.04
docker pull cnstark/pytorch:1.10.2-py3.9.12-cuda11.3.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.10.2-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.10.1-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.10.1-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.10.1-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.10.0-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.10.0-py3.8.16-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.10.0-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.10.0-py3.8.16-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.10.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.9.1-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.9.1-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.9.1-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.9.0-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.9.0-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.9.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.8.1-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.8.1-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.8.1-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.8.0-py3.9.12-cuda11.1.1-ubuntu20.04
docker pull cnstark/pytorch:1.8.0-py3.9.12-cuda11.1.1-devel-ubuntu20.04
docker pull cnstark/pytorch:1.8.0-py3.9.12-ubuntu20.04
docker pull cnstark/pytorch:1.7.1-py3.9.12-cuda11.0.3-ubuntu18.04
docker pull cnstark/pytorch:1.7.1-py3.9.12-cuda11.0.3-devel-ubuntu18.04
docker pull cnstark/pytorch:1.7.1-py3.9.12-ubuntu18.04
docker pull cnstark/pytorch:1.7.0-py3.8.13-cuda11.0.3-ubuntu18.04
docker pull cnstark/pytorch:1.7.0-py3.8.13-cuda11.0.3-devel-ubuntu18.04
docker pull cnstark/pytorch:1.7.0-py3.8.13-ubuntu18.04
docker pull cnstark/pytorch:1.6.0-py3.8.13-cuda10.2-ubuntu18.04
docker pull cnstark/pytorch:1.6.0-py3.8.13-cuda10.2-devel-ubuntu18.04
docker pull cnstark/pytorch:1.6.0-py3.8.13-ubuntu18.04
docker pull cnstark/pytorch:1.5.1-py3.8.13-cuda10.2-ubuntu18.04
docker pull cnstark/pytorch:1.5.1-py3.8.13-cuda10.2-devel-ubuntu18.04
docker pull cnstark/pytorch:1.5.1-py3.8.13-ubuntu18.04
docker pull cnstark/pytorch:1.5.0-py3.8.13-cuda10.2-ubuntu18.04
docker pull cnstark/pytorch:1.5.0-py3.8.13-cuda10.2-devel-ubuntu18.04
docker pull cnstark/pytorch:1.5.0-py3.8.13-ubuntu18.04
docker pull cnstark/pytorch:1.4.0-py3.8.13-cuda10.1-ubuntu18.04
docker pull cnstark/pytorch:1.4.0-py3.8.13-cuda10.1-devel-ubuntu18.04
docker pull cnstark/pytorch:1.4.0-py3.8.13-ubuntu18.04
docker pull cnstark/pytorch:1.2.0-py3.7.13-cuda10.0-ubuntu18.04
docker pull cnstark/pytorch:1.2.0-py3.7.13-cuda10.0-devel-ubuntu18.04
docker pull cnstark/pytorch:1.2.0-py3.7.13-ubuntu18.04

配置apt国内镜像

进入容器：

docker attach pytorch
apt update && apt install git vim 

编辑镜像源：vim /etc/apt/sources.list

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-security main restricted universe multiverse

更新镜像：

apt update && apt upgrade

启动Jupyter

进入容器：

docker attach pytorch

容器中运行：

pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip install jupyter

记录 config.py 文件的位置和文件名：

jupyter notebook --generate-config

# 输出如下内容
# Writing default config to: /root/.jupyter/jupyter_notebook_config.py

设置 jupyter 密码：

ipython

from notebook.auth import passwd
# 如果报错：ModuleNotFoundError: No module named 'notebook.auth'
# from jupyter_server.auth import passwd

passwd()

# 输入：zhangdapeng520  zhangdapeng520
# 输出：Out[4]: 'argon2:$argon2id$v=19$m=10240,t=10,p=8$AZPNix3Xa1A/sKSsZh3fLg$fiCAUy5hlDxeVZOWjVmH2P1fzrRo5kHKlNpWA9Gx8Jo'

修改 config.py 文件，新增：vim /root/.jupyter/jupyter_notebook_config.py

#  旧的Jupyter版本
c.NotebookApp.allow_remote_access = True  # 允许远程
c.NotebookApp.ip = '*'   # 允许其他ip接入
c.NotebookApp.port = 7777  # 容器端口
c.NotebookApp.password =  'argon2:$argon2id$v=xxxxxx'  # 密
c.NotebookApp.open_browser = False   # 默认不打开浏览器
c.NotebookApp.allow_root = True  # 使用root
c.NotebookApp.notebook_dir = '/workspace'  # 默认打开路径

# 2024年2月1日之后的Jupyter版本
c.ServerApp.allow_root = True
c.ServerApp.allow_remote_access = True
c.ServerApp.ip = '0.0.0.0'
c.ServerApp.password = 'zhangdapeng520'
c.ServerApp.open_browser = False
c.ServerApp.port = 8888
c.ServerApp.allow_remote_access = True
c.ServerApp.notebook_dir = '/workspace'

将容器打包为镜像：

docker commit -a "zhangdapeng" -m "pytorch+jupyter" pytorch pytorch:v1

基于新的镜像创建容器：

docker run --name pytorch -itd -p 8888:8888 -v /docker/pytorch/project:/workspace pytorch:v1 bash -c "jupyter lab"

浏览器访问：http://localhost:8888

输入密码：zhangdapeng520

新建一个notebook，输入import torch并执行，如果没有报错，说明我们的环境中有pytorch了。

如果需要停止并删除容器：

docker stop pytorch && docker rm pytorch

保存镜像：

docker save -o pytorch_v1.tar pytorch:v1

导入镜像：

docker load -i pytorch_v1.tar