wordpress以前版本星巴克seo網(wǎng)絡(luò)推廣
操作系統(tǒng):ubuntu22.04 LTS
python版本:3.12.7
最近學(xué)習(xí)了用poetry配置python虛擬環(huán)境,當(dāng)為不同的項目配置cuda時,會遇到不同的項目使用的cuda版本不一致的情況。
像torch 這樣的庫,它們會對cuda-toolkit有依賴,通過python來使用cuda,它們會依賴像
nvidia-cuda-runtime-cu12
nvidia-cublas-cu12
這樣的python庫,這種情況下,在pyproject.toml里寫上?
[tool.poetry.group.full]
optional = true
[tool.poetry.group.full.dependencies]
torch = {path = "./torch-2.2.1+cu121-cp312-cp312-linux_x86_64.whl"}
或者
torch = "2.2.1"
這樣的依賴就行了,poetry會自動下載依賴的cuda-toolkit的python庫
這時候,通過poetry show torch --tree可以查看到依賴關(guān)系:
user@user-Ubuntu2204:~/projects/cuda-test$ poetry show torch --tree
torch 2.2.1+cu121 Tensors and Dynamic neural networks in Python with strong GPU acceleration
├── filelock *
├── fsspec *
├── jinja2 *
│ └── markupsafe >=2.0
├── networkx *
├── nvidia-cublas-cu12 12.1.3.1
├── nvidia-cuda-cupti-cu12 12.1.105
├── nvidia-cuda-nvrtc-cu12 12.1.105
├── nvidia-cuda-runtime-cu12 12.1.105
├── nvidia-cudnn-cu12 8.9.2.26
│ └── nvidia-cublas-cu12 *
├── nvidia-cufft-cu12 11.0.2.54
├── nvidia-curand-cu12 10.3.2.106
├── nvidia-cusolver-cu12 11.4.5.107
│ ├── nvidia-cublas-cu12 *
│ ├── nvidia-cusparse-cu12 *
│ │ └── nvidia-nvjitlink-cu12 *
│ └── nvidia-nvjitlink-cu12 * (circular dependency aborted here)
├── nvidia-cusparse-cu12 12.1.0.106
│ └── nvidia-nvjitlink-cu12 *
├── nvidia-nccl-cu12 2.19.3
├── nvidia-nvtx-cu12 12.1.105
├── sympy *
│ └── mpmath >=1.1.0,<1.4
└── typing-extensions >=4.8.0
有的基于gpu的庫不通過python來使用cuda,這時候就要給操作系統(tǒng)安裝對應(yīng)的cuda版本才行。這時候,從poetry show 查看依賴關(guān)系是看不到它對nvidia的cuda-toolkit的依賴的:
user@user-Ubuntu2204:~/projects/cuda-test$ poetry show paddlepaddle-gpu --tree
paddlepaddle-gpu 2.6.2.post120 Parallel Distributed Deep Learning
├── astor *
├── decorator *
├── httpx *
│ ├── anyio *
│ │ ├── idna >=2.8
│ │ └── sniffio >=1.1
│ ├── certifi *
│ ├── httpcore ==1.*
│ │ ├── certifi * (circular dependency aborted here)
│ │ └── h11 >=0.13,<0.15
│ ├── idna * (circular dependency aborted here)
│ └── sniffio * (circular dependency aborted here)
├── numpy >=1.13
├── opt-einsum 3.3.0
│ └── numpy >=1.7
├── pillow *
├── protobuf >=3.20.2
└── protobuf >=3.1.0,<=3.20.2
另外,像torch這樣通過python來使用cuda,可以給nvidia的cuda-toolkit庫指定路徑:
[tool.poetry.dependencies]
python = "^3.12"
setuptools = "70.0.0"nvidia-cublas-cu12={ path = "./nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl" }
nvidia-cuda-cupti-cu12={ path = "./nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl" }