How to set up ollama on Proxmox with P100 GPU passthrough
In this video, I'll walk you through the process of setting up ollama web interface on proxmox so that it we can utilise and experiment with Large Language Models as well as debug issues related to token (performance output)
Key Steps Covered in the Video:
- Setting up the Proxmox VM enviroment by passing through graphics cards and host CPU cores
- Install Linux on the VM
- Installing the Nvidia P100 driver on the VM to gain access to nvidia-smi
- Installing ollama and the ollama web interface through Docker compose
- Diagnosing pottential issues when using Large Language Models
If you have any questions about the process, feel free to leave a comment on the video or contact me directly.