Problem of cuda driver when building

Hi. I am trying to add my models to the sota bench for image classification on imagenent, I have connected to my github repository, and during the build, when running the benchmarking script, I get the following error :

Benchmarking on ImageNet...
Traceback (most recent call last):
  File "/workspace/repository/sotabench.py", line 70, in <module>
    data_root=os.environ.get('IMAGENET_DIR', './imagenet')
  File "/workspace/venv/lib/python3.7/site-packages/sotabenchapi/core/inputs.py", line 85, in regular_evaluation
    func(*args, **kwargs)
  File "/workspace/venv/lib/python3.7/site-packages/torchbench/image_classification/imagenet.py", line 169, in benchmark
    model, device=device, num_gpu=num_gpu
  File "/workspace/venv/lib/python3.7/site-packages/torchbench/utils.py", line 73, in send_model_to_device
    model = model.to(device=device)
  File "/workspace/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 443, in to
    return self._apply(convert)
  File "/workspace/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 203, in _apply
    module._apply(fn)
  File "/workspace/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 225, in _apply
    param_applied = fn(param)
  File "/workspace/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 441, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/workspace/venv/lib/python3.7/site-packages/torch/cuda/__init__.py", line 149, in _lazy_init
    _check_driver()
  File "/workspace/venv/lib/python3.7/site-packages/torch/cuda/__init__.py", line 63, in _check_driver
    of the CUDA driver.""".format(str(torch._C._cuda_getDriverVersion())))
AssertionError: 
The NVIDIA driver on your system is too old (found version 10010).
Please update your GPU driver by downloading and installing a new
version from the URL: http://www.nvidia.com/Download/index.aspx
Alternatively, go to: https://pytorch.org to install
a PyTorch version that has been compiled with your version
of the CUDA driver.

Any idea how to solve this?

Thanks

Exactly the same error here. They need to update their machines!

However, the error message tells you it’s got cuda 10.1, and from that information you can figure out that if you put torch==1.4.0 and torchvision==0.5.0 in your requirements.txt, it should work. Tried that, as a workaround, and it works, until they do update their machines and this will not work anymore :slight_smile:

The above worked for me. Another alternative, which I have not tried but should work, is to put the exact URL for the package with cuda 10.1 and python3.7 in the requirements.txt, like so:

https://download.pytorch.org/whl/cu101/torch-1.5.0+cu101-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cu101/torchvision-0.6.0+cu101-cp37-cp37m-linux_x86_64.whl

But that too will break eventually when they update their machines. I think it is a difficult task to provide a machine with cuda version that all major frameworks support.

Thanks a lot for this answer