Skip requirements.txt, run setup.sh in venv?

Hi, I am trying to add one model to sotabench. However, our repo (https://github.com/google-research/big_transfer) is a little non-standard because we have three implementations, each in their subfolder. Hence, a top-level requirements.txt does not make sense in our case.

I thought of installing the requirements in the sotabench_setup.sh file instead using pip install .... But it seems like you run that sh file before entering the virtualenv, which makes this fail. I think it would be more useful to run it inside the virtualenv, especially if someone needs to run custom steps like Cython builds, which is your motivation in the first place.

Second comment is that for this to work, the build process should not fail on missing requirements.txt but instead just skip running the command.

Screenshot of the problems: https://imgur.com/NVbL9v2

@lbeyer thanks to your post I figured out a workaround to the driver version issue w/ pytorch 1.5… thought I’d share since it’ll help your other issue… I fixed mine by creating a parallel requirements-sotabench.txt with more specific version constraints. In sotabench_setup.sh I pip install that requirements so that by the time it tries the normal one, the reqs are installed already. This seems to work fine. And you can load the venv in the setup script to install within the venv…

Example setup script:

#!/usr/bin/env bash
source /workspace/venv/bin/activate

pip install -r requirements-sotabench.txt

apt-get git
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --no-cache-dir ./

Full examples at