How to conditionally download a file?

[jess-sol]

I'm struggling to come up with a clean way to download a file a single time in Just, and I'm not sure if it's me not understanding the tool, or the tool just not doing what I want to use it for.

Here's the example, I want to download a file, and run a command on the file if it's been downloaded. If it already exists, I want to do nothing (but also not error).

Example:

wget https://cloud.debian.org/images/cloud/bullseye/20230601-1398/debian-11-genericcloud-amd64-20230601-1398.qcow2
virt-customize --root-password password:debian

I tried a few different ways to do this in Just, and ended up with:

download_debian:
  if [ "{{ path_exists(debian_path) }}" != "true" ]; then \
    wget -P contrib "{{ debian_url }}"; \
    virt-customize --root-password password:debian -a "{{ debian_path }}"; \
  fi

But I can't help but feel like that's pretty hacky. Is there a better way to represent this? Something like #867 might help, but that didn't look like it was going anywhere.

[casey]

You can use a shebang recipe, which would allow you to avoid escaping every line, and use bash's built-in conditionals:

download_debian:
  #!/usr/bin/env bash
  if [[ ! -f "{{debian_path}}" ]]; then
    wget -P contrib "{{ debian_url }}"
    virt-customize --root-password password:debian -a "{{ debian_path }}"
  fi

[jess-sol]

Ah okay I had tried that but for some reason substitution wasn't working, so I assumed you couldn't substitute variables in shebang mode.

I was also wanting to use file_exists instead of if -f which wasn't a good idea.

Thanks!

[danielb2]

@casey i think this is where a makefile shines. It would be awesome to be able to specify if a recipe is actually a file and not run (nothing to do a la make) if the file is present.

Perhaps something like this:

@debian.tar.gz:
  wget -o debian.tar.gz {{ debian_url }}

download_debian: @debian.tar.gz

I end up using both Justfiles and Makefiles since just can't handle this

[edvardm]

I tend to do simply test -f <file> || <action to acquire file, maybe a just invocation?>, works often in single line

[bukowa]

Here is my recipe downloading and unzipping with some SHA cache-like-check maybe you will find it useful:

[group('webview')]
[doc('get microsoftedge installed version')]
[windows]
browser-version:
    #!/bin/bash
    wmic datafile where \
      'name="C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"' \
      get version | tail -n2 | xargs

[group('webview')]
[doc('get the microsoftedge webdriver url')]
[windows]
webdriver-url:
    #!/bin/bash
    echo "https://msedgedriver.azureedge.net/$(just browser-version)/edgedriver_win64.zip"

[group('webview')]
[doc('downloads the microsoftedge webdriver')]
[windows]
webdriver-download: init
    #!/bin/bash
    if [ -n "$JUSTFILE_TRACE" ]; then set -x; fi
    set -euo pipefail

    version=$(just browser-version)
    bytes=$(echo $version | wc -c)
    if [ $bytes -ne 14 ]; then
        echo "ERROR found incorrect number of bytes in the version number: '$version'"
        echo "$version"
        exit 1
    fi

    zip_file=$PATH_BUILD_TEMP/webdriver.zip
    unzip_dir=$PATH_BUILD_TEMP/webdriver
    unzip_file=$E2E_WEBDRIVER_BINARY
    unzip_file_compares=$unzip_dir/msedgedriver.$version.exe

    # if there's unzip_file and unzip_file_compares
    # compare their sha, if they are the same, then
    # we don't need to download the webdriver
    if [ -f $unzip_file ] && [ -f $unzip_file_compares ]; then
        sha1sum $unzip_file $unzip_file_compares | awk '{print $1}' | uniq | wc -l | grep -q 1
        if [ $? -eq 0 ]; then
            echo "Webdriver found at: '$unzip_file'"
            exit 0
        fi
    fi

    curl -L $(just webdriver-url) -o $zip_file
    unzip -o $zip_file -d $unzip_dir
    cp $unzip_dir/msedgedriver.exe $unzip_file
    cp $unzip_dir/msedgedriver.exe $unzip_file_compares
    test -f $E2E_WEBDRIVER_BINARY