Submit the workload¶
To submit a workload to Bacalhau, we will use the bacalhau docker run command.
%%bash --out job_id
bacalhau docker run \
--wait \
--wait-timeout-secs 100 \
--id-only \
-i ipfs://Qmd9CBYpdgCLuCKRtKRRggu24H72ZUrGax5A9EYvrbC72j:/inputs \
linuxserver/ffmpeg -- \
bash -c 'find /inputs -iname "*.mp4" -printf "%f\n" | xargs -I{} ffmpeg -y -i /inputs/{} -vf "scale=-1:72,setsar=1:1" /outputs/scaled_{}'
The job has been submitted and Bacalhau has printed out the related job id. We store that in an environment variable so that we can reuse it later on.
%env JOB_ID={job_id}
The bacalhau docker run command allows one to pass input data volume with a -i ipfs://CID:path argument just like Docker, except the left-hand side of the argument is a content identifier (CID). This results in Bacalhau mounting a data volume inside the container. By default, Bacalhau mounts the input volume at the path /inputs inside the container.
We created a 72px wide video thumbnails for all the videos in the inputs directory. The outputs directory will contain the thumbnails for each video. We will shard by 1 video per job, and use the linuxserver/ffmpeg container to resize the videos.
:::tip
Bacalhau overwrites the default entrypoint so we must run the full command after the -- argument. In this line you will list all of the mp4 files in the /inputs directory and execute ffmpeg against each instance.
:::
Checking the State of your Jobs¶
- Job status: You can check the status of the job using
bacalhau list.
%%bash
bacalhau list --id-filter=${JOB_ID} --no-style
When it says Published or Completed, that means the job is done, and we can get the results.
- Job information: You can find out more information about your job by using
bacalhau describe.
%%bash
bacalhau describe ${JOB_ID}
- Job download: You can download your job results directly by using
bacalhau get. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
%%bash
mkdir -p ./results # Temporary directory to store the results
bacalhau get --output-dir ./results ${JOB_ID} # Download the results
After the download has finished you should see the following contents in results directory.
Viewing your Job Output¶
To view the file, run the following command:
%%bash
# Copy the files to the local directory, to allow the documentation scripts to copy them to the right place
cp results/outputs/* ./ && rm -rf results/outputs/*
# Remove any spaces from the filenames
for f in *\ *; do mv "$f" "${f// /_}"; done
Display the videos¶
To view the videos, we will use glob to return all file paths that match a specific pattern.
import glob
from IPython.display import Video, display
for file in glob.glob('*.mp4'):
display(Video(filename=file))
<video src={require('./scaled_Bird_flying_over_the_lake.mp4').default} controls >
Your browser does not support the video element.