Prevent Unnecessary Task Execution

By fingerprinting locally generated files and their sources

We can avoid unnecessary workload of tasks to reduce the time taken. Meaning that the tasks will only run when there are changes in the required files.

For example, when you work on a Node.JS application, you may have a task to install dependencies and build the application.

npm install - run this command when there are changes in the package.json file.
npm run build - run this command where there are changes in any **/*.js files.

Taskfile.yaml

version: '3'
 
tasks:
  # if the package-lock.json is deleted, the task will not run again
  npm:install:without-generates:
    cmds:
      - npm install
    sources: # it will compare the checksum to see whether the files got any changes
      - package.json
 
  # if the package-lock.json is deleted, the task will run again
  npm:install:
    cmds:
      - npm install
    sources: # it will compare the checksum to see whether the files got any changes
      - package.json
    generates: # this will be the files generated by the command, if this file is deleted, the task will run again
      - package-lock.json
  
  npm:build:
    cmds:
      - npm run build
    sources:
      - '**/*.js'
      - exclude: ignore.js
 
  python_build:
    cmds:
      - python3 -m build
    sources:
      - setup.py
      - '**/*.py'
      - exclude: tests/** # exclude all files in the tests directory
    generates:
      - dist/*.whl # generated wheel file
    method: timestamp

sources and generates can be files or glob patterns
- sources - it will compare the checksum to see whether the files got any changes
- generates - this will be the files generated by the command, if this file is deleted, the task will run again
exclude - exclude files from fingerprinting, but you have to make sure the sources are in glob format and must come after the positive glob it is negating.
method: timestamp - If you prefer to check the timestamp of the files instead of their checksum, you can use method: timestamp. This is useful for large files where checksum calculation might be expensive.
- method: none - Skips the validation, meaning the task will always run regardless of changes in the source files.
- method: checksum - This is the default method, which checks the checksum of the files to determine if they have changed.
status - You can also use the status command to check the status of the task. It will show whether the task is up to date or not. Refer Using programmatic checks to indicate a task is up to date

Demo and Output

ubuntu@touted-mite:~/nodejsfun$ task npm:install
task: [npm:install] npm install
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'ansi-escapes@6.2.1',
npm WARN EBADENGINE   required: { node: '>=14.16' },
npm WARN EBADENGINE   current: { node: 'v12.22.9', npm: '8.5.1' }
npm WARN EBADENGINE }
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: 'marked-terminal@5.2.0',
npm WARN EBADENGINE   required: { node: '>=14.13.1 || >=16.0.0' },
npm WARN EBADENGINE   current: { node: 'v12.22.9', npm: '8.5.1' }
npm WARN EBADENGINE }
 
up to date, audited 62 packages in 723ms
 
3 packages are looking for funding
  run `npm fund` for details
 
4 moderate severity vulnerabilities
 
Some issues need review, and may require choosing
a different dependency.
 
Run `npm audit` for details.
 
# second time - no changes in package.json
ubuntu@touted-mite:~/nodejsfun$ task npm:install
task: Task "npm:install" is up to date

A .task directory will be created in the source directory to store the fingerprint of the files. This directory is used to track changes in the files specified in the sources section of the task. If you want to avoid committing this directory to version control, consider adding it to your .gitignore file.

If you want to change the location of the .task directory, you can set the TASK_TEMP_DIR environment variable to the desired path.

export TASK_TEMP_DIR=/path/to/custom/.task

When you rerun the task, it will not run the npm install command as there are no changes in the package.json file. You will see a message indicating that the task is up to date.

Using programmatic checks to indicate a task is up to date

Use task --status <tasks> to check the status of a task is up to date or not.

You can also use programmatic checks to indicate whether a task is up to date. In this case, you can use the status command to check the status of the task before running the commands. If the task is up to date, it will skip running the commands.

This method is quite useful when you want to create artifacts or files that are not directly related to the source files, or when you want to ensure that certain files exist before running a task.

Here is how it works: - Task always checks the status first.

On the first run, the status checks will fail (because the directory and files don't exist), so the commands will execute to create them.
On subsequent runs, the status checks will pass (because the directory and files now exist), so the commands will not run again. The task is skipped.

Taskfile.yaml

version: '3'
 
tasks:
  generate-files:
    cmds:
      - mkdir directory
      - touch directory/file1.txt
      - touch directory/file2.txt
    # test existence of files
    status:
      - test -d directory
      - test -f directory/file1.txt
      - test -f directory/file2.txt

Demo and Output

ubuntu@touted-mite:~$ task generate-files 
task: [generate-files] mkdir directory
task: [generate-files] touch directory/file1.txt
task: [generate-files] touch directory/file2.txt
 
ubuntu@touted-mite:~$ task generate-files 
task: Task "generate-files" is up to date

Combination of `sources` and `status` checks:

With the combination of sources and status checks, if either the sources change or programmatic checks fail, then the task will run again.

Taskfile.yaml

version: '3'
 
tasks:
  npm:install:
    cmds:
      - npm install
    sources:
      - package.json
    status:
      - test -f package-lock.json

Using programmatic checks to cancel the execution of a task and its dependencies (Preconditions Checks)

You can setup preconditions checks to run before the task commands. If any of the checks fail, the task will not run. It is very similar to the status command just that it support sh expansion. If any preconditions fail, the task and its dependencies are canceled and do not run.

Taskfile.yaml

version: '3'
tasks:
  generate-files:
    cmds:
      - mkdir directory
    # test existence of files
    preconditions:
      - test -f .env
      - sh: '[ 1 = 0 ]'
        msg: "This will not run because the condition is false"

Here is an example of the dependencies are canceled when the preconditions fail:

Taskfile.yaml

version: '3'
 
tasks:
  task-will-fail:
    preconditions:
      - sh: 'exit 1'
 
  task-will-also-fail:
    deps:
      - task-will-fail
 
  task-will-still-fail:
    cmds:
      - task: task-will-fail
      - echo "I will not run"

Difference between `status` and `preconditions`

Feature	`status`	`preconditions`
Purpose	Check if task is up to date	Ensure requirements are met before running
Effect on Task Execution	Skips task if up to date and continue executing tasks that depend on it	Fails task and any dependent task if conditions are not met

Limiting when tasks run

More Information

run can also be set at the root level of the Taskfile to change the behavior of all tasks in the Taskfil unless overridden in individual tasks.

Sometimes you may want to limit when tasks run based on certain conditions especially there are multiple cmds or deps in the task. In this case, you can use the run to change the behavior of the task execution.

Supported run options:

always (default) - The task will always run regardless of the number of times it has been run.
once - The task will only run once, even if it has been run multiple times.
when_changed - The task will only run once for each unique set of variables passed into the task.

Taskfile.yaml

version: '3'
 
tasks:
  default:
    cmds:
      - task: generate-file
        vars: { CONTENT: '1' }
      - task: generate-file
        vars: { CONTENT: '2' }
      - task: generate-file
        vars: { CONTENT: '2' }
 
  generate-file:
    run: when_changed
    deps:
      - install-deps
    cmds:
      - echo {{.CONTENT}}
 
  install-deps:
    run: once
    cmds:
      - sleep 5 # long operation like installing packages

Demo and Output

ubuntu@touted-mite:~$ task default
task: [install-deps] sleep 5
task: [generate-file] echo 1
1
task: [generate-file] echo 2
2

As you can see, the install-deps task only runs once, even though it is called multiple times in the default task. The generate-file task runs only once for each unique set of variables passed into it.

Prevent Unnecessary Task Execution

By fingerprinting locally generated files and their sources

Using programmatic checks to indicate a task is up to date

Combination of sources and status checks:

Using programmatic checks to cancel the execution of a task and its dependencies (Preconditions Checks)

Difference between status and preconditions

Limiting when tasks run

On this page

Combination of `sources` and `status` checks:

Difference between `status` and `preconditions`