Summarize LSF job properties by parsing log files of workflows executed by Snakemake.
$ pip install lsf_stats$ lsf_stats --help
Usage: lsf_stats [OPTIONS] COMMAND [ARGS]...
Summarize LSF job properties by parsing log files.
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
gather Aggregate information from log files in single dataframe.
summarize Summarize and visualize aggregated information.Assume that you executed your Snakemake workflow using the lsf-profile and all generated log files are stored in the directory ./logs/:
$ snakemake --profile lsf
[..]You can then quickly aggregate resource, rule and other types of information about the workflow execution into a single dataframe:
$ lsf_stats gather -o workflow_stats.csv.gz ./logs/
[..]This dataframe can then be summarized in various ways:
$ lsf_stats summarize \
--query 'status == "Successfully completed."' \
--split-wildcards \
--grouping-variable category \
workflow_stats.csv.gz
[..]For example, the following plots will be generated:
| Job execution | Job resources |
|---|---|
 |
 |