Since all arguments to wfbench are passed on the command line, for large workflows/benchmarks where a task can have a very large number of (typically) input files wfbench can fail with the dreaded "Command line too long" error.
The fix would be to allow wfbench to take its input either from command-line arguments or from a single argument that specifies a path to a JSON file that describes all or some of the arguments. Perhaps just allowing this for input file and output files is ok. Something like:
--input-files xxx
--output-files xxx
as we have now, but if the passed string ends with .json assume it's a file instead of a JSON string? It should be sufficient since the non-file command line arguments above need to end with a '}' or a ']'
Since all arguments to
wfbenchare passed on the command line, for large workflows/benchmarks where a task can have a very large number of (typically) input fileswfbenchcan fail with the dreaded "Command line too long" error.The fix would be to allow
wfbenchto take its input either from command-line arguments or from a single argument that specifies a path to a JSON file that describes all or some of the arguments. Perhaps just allowing this for input file and output files is ok. Something like:as we have now, but if the passed string ends with
.jsonassume it's a file instead of a JSON string? It should be sufficient since the non-file command line arguments above need to end with a '}' or a ']'