Hi All, We've been using Pulsar to send jobs to our cluster on campus. Generally we send a dataset list of files over. We're wondering if there is a way to get the metadata for the files after they've been sent to the cluster. We're specifically trying to find the filepaths of the files before they were through Pulsar to the cluster. Any guidance would be great. Thanks, Joe
I perhaps don't understand the question. As long as the "datasets" are Galaxy datasets and they appear in the command-line specific by the "command" block of your tool - it should be transmitted. When you say a "dataset list of files" - do you mean (1) a Galaxy collection, (2) a set of files selected with a <param type="data" multiple="true" /> block in your tool, or (3) do you have a special datatype that references lists of paths on the file system not tracked by Galaxy? Pulsar doesn't really have an API to expose these things to your tools - as long as you are sticking to Galaxy's abstractions these things should just work. If you have tools or datatypes that reference arbitrary other files on the file system (like 3 above) that is going to break some Galaxy abstractions including Pulsar, quota tracking, purging datasets, object stores, etc.... -John On Thu, Feb 2, 2017 at 8:49 AM, Joseph Brent Greer <joseph.greer@northwestern.edu> wrote:
Hi All,
We’ve been using Pulsar to send jobs to our cluster on campus. Generally we send a dataset list of files over. We’re wondering if there is a way to get the metadata for the files after they’ve been sent to the cluster. We’re specifically trying to find the filepaths of the files before they were through Pulsar to the cluster.
Any guidance would be great.
Thanks,
Joe
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
John Chilton
-
Joseph Brent Greer