- galaxy-dev - lists.galaxyproject.org

Still an issue with PBS Pro
by SAPET, Frederic 24 Aug '16

24 Aug '16

Hello Despite the last update made to the file lib/galaxy/jobs/runners/drmaa.py I still need to modify this file in order to truncate the job name to 15 characters. I replace jobName=ajs.job_name by jobName=ajs.job_name[:15], (line 135). The line 396 (job_name = job_name[:self.restrict_job_name_length]) seems to be inefficient. Maybe I missed something ? Thank you. Fred

1 0

Issue when using a symbolyc link and try to import history
by SAPET, Frederic 24 Aug '16

24 Aug '16

Hello I'm trying the latest Galaxy (git status gives me : # On branch release_16.07) Within the database directory, I've create a symbolic link : tmp -> /tmp But if this symbolic link is active, I'm unable to import an history using the "Import from file" link. Here is the error : galaxy.jobs ERROR 2016-08-24 16:04:24,266 Unable to cleanup job 127 Traceback (most recent call last): File "/softs/bioinfo/galaxy-prod/lib/galaxy/jobs/__init__.py", line 1440, in cleanup galaxy.tools.imp_exp.JobImportHistoryArchiveWrapper( self.app, self.job_id ).cleanup_after_job() File "/softs/bioinfo/galaxy-prod/lib/galaxy/tools/imp_exp/__init__.py", line 78, in cleanup_after_job history_attr_str = read_file_contents( history_attr_file_name ) File "/softs/bioinfo/galaxy-prod/lib/galaxy/tools/imp_exp/__init__.py", line 44, in read_file_contents fp = open( file_path, 'rb' ) IOError: [Errno 2] No such file or directory: '/softs/bioinfo/galaxy-prod/database/tmp/tmp3SWdq_/history_attrs.txt' If I remove the link and re-create a real directory, it works. Fred

1 0

sending data
by Labdros drosophila 22 Aug '16

22 Aug '16

In the last two days I'm dificuldaddes techniques to send my data by BAREFTP , I wanted to know if there is anything wrong by port 21 or the server? Thank you for your attention and sorry for any inconvenience.

2 1

ansible install tools: tools not in section
by Tamir,Ido 19 Aug '16

19 Aug '16

Hi, I am trying to install tools with the galaxy-tools-playbook, but my tools don’t end up in the sections. I added section test in tool_conf.xml and test2 in shed_tool_conf.xml, and after a restart they are together in integrated_tool_panel.xml. <section id="test" name="test section" version=""> </section> <section id="test2" name="test section 2" version=""> </section> Then I started the ansible playbook with these entries: - name: 'drop_column' owner: 'brigidar' tool_panel_section_id: 'test' - name: 'transpose' owner: 'jmsong' tool_panel_section_id: ‘test2' The install did work, but the tools don’t end up in the sections. I also tried the tool_panel_section_label (without having the label defined in any .xml), and this also placed the tools outside of the sections. ansible log output: ok: [localhost] => {"ansible_facts": {"tools": [{"name": "cutadapt", "owner": "lparsons", "tool_panel_section_id": "ngs_cutadapt"}, {"name": "drop_column", "owner": "brigidar", "tool_panel_section_id": "test"}, {"name": "map_chromosomes", "owner": "pmac", "tool_panel_section_id": "test2"}]}, "changed": false, "invocation": {"module_args": {"_raw_params": "files/vbcf_tool_list.yaml"}, "module_name": "include_vars"}} changed: [localhost] => (item={u'owner': u'brigidar', u'tool_panel_section_id': u'test', u'name': u'drop_column'}) => {"changed": true, "cmd": ["/tmp/venv/bin/python", "install_tool_shed_tools.py", "-y", "name: drop_column\nowner: brigidar\ntool_panel_section_id: test\n", "-a", "7e7c8ec27a0a99acee883d2beea8a60b", "-g", "http://127.0.0.1:9090/"], "delta": "0:00:09.867642", "end": "2016-08-18 13:19:07.389878", "failed": false, "failed_when_result": false, "invocation": {"module_args": {"_raw_params": "/tmp/venv/bin/python install_tool_shed_tools.py -y \"name: drop_column\nowner: brigidar\ntool_panel_section_id: test\n\" -a 7e7c8ec27a0a99acee883d2beea8a60b -g http://127.0.0.1:9090/", "_uses_shell": false, "chdir": "/tmp", "creates": null, "executable": null, "removes": null, "warn": true}, "module_name": "command"}, "item": {"name": "drop_column", "owner": "brigidar", "tool_panel_section_id": "test"}, "rc": 0, "start": "2016-08-18 13:18:57.522236", "stderr": "(1/1) Installing tool drop_column from brigidar to section \"test\" at revision befa27962b5e (TRT: 0:00:00.996404)\r\n\tTool drop_column installed successfully (in 0:00:08.505926) at revision befa27962b5e\r\nInstalled tools (1): [('drop_column', u'befa27962b5e')]\r\nSkipped tools (0): []\r\nErrored tools (0): []\r\nAll tools listed in 'None' have been processed.\r\nTotal run time: 0:00:09.502762", "stdout": "", "stdout_lines": [], "warnings": []} changed: [localhost] => (item={u'owner': u'pmac', u'tool_panel_section_id': u'test2', u'name': u'map_chromosomes'}) => {"changed": true, "cmd": ["/tmp/venv/bin/python", "install_tool_shed_tools.py", "-y", "name: map_chromosomes\nowner: pmac\ntool_panel_section_id: test2\n", "-a", "7e7c8ec27a0a99acee883d2beea8a60b", "-g", "http://127.0.0.1:9090/"], "delta": "0:00:08.982071", "end": "2016-08-18 13:19:17.786115", "failed": false, "failed_when_result": false, "invocation": {"module_args": {"_raw_params": "/tmp/venv/bin/python install_tool_shed_tools.py -y \"name: map_chromosomes\nowner: pmac\ntool_panel_section_id: test2\n\" -a 7e7c8ec27a0a99acee883d2beea8a60b -g http://127.0.0.1:9090/", "_uses_shell": false, "chdir": "/tmp", "creates": null, "executable": null, "removes": null, "warn": true}, "module_name": "command"}, "item": {"name": "map_chromosomes", "owner": "pmac", "tool_panel_section_id": "test2"}, "rc": 0, "start": "2016-08-18 13:19:08.804044", "stderr": "(1/1) Installing tool map_chromosomes from pmac to section \"test2\" at revision e60f92a8e1c8 (TRT: 0:00:01.040988)\r\n\tTool map_chromosomes installed successfully (in 0:00:07.576596) at revision e60f92a8e1c8\r\nInstalled tools (1): [('map_chromosomes', u'e60f92a8e1c8')]\r\nSkipped tools (0): []\r\nErrored tools (0): []\r\nAll tools listed in 'None' have been processed.\r\nTotal run time: 0:00:08.618238", "stdout": "", "stdout_lines": [], "warnings": []}

2 2

GalaxyAdmins online meetup this Thursday: Dynamic Job expansion: Experiences using Makeflow in Galaxy
by Dave Clements 18 Aug '16

18 Aug '16

Hello all, The GalaxyAdmins group will meet *online* this Thursday <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_08_18> at your local time <http://bit.ly/235SYgf>. The August 2016 meetup will feature a presentation by Nick Hazekamp <https://engineering.nd.edu/profiles/nhazekamp> of the University of Notre Dame on Dynamic Job Expansion: Experiences using Makeflow <http://ccl.cse.nd.edu/software/makeflow/> in Galaxy Makeflow <http://ccl.cse.nd.edu/software/makeflow/> is a workflow engine for executing large complex workflows on clusters, clouds, and grids. Nick will talk about Notre Dame's experiences with combining Makeflow and Galaxy. More information on this work is available here: Scaling Up Bioinformatics Workflows with Dynamic Job Expansion: A Case Study Using Galaxy and Makeflow <http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7304…>, Nicholas Hazekamp, Joseph Sarro, Olivia Choudhury, Sandra Gesing, Scott Emrich, Douglas Thain; e-Science (e-Science), 2015 IEEE 11th International Conference on, Aug. 31 2015-Sept. 4 2015, DOI:10.1109/eScience.2015.39 It takes a few minutes to connect to the call <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_08_18#Ca…> so please start connecting in a few minutes early on Thursday. Thanks, and hope to see you on Thursday, Dave Clements and Hans-Rudolf Hotz -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 1

Issue when using xsendfile with galaxy release_16.07
by SAPET, Frederic 18 Aug '16

18 Aug '16

Hello I'm trying the latest Galaxy (git status gives me : # On branch release_16.07) I've set the xsend option in galaxy.ini (in order to run Galaxy behind a web server proxy, and apply all methods described here : https://wiki.galaxyproject.org/Admin/Config/ApacheProxy) apache_xsendfile = True HTML, gff3,tabular files are just fine but the preview of some other files (txt, csv) is broken on the interface. When I want to preview a txt file : Not Found The requested URL /datasets/fd1df6a9052810d5/display/ was not found on this server. And apache says : [Thu Aug 18 15:33:48 2016] [error] [client 192.168.29.12] (13)Permission denied: xsendfile: cannot open file: /softs/bioinfo/galaxy-prod/database/files/000/dataset_19.dat, referer: http: But it looks like this is not really a problem related to access. Indeed, from the same file, I switch it from txt to tabular (edit attributes), the preview works. Are you able to reproduce the behavior ? The same problem occurs on release_16.01. It's fine with the release v15.05 which is my production server right now. Fred

1 0

liftOver output database error
by Sarah DIEHL 18 Aug '16

18 Aug '16

Dear all, I just ran the liftOver tool that comes with Galaxy (https://github.com/galaxyproject/galaxy/blob/dev/tools/extract/liftOver_wra…) and for the "mapped coordinates" output dataset the database shows as "<Cheetah.Template.Template object at 0x10adabd0>" (see attached screenshot). It seems that some of the code that determines the output database doesn't work anymore. This is the associated code in the xml: <data format="input" name="out_file1" label="${tool.name} on ${on_string} [ MAPPED COORDINATES ]"> <actions> <action type="metadata" name="dbkey"> <option type="from_data_table" name="liftOver" key="name" column="1" offset="0"> <filter type="param_value" column="0" value="#" compare="startswith" keep="False"/> <filter type="param_value" ref="to_dbkey" column="2"/> </option> </action> </actions> </data> Does someone have a clue how to fix this? Thanks, Sarah ---- Sarah Diehl HPC System Administrator UNIVERSITÉ DU LUXEMBOURG LUXEMBOURG CENTRE FOR SYSTEMS BIOMEDICINE Campus Belval | Biotech II 6, avenue du Swing L-4371 Belvaux T +352 46 66 44 5360 sarah.diehl(a)uni.lu<mailto:sarah.diehl@uni.lu> http://lcsb.uni.lu<http://lcsb.uni.lu/> ----- This message is confidential and may contain privileged information. It is intended for the named recipient only. If you receive it in error please notify me and permanently delete the original message and any copies. -----

2 1

Galaxy Workflow delete intermediate files
by D K 17 Aug '16

17 Aug '16

Hi Galaxy-devs I've created a workflow that processes some NGS data. Since many of the intermediate files are really large, is it possible to automatically delete (rather than just hide) some of these files? I've seen that you can delete intermediate files that aren't used as inputs to other tools but didn't see a way from within the interface to delete other files created by the workflow that are used as inputs to other tools (but are not needed after the workflow has completed). Thanks!

2 2

Consuming dataset collections
by Steve Cassidy 17 Aug '16

17 Aug '16

Hi all, I’m staring at the discussion of handling dataset collections: http://planemo.readthedocs.io/en/latest/_writing_collections.html but failing to see the solution to my problem. I have a tool that creates a dataset collection, a group of files with names like 1_1308_1_2_092-ch6-speaker16.TextGrid where the 1_1308_1_2_092 part is a unique identifier that I’d like to keep track of. I’ve used a discover_datasets tag in the tool xml file to match my output filenames and extract the designation (1_1308_1_2_092-ch6-speaker16.TextGrid) and the ext (TextGrid). I have another tool that runs a query over these files and generates a single tabular result that will ideally include the identifier in some form. Here’s the command section for that tool: query_textgrids.py --textgrid "${",".join(map(str, $textgrid))}" --tier $tier --regex '$regex' --output_path $output where ‘$textgrid’ is one of my input parameters that has multiple=“true” set so that it can be a dataset collection. That works ok but the input I get are the filenames (dataset_1.dat, etc.) not the name of the datasets. The page above mentions something called the ‘element_identifier’ and gives this funky example: merge_rows --name "${re.sub('[^\w\-_]', '_', $input.element_identifier)}" --file "$input" --to $output; I can’t see what this element_identifier thing is - the suggestion is that it might be the dataset name, but I’m not sure. Also I don’t understand why the command above is doing replacement of whitespace with underscores. If this is the name I’m after, it would seem that I’d need to pass these names along with the textgrid files and then pair them up inside my script - is that what I need to do? All of this cries out to me for a more explicit representation of a dataset collection that my tool can create and consume rather than this hacky treatment of filenames. If I could generate a manifest file of some kind describing my dataset collection then none of this parsing of filenames would be needed. I could also consume the manifest file as well and it could be used for collection level metadata. Is this a silly idea? Anyway, any help with my immediate problem would be appreciated. Thanks, Steve — Department of Computing, Macquarie University http://web.science.mq.edu.au/~cassidy

2 3

problems with 'action type="metadata" name="dbkey"' in tool xml
by Hans-Rudolf Hotz 17 Aug '16

17 Aug '16

Dear All I am trying to move our Galaxy installation to new hardware and at the same time to a new version (the original is "15.10",minor version "2" via mercurial; the new is "16.04" via github): So far, I am making good progress, but I am struggling with a tool which was written by someone else who has left more than 2 years ago. The problem is: it doesn't set the dbkey correctly. I get: <Cheetah.Template.Template object at 0x7f1e40210f50> The tool xml is as follows: <inputs> <param name="sample_file" type="data" format="qsamples,qsamplespaired" label="Sample File" help="set of sequence files created by the 'select sequence files' tool"> </param> <param name="index" type="select" label="Reference Genome" help="all reads will be mapped to this reference"> <options from_data_table="fmi__quasr_genomes"/> </param> ... </inputs> <outputs> <data name="output01" type="data" format="qproject" label="qProject of ${sample_file.name}"> <actions> <action type="metadata" name="dbkey"> <option type="from_param" name="index"/> </action> </actions> </data> </outputs> The user can select a genome (based on the Bioconductor BSgenome packages) from the tab delimited fmi__quasr_genomes table, which looks like: hg19BSg hg19BSg BSgenome Hsapiens (UCSC hg19) BSgenome.Hsapiens.UCSC.hg19 hg18BSg hg18BSg BSgenome Hsapiens (UCSC hg18) BSgenome.Hsapiens.UCSC.hg18 ce10BSg ce10BSg BSgenome Celegans (UCSC ce10) BSgenome.Celegans.UCSC.ce10 ce6BSg ce6BSg BSgenome Celegans (UCSC ce6) BSgenome.Celegans.UCSC.ce6 ... The tool works: - it displays the available genomes (BSgenome Hsapiens (UCSC hg19), hg18BSg hg18BSg BSgenome Hsapiens (UCSC hg18), etc) - once the job is submitted, it uses the right Bioconductor package (BSgenome.Hsapiens.UCSC.hg19, BSgenome.Hsapiens.UCSC.hg18, etc) However, instead of the dbkey (hg19BSg, hg18BSg, etc) I get: <Cheetah.Template.Template object at 0x7f1e40210f50> Has anything changed with the 'action' tag since release 15.10? which makes this simple xml no longer working? I have to admit, it does look different than the example on the wiki (https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Cactions.3E_… ). I tried the code from the wiki which is based on the bowtie_wrapper (https://github.com/galaxyproject/tools-devteam/blob/master/tools/bowtie_wra…) as well: <actions> <action type="metadata" name="dbkey"> <option type="from_data_table" name="fmi__quasr_genomes" column="1" offset="0"> <filter type="param_value" column="0" value="#" compare="startswith" keep="False"/> <filter type="param_value" ref="index" column="0"/> </option> </action> </actions> but it results in the same kind of error: <Cheetah.Template.Template object at 0x7ff82c0e43d0> Thank you very much for any help/comments Regards, Hans-Rudolf -- Hans-Rudolf Hotz, PhD Bioinformatics Support Friedrich Miescher Institute for Biomedical Research Maulbeerstrasse 66 4058 Basel/Switzerland

3 7