January 2015 - galaxy-dev - lists.galaxyproject.org

Running jobs as real user and extra_file_path
by Louise-Amélie Schmitt 28 Apr '17

28 Apr '17

Hi everyone, I just wanted to ask how the extra_file_path is handled in case of job running as the real user since the file_path is only writable by the galaxy user. Any clue? Thanks, L-A

3 3

Re: [galaxy-dev] how to create users through Galaxy API?
by Martin Čech 02 Sep '16

02 Sep '16

Hi, you specify username, password and email in the body (payload) of the POST as Key:Value pairs. Code from the API method: ( https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/ga…) username = payload[ 'username' ] email = payload[ 'email' ] password = payload[ 'password' ] There are also other conditions that need to be fulfilled (e.g. user creation has to be turned on in the configuration) - you will find these when you look at the source code of the method (because the documentation is not perfect yet, sorry). M. On Sun, Dec 29, 2013 at 11:22 PM, xlwang <xlwang_0903(a)163.com> wrote: > hi, > I still have some confusion. > post /api/users , when i use postman, it seems like this: > > > No other parameters? How to confirm the new user's information?Such as > user name, password. > Can you give me a example please?Thank you! > > At 2013-12-30 06:04:15,"Martin Čech" <marten(a)bx.psu.edu> wrote: > > Hello, > > to create a user through API you send a POST request to /api/users > > More information about User API here: > > > https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.… > > Martin > > > On Fri, Dec 27, 2013 at 5:11 AM, xlwang <xlwang_0903(a)163.com> wrote: > >> hello, >> I want to create users through Galaxy API.For example, I can get users >> information through url like this(use postman): >> http://localhost:8080/api/users?key=my_key >> But how to create a user? >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > > > >

6 7

error loading files into galaxy
by Hakeem Almabrazi 04 Mar '16

04 Mar '16

Hi, I started getting the following error whenever I try to load a file into Galaxy local. Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/tools/data_source/upload.py", line 8, in <module> from galaxy import eggs ImportError: cannot import name eggs I will appreciate if someone can tell me what could happen to cause such issue and how to resolve it. Regards,

2 1

Upload issue in local installation
by Batsal Devkota 05 Feb '16

05 Feb '16

I installed galaxy locally in a linux server. However, I cannot upload the files (no matter how small, I have tried few kb size fasta files). When I try to upload, the link to the file shows up in the History and gets a new number (purple box). When I click on the link I get 'Dataset is uploading' forever. In the terminal window where I start galaxy, I get the following error report: 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET / HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /root/tool_menu HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:33 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:40 -0400] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://redhat:8080/root/tool_menu" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /tool_runner/upload_async_create HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/tool_runner/upload_async_message" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:04 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:08 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" The last line keeps going for ever... writes new line every 4 secs. I am stuck and don't know where to look. Please help. Batsal.

6 7

Rename output from a repeat
by Ulf Schaefer 27 Aug '15

27 Aug '15

Hi all We frequently use the syntax below to rename outputs of workflows that we run in batch. It is convenient to have sample names from fastqs carried over to sams, bams, vcfs, etc. #{input1 | basename}.bam This does not seem to be working for inputs that are in repeats, e.g. the VelvetOptimiser. Does anybody know if there is a syntax to make this work, maybe #{repeatname[0].input1 | basename}.bam ? Thanks a lot for your help Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

3 2

Object-Store, setting filetypes crashes Galaxy
by bjoern.gruening＠googlemail.com 21 Jul '15

21 Jul '15

Hi, I have configured to use the hierarchical object store but as soon as I try to reset the filetpye of a dataset Galaxy is crashing with: galaxy.objectstore DEBUG 2014-06-12 14:39:21,180 Using preferred backend 'files3' for creation of MetadataFile 5963 132.230.153.57 - - [12/Jun/2014:14:39:20 +0200] "POST /datasets/966f24627ef70c12/edit HTTP/1.1" 500 - " http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Fire fox/29.0" Error - <type 'exceptions.OSError'>: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' URL: http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py', line 295 in edit trans.app.datatypes_registry.set_external_metadata_tool.tool_action.execute( trans.app.datatypes_registry.set_external_metadata_tool, trans, incoming = { 'input1':data }, overwrite = False ) #overwrite is False as per existi ng behavior File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 18 in execute overwrite, history, job_params ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 79 in execute_via_app kwds = { 'overwrite' : overwrite } ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 717 in setup_external_metadata shutil.copy( dataset.metadata.get( meta_key, None ).file_name, metadata_temp.file_name ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 575 in file_name self._filename = abspath( tempfile.NamedTemporaryFile( dir = self.tmp_dir, prefix = "metadata_temp_file_" ).name ) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 454 in NamedTemporaryFile (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 235 in _mkstemp_inner fd = _os.open(file, flags, 0600) OSError: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' I have attached my object_store_conf.xml file. Thanks, Bjoern

5 7

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)
by Dooley, Damion 20 Apr '15

20 Apr '15

I was successful getting library dataset selection and form submission to happen within tool form tool form input <param type="library_data" ...> - Yay! But it appears one last fix (besides the "url: "/visualization/list_libraries",' bug) is needed to enable history tool item to be RERUN. So if a galaxy dev can tackle this, great! Otherwise I'll have to drop this approach :<( ***************** For others who may need... tool form's param: <param name="library_datasets" type="library_data" label="Reference Bin file" help="Select one reference bin at a time from the popup window's Reference Bin library."/> <command> tag contents.... #if $library_datasets -B " #for $i, $dataset in enumerate($library_datasets) #$dataset.get_file_name() $dataset.id #end for " #end if Successfully submits either the full file path of selected dataset, or its id. ***************** However trying to rerun this history tool item gets us an error (not related to the above <command> contents): Error Traceback: View as: Interactive | Text | XML (full) ⇝ AttributeError: 'int' object has no attribute 'id' .... Module galaxy.tools.parameters.basic:146 in value_to_basic view >> return self.to_string( value, app ) Module galaxy.tools.parameters.basic:2406 in to_string app <galaxy.app.UniverseApplication object at 0x59a0710> ldda 331 self <galaxy.tools.parameters.basic.LibraryDatasetToolParameter object at 0x7fced01d2090> value [331] >> return [ldda.id for ldda in value] AttributeError: 'int' object has no attribute 'id' which is in /lib/galaxy/tools/parameters/basic.py: class LibraryDatasetToolParameter( ToolParameter ): """ Parameter that lets users select a LDDA from a modal window, then use it within the wrapper. """ ... def to_string( self, value, app ): if not value: return value return [ldda.id for ldda in value] "Value" is getting an array of dataset ids, whereas this method is expecting value to be a list of dataset objects. I tried changing ldda.id to just "id" but found other parts of LibraryDatasetTool code were broken as they too were expecting an object. Maybe this is related to the tool job's history detail link showing "Reference Bin file" as a string label of an object?: Input Parameter Value Note for rerun BLAST results as XML 240: megablast Pasted Entry vs 16SMicrobial.fasta Reference Bin file [<galaxy.model.LibraryDatasetDatasetAssociation object at 0x7fcee04d4350>] Tabular Report Column Labels Short name HTML Report template templates.html_report ******************************** The other bug: 2) Tool_form.mako has a hardcoded url that doesn't work for galaxy installs that have a prefix so that needs adjustment: $(".add-librarydataset").click(function() { var link = $(this); $.ajax({ url: "/visualization/list_libraries",

2 3

Visualization in anonymous mode ?
by Loraine Guéguen 09 Apr '15

09 Apr '15

Hi, I'd like to use the visualization in anonymous mode, but the visualize icon on the dataset is not available in anonymous mode (although it works fine when connected). Is it possible to activate the dataset visualization in anonymous mode ? Thanks, Loraine -- Loraine Guéguen Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences) Service Informatique et Bioinformatique Station Biologique de Roscoff FR 2424 CNRS UPMC Place Georges Teissier CS 90074 29688 Roscoff cedex tel: ++33 (0)2 98 29 56 46 (interne : 415) http://abims.sb-roscoff.fr/ http://www.sb-roscoff.fr/

2 5

test/base/twilltestcase.py - Converting local (test-data) bam to sam failed
by Peter Cock 02 Apr '15

02 Apr '15

Hi all, I have a query about a failing tool test on the Tool Shed, where it seems Galaxy is trying to convert both the expected BAM output and the tool's BAM output into SAM for comparison - but this fails. https://toolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-27 02:13:51 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/peterjc/sa…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 40, in do_it self._verify_outputs( testdef, test_history, jobs, shed_tool_id, data_list, galaxy_interactor ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 80, in _verify_outputs galaxy_interactor.verify_output( history, jobs, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/interactor.py", line 66, in verify_output self.twill_test_case.verify_hid( outfile, hda_id=hid, attributes=attributes, dataset_fetcher=fetcher, shed_tool_id=shed_tool_id ) File "/tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 2341, in verify_hid raise AssertionError( errmsg ) AssertionError: History item different than expected, difference (using diff): ( /tmp/shed_tools/toolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/2d303f2e09e0/samtools_depad/test-data/sam_spec_padded.depad.bam v. /tmp/buildslave/buildbot-install-test-main-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpExQWRD/tmpSknKqLsam_spec_padded.depad.bam ) Converting local (test-data) bam to sam failed (and same error on the second test) The same occurs on the Test Tool Shed (with a very slightly updated version of this tool), although with a more concise traceback: https://testtoolshed.g2.bx.psu.edu/view/peterjc/samtools_depad 2015-01-25 08:21:59 ... Tool id: samtools_depad Tool version: samtools_depad Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterj…) Stderr: Traceback: Traceback (most recent call last): File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 268, in test_tool self.do_it( td ) File "/tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 67, in do_it raise e JobOutputsError: History item different than expected, difference (using diff): ( /tmp/shed_tools/testtoolshed.g2.bx.psu.edu/repos/peterjc/samtools_depad/8e1e3a1ecad7/samtools_depad/test-data/sam_spec_padded.depad.bam v. /tmp/buildslave/buildbot-install-test-test-tool-shed-py27/build/test/install_and_test_tool_shed_repositories/repositories_with_tools/tmp/tmpNvCGJi/tmphX3A_Fsam_spec_padded.depad.bam ) Converting local (test-data) bam to sam failed Both tests should produce a BAM file matching the provided test-data/sam_spec_padded.depad.bam example. The error comes from test/base/twilltestcase.py, p = subprocess.Popen( args='samtools view -h -o "%s" "%s"' % ( temp_local.name, local_name ), shell=True ) assert not p.wait(), 'Converting local (test-data) bam to sam failed' Running this command locally at the terminal seems to work fine, both with samtools 0.1.19 and 1.1. $ ~/bin/samtools_0.1.19 view -h -o temp.sam sam_spec_padded.depad.bam $ more temp.sam @HD VN:1.5 SO:coordinate @SQ SN:ref LN:45 ref 516 ref 1 0 45M * 0 0 AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT * r001 99 ref 7 30 8M2I4M1D3M = 37 41 TTAGATAAAGGATACTG * * 768 ref 8 30 1M * 0 0 * * CT:Z:.;Warning;Note=Ref wrong? r002 0 ref 9 30 3S6M1P1I4M * 0 0 AAAAGATAAGGATA * PT:Z:1;4;+;homopolymer r003 0 ref 9 30 5H6M * 0 0 AGCTAA * NM:i:1 r004 0 ref 16 30 6M14D5M * 0 0 ATAGCTTCAGC * r003 2064 ref 29 30 6H5M * 0 0 TAGGC * NM:i:0 r001 147 ref 37 30 9M = 7 -41 CAGCGGCAT * NM:i:1 $ ~/bin/samtools_1.1 view -h -o temp.sam sam_spec_padded.depad.bam [galaxy@ppserver test-data]$ more temp.sam @HD VN:1.5 SO:coordinate @SQ SN:ref LN:45 ref 516 ref 1 0 45M * 0 0 AGCATGTTAGATAAGATAGCTGTGCTAGTAGGCAGTCAGCGCCAT * r001 99 ref 7 30 8M2I4M1D3M = 37 41 TTAGATAAAGGATACTG * * 768 ref 8 30 1M * 0 0 * * CT:Z:.;Warning;Note=Ref wrong? r002 0 ref 9 30 3S6M1P1I4M * 0 0 AAAAGATAAGGATA * PT:Z:1;4;+;homopolymer r003 0 ref 9 30 5H6M * 0 0 AGCTAA * NM:i:1 r004 0 ref 16 30 6M14D5M * 0 0 ATAGCTTCAGC * r003 2064 ref 29 30 6H5M * 0 0 TAGGC * NM:i:0 r001 147 ref 37 30 9M = 7 -41 CAGCGGCAT * NM:i:1 Which version of samtools is on the $PATH on the test system? Could the test system have run out of disk space again? The tests are passing on my local development instance (before and after updating to latest galaxy-central), and on TravisCI using the latest Galaxy: https://travis-ci.org/peterjc/pico_galaxy/builds/48485673 Thanks, Peter

2 5

GalaxyAdmins Online Meetup on 19 February
by Dave Clements 31 Mar '15

31 Mar '15

Hello all, The first GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> meetup of 2015 <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_02_19> will happen online on Thursday, 19 February <http://bit.ly/1yZJSkN>. GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> is a special interest group for Galaxy community members who are responsible for Galaxy installations. A summary of both the user and admin/developer 2014 Galaxy Community Questionnaires <https://wiki.galaxyproject.org/News/2014Questionnaire> will be presented by Dave Clements <https://wiki.galaxyproject.org/DaveClements>, a *Galaxy Project Update* will be offered by a Galaxy Team member <https://wiki.galaxyproject.org/GalaxyTeam>, and Hans-Rudolf Hotz <https://wiki.galaxyproject.org/HansrudolfHotz> will lead a short discussion on *GalaxyAdmins direction*. Thanks to everyone for letting us know what dates and times worked best for you. This time and day of the week worked for a remarkable 91% of respondents. We are looking forward to starting this group up again, Hans-Rudolf Hotz and Dave Clements -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

3 4