March 2015 - galaxy-dev - lists.galaxyproject.org

Saving a Galaxy visualization (e.g. PNG or PDF) into the history / workflow
by Peter Cock 13 Mar '15

13 Mar '15

Hello all, I'm hoping to construct a workflow which produces a simple tally table (using the "Count1" tool, "Count occurrences of each record", which is still shipped with the Galaxy core) and then draws this as a Pie Chart. 1. Using "Count1", generate tally table (a simple two column tabular file, counts and labels as c1 and c2) in the history. 2. Click on the tally table in the history to expand the green box 3. Click on the "Visualize" icon, pick "Charts", click on "Pie Chart" 4. Click "Draw" or the "Configuration" tab. 5. Change donut ratio to zero, show legend to true, and show labels output to true. 6. Click "Draw" or the "Add Data" tab. 7. Under "Select Columns", "Labels", pick c2 8. Under "Select Columns", "Values", keep default c1 9. Click "Draw" 10. Admire interactive Pie Chart 11. Click "Screenshot", offered PNG / SVG / PDF but these are only direct downloads (well, PDF is more complicated). Question: How can I "save" a static snapshot of the Pie Chart as a new history item (other than downloading it locally and then uploading it)? Question: If so, can I then turn this history into a workflow which would recreate the pie chart? As a work around I could write a traditional Galaxy tool which takes as input data files(s) and pre-sets the columns and visualisation settings, and produces static image(s) as output (e.g. PDF and PNG). Peter

2 3

TopHat error
by Scott Szakonyi 11 Mar '15

11 Mar '15

Hello all, I have a user who is getting the following error when analyzing a FASTQ file using TopHat for Illumina. TopHat v2.0.10 [bam_header_read] EOF marker is absent. The input is probably truncated. [bam_header_read] invalid BAM binary header (this is not a BAM file). [bam_index_core] Invalid BAM header. [bam_index_build2] fail to index the BAM file. Error indexin I've tried reloading the tool and all it's dependencies, to no avail. We've been able to run the same FASTQ file successfully on another Galaxy server with identical tool configuration. I'm out of ideas, being relatively new to Galaxy. Has anyone seen a similar error? Can you offer an possible solutions? Thanks! -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

2 1

(no subject)
by hanxl 11 Mar '15

11 Mar '15

If the toxoplasma gondii (me49) genome may add into bowtie2 in galaxy, Thank you. --- 此电子邮件没有病毒和恶意软件，因为 avast! 防病毒保护处于活动状态。 http://www.avast.com

2 1

liftOver functional tests failed
by Anne Pajon 09 Mar '15

09 Mar '15

Dear All, I am still trying to migrate my production server to new hardware and would like to have all functional tests pass before turning off my old server. I am facing an issue with liftOver functional tests, they failed when running 'sh run_tests.sh -id liftOver1' on the command line with these errors: ====================================================================== ERROR: test_tool_000000 (functional.test_toolbox.TestForTool_liftOver1) Convert genome coordinates ( liftOver1 ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it data_list, jobs = galaxy_interactor.run_tool( testdef, test_history ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/base/interactor.py", line 218, in run_tool raise Exception( datasets_object[ 'message' ] ) Exception: {u'type': u'error', u'data': {u'to_dbkey': u'An invalid option was selected, please verify'}} -------------------- >> begin captured logging << -------------------- ====================================================================== ERROR: test_tool_000001 (functional.test_toolbox.TestForTool_liftOver1) Convert genome coordinates ( liftOver1 ) > Test-2 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it data_list, jobs = galaxy_interactor.run_tool( testdef, test_history ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/base/interactor.py", line 218, in run_tool raise Exception( datasets_object[ 'message' ] ) Exception: {u'type': u'error', u'data': {u'to_dbkey': u'An invalid option was selected, please verify'}} -------------------- >> begin captured logging << -------------------- ====================================================================== ERROR: test_tool_000002 (functional.test_toolbox.TestForTool_liftOver1) Convert genome coordinates ( liftOver1 ) > Test-3 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it data_list, jobs = galaxy_interactor.run_tool( testdef, test_history ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/base/interactor.py", line 218, in run_tool raise Exception( datasets_object[ 'message' ] ) Exception: {u'type': u'error', u'data': {u'to_dbkey': u'An invalid option was selected, please verify'}} -------------------- >> begin captured logging << -------------------- ====================================================================== ERROR: test_tool_000003 (functional.test_toolbox.TestForTool_liftOver1) Convert genome coordinates ( liftOver1 ) > Test-4 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 122, in test_tool self.do_it( td ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/functional/test_toolbox.py", line 37, in do_it data_list, jobs = galaxy_interactor.run_tool( testdef, test_history ) File "/mnt/nfs/data/galaxy/galaxy-dist/test/base/interactor.py", line 218, in run_tool raise Exception( datasets_object[ 'message' ] ) Exception: {u'type': u'error', u'data': {u'to_dbkey': u'An invalid option was selected, please verify'}} -------------------- >> begin captured logging << -------------------- But when tested using the GUI, they did pass. I have configured the alignseq.loc and liftOver.loc, they both contained TAB delimited data and file locations have been checked. I’ve downloaded 5.bed into the GUI, used for testing liftOver, and configure the parameters according to those set in the test: <test> <param name="input" value="5.bed" dbkey="hg18" ftype="bed" /> <param name="to_dbkey" value="panTro2" /> <param name="minMatch" value="0.95" /> <param name="choice" value="0" /> <output name="out_file1" file="5_liftover_mapped.bed"/> <output name="out_file2" file="5_liftover_unmapped.bed"/> </test> [cid:5D5398C5-3D95-4074-9A86-81486A8CD20F@cri.camres.org] [cid:45B5289B-C7A8-46CE-A5B2-C879D26B1D18@cri.camres.org] And here are the results: [cid:D4910E74-8A0E-4C58-9D17-46C37DEAF99F@cri.camres.org] Any ideas of what could be that the functional tests on the command line failed? Any help to where to look would be greatly appreciated. Many thanks in advance. Best wishes, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk<mailto:anne.pajon@cruk.cam.ac.uk> | +44 (0)1223 769 631

2 2

Re: [galaxy-dev] Difference between hg pull -u (and) hg update stable?
by Dannon Baker 06 Mar '15

06 Mar '15

Hi Carlos, Yes, there's a difference in hg pull and hg update. Here's a bit of an explanation I found on stackoverflow: https://stackoverflow.com/questions/4394273/whats-the-difference-between-up… 'hg update stable' probably switched you from the default branch to the stable branch (where you probably want to be -- this is a good thing), hence the changes you saw. From now on, only 'hg pull -u' and you should be ok. -Dannon On Fri, Mar 6, 2015 at 12:52 PM Carlos Lijeron <clijeron(a)hunter.cuny.edu> wrote: > Hello all, > > > > Is there any major difference between the commands hg pull -u (and) hg > update stable when trying to update an existing local Galaxy > installation? I tried running hg pull –u as suggested in the IRC channel, > but nothing was updated. Then I ran the hg update stable and got this > result: > > > > 324 files updated, 0 files merged, 86 files removed, 0 files unresolved > > > > I’m not clear about which one should be performed every once in a while. > Any inputs or suggestions will be appreciated. > > > > Thanks. > > > > > Carlos. > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/

1 0

Difference between hg pull -u (and) hg update stable?
by Carlos Lijeron 06 Mar '15

06 Mar '15

Hello all, Is there any major difference between the commands hg pull -u (and) hg update stable when trying to update an existing local Galaxy installation? I tried running hg pull -u as suggested in the IRC channel, but nothing was updated. Then I ran the hg update stable and got this result: 324 files updated, 0 files merged, 86 files removed, 0 files unresolved I'm not clear about which one should be performed every once in a while. Any inputs or suggestions will be appreciated. Thanks. Carlos.

1 0

Changing fasta_to_tabular_converter.py
by Peter Cock 06 Mar '15

06 Mar '15

Hi all, I was just surprised to find what I consider to be a major bug in fasta_to_tabular_converter.py used to convert FASTA into tabular. Consider this toy example: >alpha ACGTAC >beta AGTGTA >gamma with some description AGGTACCA What the converter gives is two columns (title line and sequence), but the '>' is left in: >alpha (tab) ACGTAC >beta (tab) AGTGTA >gamma with some description (tab) AGGTACCA Given just two columns, what I was expecting was: alpha (tab) ACGTAC beta (tab) AGTGTA gamma with some description (tab) AGGTACCA I think this is a bug. In support of this view, I note the user-facing (now in the Tool Shed) removes the '>' symbol: https://toolshed.g2.bx.psu.edu/view/devteam/fasta_to_tabular https://github.com/galaxyproject/tools-devteam/tree/master/tools/fasta_to_t… I have submitted a pull request to address this: https://github.com/galaxyproject/galaxy/pull/11 Note what I really wanted was three columns, the ID, comment and sequence: alpha (tab) (empty) (tab) ACGTAC beta (tab) (empty) (tab) AGTGTA gamma (tab) with some description (tab) AGGTACCA The user-facing tool does support this. I appreciate that changing the built-in implicit converter to give three column output could be a problem for backward compatibility (if anyone has written a workflow using the '>' version of the implicit conversion?), so I can make this conversion explicit in my workflow. Regards, Peter

1 0

Problem to effectively trace logs
by Amandine VELT 05 Mar '15

05 Mar '15

Dear Galaxy dev team, I try to use a tool allowing to sort a bam file with the "samtools sort -n" command. This tool works on BAM test files but when I run it on a real BAM file (of 3.5 G), I got an error named "error" (see attachment) ... this does not help me. In the handler.log, the job is finished normally and I don't find the error line named "error". When I run the galaxy command directly on the server, it works. Is there a log file to follow more precisely the errors? Indeed, I don't find where the "error" comes from and so I can't resolve my problem. Here my log files : more galaxy_20175.e [bam_sort_core] merging from 34 files... more galaxy_20175.ec 0 more galaxy_20175.o [bam_index_core] the alignment is not sorted (HISEQ:123:HBFN8ADXX:1:1101:1341:3979): 24-th chr > 13-th chr [bam_index_build2] fail to index the BAM file. These "errors" are not the problem, because I have the same on the BAM test files and they give no problem (I have a green box at the end of the run, only for tests data of course). Moreover, I got the following lines in my wrapper to be sure that they are only warnings : <stdio> <regex match="fail to index" source="both" level="warning" description="Warning: fail to index the bam file" /> <regex match="the alignment is not sorted" source="both" level="warning" description="Warning: the alignment is not sorted" /> </stdio> more metadata_results_HistoryDatasetAssociation_35900_PbjjDh [true, "Metadata has been set successfully"] more handler0.log galaxy.jobs DEBUG 2015-03-03 10:06:25,824 (20175) Working directory for job is: /galaxy_data/job_working_directory/020/20175 galaxy.jobs.handler DEBUG 2015-03-03 10:06:25,850 (20175) Dispatching to drmaa runner galaxy.jobs DEBUG 2015-03-03 10:06:25,943 (20175) Persisting job destination (destination id: sge) galaxy.jobs.handler INFO 2015-03-03 10:06:25,994 (20175) Job dispatched galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:06:26,584 (20175) submitting file /galaxy_data/job_working_directory/020/20175/galaxy_20175.sh galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:06:26,584 (20175) command is: samtools sort -n /galaxy_data/files/028/dataset_28339.dat /galaxy_data/files/028/dataset_28631.dat ; mv /galaxy_data/files/028/dataset_28631.dat.bam /galaxy_data/files/028/dataset_28631.dat; return_code=$?; cd /data12/galaxy/galaxy-env/galaxy; /data12/galaxy/galaxy-env/galaxy/set_metadata.sh /galaxy_data/files /galaxy_data/job_working_directory/020/20175 . /data12/galaxy/galaxy-env/galaxy/universe_wsgi.ini /galaxy_data/tmp/tmp28hp1G /galaxy_data/job_working_directory/020/20175/galaxy.json /galaxy_data/job_working_directory/020/20175/metadata_in_HistoryDatasetAssociation_35900_vmNjk7,/galaxy_data/job_working_directory/020/20175/metadata_kwds_HistoryDatasetAssociation_35900_IHm6c9,/galaxy_data/job_working_directory/020/20175/metadata_out_HistoryDatasetAssociation_35900_CboGKT,/galaxy_data/job_working_directory/020/20175/metadata_results_HistoryDatasetAssociation_35900_PbjjDh,,/galaxy_data/job_working_directory/020/20175/metadata_override_HistoryDatasetAssociation_35900_XD1a0c; sh -c "exit $return_code" galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:06:26,585 (20175) native specification is: -q galaxy galaxy.jobs.runners.drmaa INFO 2015-03-03 10:06:26,818 (20175) queued as 160573 galaxy.jobs DEBUG 2015-03-03 10:06:26,874 (20175) Persisting job destination (destination id: sge) galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:06:27,907 (20175/160573) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:06:35,538 (20175/160573) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:12:10,200 (20173/160571) state change: job finished normally galaxy.datatypes.metadata DEBUG 2015-03-03 10:12:10,451 loading metadata from file for: HistoryDatasetAssociation 35898 galaxy.jobs DEBUG 2015-03-03 10:12:10,547 job 20173 ended galaxy.datatypes.metadata DEBUG 2015-03-03 10:12:10,551 Cleaning up external metadata files galaxy.jobs.runners.drmaa DEBUG 2015-03-03 10:46:16,161 (20175/160573) state change: job finished normally galaxy.jobs DEBUG 2015-03-03 10:46:16,371 setting dataset state to ERROR galaxy.jobs DEBUG 2015-03-03 10:46:16,464 job 20175 ended I don't know why galaxy give the "setting dataset state to ERROR" line. In attachment, you can see "21 : Bam sorted" on the test file and the "20 : Sort BAM on data 17" on the real file. You can see that it's not the "[bam_index_core] the alignment is not sorted (HISEQ:123:HBFN8ADXX:1:1101:1341:3979): 24-th chr > 13-th chr [bam_index_build2] fail to index the BAM file." line which generate the error. Thank you in advance for your help. Have a nice day. Best regards, Amandine Velt Plateforme Biopuces et Séquençage +33388653529

2 1

Galaxy Update Problem
by 孙兆朕 05 Mar '15

05 Mar '15

Hi all, I installed my Galaxy instance from https://bitbucket.org/galaxy/galaxy-dist/. But I did not use Mercurial to install Galaxy at that time, because I was not farmiliar with Mercurial and failed severed times. The instance imported many tool kits later, as it would be very difficult to restart everthing for me. I tried to use hg command: hg clone https://bitbucket.org/galaxy/galaxy-dist/ cd galaxy-dist hg update stable to make a new copy and copied the ".hg" directory to the former instance and used the "hg update stable" command. Then it reports that there were not files to change. I wonder if there could be any way to update the instance. Thanks, Zhaozhen

2 1

RFC: new param attribute for Galaxy tool XML
by Björn Grüning 04 Mar '15

04 Mar '15

Hi all, we are planning to work on a project to implement a Galaxy fuse based shell. Probably starting with the work from Clare [1]. Next to our Galaxy IPython integration it should attract more bioinformaticians and should offer a new way to interact with Galaxy. This includes moving, deleting datasets, but also executing tools and workflows. For the latter I would like to have some sort of bash auto-completion. Type in your tool/workflow and you will see all the parameters you can/should modify, in addition to your normal help page. Currently the parameter identifiers (<param name="foo">) do not need to be meaningful. In fact many tools invent their own unique names, to identify a parameter in the cheetah section. This name is always unique but it's hard to guess it's meaning from just the name and are also not mappable to the original parameter name. This makes tool execution from the API sometimes hard. I would like to propose a new attribute for all <param> tags. This should specify a unique value that 100% matches the original parameter name of the underlying tool. This attribute could be used to: * automatically enhance the help text of each parameter. Currently best practise it to include this parameter in brackets at the end of each help text. We can do this now automatically, or only show it by mouse over etc ... * In a Galaxy shell, the user could just type in a normal command (-i history1/foo.bam -o history1/bar.sam) and Galaxy shell would be able to map this parameter to the correct <param> tag in Galaxy I greatly appreciate any comments! Thanks, Bjoern [1] https://github.com/claresloggett/gvl_commandline_utilities/blob/master/gala…

3 3