January 2012 - galaxy-dev - lists.galaxyproject.org

dbfetch/wsdbfetch from EMBL-EBI
by Rodrigo Lopez 10 Jan '12

10 Jan '12

Hi, Happy New Year 2012! Having read the public Galaxy server now has ENA/SRA data available I'm wondering if EBI's dbfetch or wsdbfetch is an option to consider as well. Dbfetch has a web interface and is also supported using Web Services: Web: http://www.ebi.ac.uk/Tools/dbfetch/ and WS: http://www.ebi.ac.uk/Tools/webservices/services/dbfetch (both SOAP and REST are supported). The REST interface is available from: http://www.ebi.ac.uk/Tools/webservices/services/dbfetch_rest Briefly, dbfetch provides access to 47 datasets. These include mainstream nucleotide (EMBL-Bank, ENA/* & RefSeq) and protein sequence databases (UniProt and UniParc), ENSEMBL, ENSEMBL Genomes, PUBMED/MEDLINE, PDBe, etc. It returns entries and documents in flatfile, raw, hypertext, fasta and current XML formats. Yours sincerely, Rodrigo Lopez EMBL-EBI/ES

1 0

Re: [galaxy-dev] (no subject)
by Hans-Rudolf Hotz 10 Jan '12

10 Jan '12

On 01/10/2012 06:31 AM, Vidya .H.K wrote: > Hi, > I think macs is exceutable with galaxy user. >> >> what happens if you execute "macs_wrapper.py" on the command line? Regards, Hans PS: Please send all follow-up with a cc to the mailing list ("reply-all"). >> On 01/09/2012 05:04 AM, Vidya .H.K wrote: >>> Hi, >>> >>> I am new user to galaxy. I have installed galaxy on the Linux local >>> server >>> for ChIP-seq analysis. While running MACS, an error is flagged >>> "An error occurred running this job: /bin/sh: macs: not found" . >>> I have set the path for macs in /bin. But still the error persists. It >>> would be great if any one help me resolve this problem. >>> >> >> Hi Vidya >> >> Is macs executable for the galaxy user (ie the user the galaxy server is >> running as) ? >> >> Regards, Hans >> >>> Thanks in advance. >>> >>> >>> Regards >>> vidya >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> > > > Regards > H.K.VIDYA > PGDB-10-10-33 > ----------------------------------------- > Institute of Bioinformatics and Applied Biotechnology, Bangalore >

1 0

error on viewing workflow details via API
by Leon Mei 09 Jan '12

09 Jan '12

Hi, I received an error when I was trying to view the details of existing workflows through API. I can see 4 workflows owned by myself successfully. But viewing the details of any workflow got me a 500 error. ./display.py <my_key> http://galaxy.nbic.nl/galaxy/api/workflows Collection Members ------------------ #1: /galaxy/api/workflows/e037fdb493429c2a name: test id: e037fdb493429c2a #2: /galaxy/api/workflows/13120e62d0fbb985 name: imported: Partial Workflow for Student Project (1) id: 13120e62d0fbb985 #3: /galaxy/api/workflows/c9468fdb6dc5c5f1 name: GAPSS SNP calling id: c9468fdb6dc5c5f1 #4: /galaxy/api/workflows/3f5830403180d620 name: GAPPS CAGE id: 3f5830403180d620 4 element(s) in collection ./display.py <my_key> http://galaxy.nbic.nl/galaxy/api/workflows/e037fdb493429c2a HTTP Error 500: Internal Server Error 500 Internal Server Error The server has either erred or is incapable of performing the requested operation. Here is the error message in paster.log: /* galaxy.web.framework ERROR 2012-01-06 17:46:53,349 Uncaught exception in exposed API method: Traceback (most recent call last): File "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/framework/__init__.py", line 147, in decorator return simplejson.dumps( func( self, trans, *args, **kwargs ) ) File "/opt/galaxy/prog/galaxy-2011-12-5/lib/galaxy/web/api/workflows.py", line 68, in show inputs[step.id] = {'label':step.tool_inputs['name'], 'value':""} TypeError: 'NoneType' object is unsubscriptable <my_IP> - - [06/Jan/2012:17:46:53 +0200] "GET /galaxy/api/workflows/e037fdb493429c2a?key=<my_key> HTTP/1.1" 500 - "-" "Python-urllib/2.6" */ Any suggestion on how to solve this problem? Thanks, Leon -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing Skype: leon_mei Mobile: +31 6 41709231

2 3

Using workflows with steps with several inputs through the API
by Carlos Borroto 09 Jan '12

09 Jan '12

Hi, Looking at workflow_execute.py, it seems to me there can be only one input for step, is this correct? or is there a wait to map more than one input and match them to an specific position in a particularly step? This is the workaround I found. I created 'Input Dataset' boxes for all of my inputs, then I linked them to the step where they are needed. This is actually very convenient if the same input is used in more than one step, as when I run the workflow manually I only need to select each input once. I tried doing this, as each 'Input Dataset' would have his own step. This is an example workflow: http://main.g2.bx.psu.edu/u/cjav/w/fastq-joiner---input-datasets Initially I ran into this issue: $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows fcfa9ae643d37c32 'Fastq joiner - Input Datasets' '1=ldda=b4c5be159c8057c2' '2=ldda=ea0e32961d454539' HTTP Error 400: Bad Request "Workflow cannot be run because an expected input step '517620' has no input dataset." I then found you can do: $ ./display.py my_key http://main.g2.bx.psu.edu/api/workflows/fcfa9ae643d37c32 Member Information ------------------ url: /api/workflows/fcfa9ae643d37c32 inputs: {'517620': {'value': '', 'label': 'Left-hand Reads'}, '517622': {'value': '', 'label': 'Right-hand Reads'}} id: fcfa9ae643d37c32 name: Fastq joiner - Input Datasets I see inputs get different keys and I can use them when executing the workflow: $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows fcfa9ae643d37c32 'hist_id=9fbfd3a1be8b7f8e' '517620=hda=8d89405199368bb0' '517622=hda=5c93f30078e5ac25' Response -------- {'outputs': ['f7c4b0900d8f3b1b'], 'history': '9fbfd3a1be8b7f8e'} This worked perfectly. I still think it would be easier to be able to use the step number, but this works. Lastly, I did run into one issue where I had to import the datasets firsts, as trying to use ldda like this: $ ./workflow_execute.py my_key http://main.g2.bx.psu.edu/api/workflows fcfa9ae643d37c32 'Fastq joiner - Input Datasets' '517620=ldda=b4c5be159c8057c2' '517622=ldda=ea0e32961d454539' Response -------- {'outputs': ['44c033fad737acc5'], 'history': '9fbfd3a1be8b7f8e'} I got this error in the history: input data 1 (file: /galaxy/main_database/files/002/780/dataset_2780871.dat) was deleted before the job started This didn't happen in our local instance using galaxy-central. Thanks, Carlos

2 2

(no subject)
by Vidya .H.K 09 Jan '12

09 Jan '12

Hi, I am new user to galaxy. I have installed galaxy on the Linux local server for ChIP-seq analysis. While running MACS, an error is flagged "An error occurred running this job: /bin/sh: macs: not found" . I have set the path for macs in /bin. But still the error persists. It would be great if any one help me resolve this problem. Thanks in advance. Regards vidya

2 1

Python/R Script Not Integrating with Galaxy Correctly
by Christopher Fletez-Brant 09 Jan '12

09 Jan '12

Hello Galaxy-Dev List, I am having some issues in getting a script to work correctly in Galaxy. My script is written in Python, uses RPy2 to pass some data to the R environment and then calls a custom R function; the output of this function, and of the greater Python/R tool, is a file. I am having 2 problems: the first is that R cannot find the R file holding the function, and the second is that Galaxy cannot find the output file. Before I elaborate on these issues, I should point out that when called from the command line this script executes exactly as I intend it to. First issue: When called by Galaxy, the Python portion of the script runs fine, and the RPy2 portion devoted to setting up the R environment so that the script can run seems to execute just fine. The script fails when it tells R to call the custom function, with the following output: Python output: Traceback (most recent call last): File "/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh/svm_train_classify_cv.py", line 865, in <module> if __name__=='__main__': main() File "/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh/svm_train_classify_cv.py", line 851, in main robjects.r.source(current) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/rpy2/robjects/functions.py", line 82, in __call__ return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/rpy2/robjects/functions.py", line 34, in __call__ res = super(Function, self).__call__(*new_args, **new_kwargs) rpy2.rinterface.RRuntimeError: Error in file(file, "r", encoding = encoding) : cannot open the connection R output: /Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106/rocprc.R [1] "/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106" Error in file(file, "r", encoding = encoding) : cannot open the connection In addition: Warning message: In file(file, "r", encoding = encoding) : cannot open file '/Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/106/rocprc.R': No such file or directory The first line of the R output is the return of a call to the Python function os.path.abspath('rocprc.R') (note that os.path.realpath('rocprc.R') returns the same path), while the second line, for comparison, is a call to R's getwd() - the R file is in the same directory as the R environment's setting, but for some reason R cannot find the file. Because this works from the command line, I suspect that I am not taking some Galaxy setting into account. Second issue: Trying to identify even the kind of error, I hard-coded the directory in which I know the R file to reside in a call to R's setwd(). This time, the script runs correctly, but there is no file returned by Galaxy. The script execution ends with the following output message: empty format: pdf, database: ? Info: [1] "/Users/kipperfletez-brant/galaxy-dist/tools/myTools/svm_enh" Watching the file in which the R script is held, I know that the output PDF is saved in the same file as that specified in my tool's XML, which, for other tools I've written, has resulted in an output file. Looking at the log, the following DEBUG message is written: galaxy.jobs DEBUG 2012-01-09 10:16:49,899 finish(): Could not move /Users/kipperfletez-brant/galaxy-dist/database/job_working_directory/111/rocprc.pdf to /Users/kipperfletez-brant/galaxy-dist/database/files/000/dataset_111.dat as directed by from_work_dir As with the first issue, I suspect there is some sort of Galaxy setting I am unaware of. Generally, because both issues relate to finding files not directly created/referenced by the caller Python script, I suspect that there will be some commonality between the resolutions of the two issues, but I am need help to see that. Thank you for reading, Kipper Fletez-Brant

1 0

Re: [galaxy-dev] broken pipe when integrating VCF-merge from vcf-tools
by James Boocock 09 Jan '12

09 Jan '12

Hi, We are attempting to integrate vcftools specifically vcf-merge into galaxy. We have two or files that we are attempting to merge, the command when run through terminal does not give a broken pipe, but when galaxy runs the command a broken pipe is given at the beginning and the start of the execution. The output from the merge appears to be exactly the same. But the broken pipe error causes galaxy to go read on the output wondering if anyone can help that would be much appreciated. The shell script is pasted below. _SHELL SCRIPT THAT GALAXY RUNS _ 1 #!/bin/bash 2 # @Date 9/01/2012 3 # @Author Ed Hills, James Boocock 4 # 5 # This file will take a white space separated list of file names, 6 # and run the vcf-merge tool and merge all files into a new file. 7 # 8 # INPUTS 9 # $1 = First input File 10 # $2 = Second input File 11 # $N = Extra input files 12 13 FILE_LIST="" 14 15 cat $1 | bgzip -c > ~tmp1.vcf.gz 16 cat $2 | bgzip -c > ~tmp2.vcf.gz 17 18 tabix -p vcf ~tmp1.vcf.gz 19 tabix -p vcf ~tmp2.vcf.gz 20 21 FILE_LIST="~tmp1.vcf.gz ~tmp2.vcf.gz" 22 23 if [ $# > 2 ] 24 then 25 for ((i=3; i <= $#; i++)) 26 do 27 eval EXTRA_FILE=\$${i} 28 cat $EXTRA_FILE | bgzip -c > ~tmp${i}.vcf.gz 29 tabix -p vcf ~tmp${i}.vcf.gz 30 FILE_LIST="${FILE_LIST} ~tmp${i}.vcf.gz" 31 done 32 fi 33 34 perl ~/galaxy-dist/tool-data/shared/vcfperltools/vcf-merge ${FILE_LIST} 2> /dev/null 35 36 rm -f ~tmp* 37 _ERROR MESSAGE_ Warning: The AC tag(s) will not be merged correctly for multiallelic sites. To be handled correctly, please redefine with Number=. or set the environment variable DONT_FIX_VCF40_AG_TAGS=0. gzip: stdout: Broken pipe Using column name 'HG00096' for ~tmp1.vcf.gz:HG00096 gzip: stdout: Broken pipe Using column name 'NA06986' for ~tmp2.vcf.gz:NA06986 Cheers James.

1 0

Re: [galaxy-dev] broken pipe when running VCF-merge.
by James Boocock 09 Jan '12

09 Jan '12

Hi, We are attempting to integrate vcftools specifically vcf-merge into galaxy. We have two or files that we are attempting to merge, the command when run through terminal does not give a broken pipe, but when galaxy runs the command a broken pipe is given at the beginning and the start of the execution. The output from the merge appears to be exactly the same. But the broken pipe error causes galaxy to go read on the output wondering if anyone can help that would be much appreciated. The shell script is pasted below. _SHELL SCRIPT THAT GALAXY RUNS _ 1 #!/bin/bash 2 # @Date 9/01/2012 3 # @Author Ed Hills, James Boocock 4 # 5 # This file will take a white space separated list of file names, 6 # and run the vcf-merge tool and merge all files into a new file. 7 # 8 # INPUTS 9 # $1 = First input File 10 # $2 = Second input File 11 # $N = Extra input files 12 13 FILE_LIST="" 14 15 cat $1 | bgzip -c > ~tmp1.vcf.gz 16 cat $2 | bgzip -c > ~tmp2.vcf.gz 17 18 tabix -p vcf ~tmp1.vcf.gz 19 tabix -p vcf ~tmp2.vcf.gz 20 21 FILE_LIST="~tmp1.vcf.gz ~tmp2.vcf.gz" 22 23 if [ $# > 2 ] 24 then 25 for ((i=3; i <= $#; i++)) 26 do 27 eval EXTRA_FILE=\$${i} 28 cat $EXTRA_FILE | bgzip -c > ~tmp${i}.vcf.gz 29 tabix -p vcf ~tmp${i}.vcf.gz 30 FILE_LIST="${FILE_LIST} ~tmp${i}.vcf.gz" 31 done 32 fi 33 34 perl ~/galaxy-dist/tool-data/shared/vcfperltools/vcf-merge ${FILE_LIST} 2> /dev/null 35 36 rm -f ~tmp* 37 _ERROR MESSAGE_ Warning: The AC tag(s) will not be merged correctly for multiallelic sites. To be handled correctly, please redefine with Number=. or set the environment variable DONT_FIX_VCF40_AG_TAGS=0. gzip: stdout: Broken pipe Using column name 'HG00096' for ~tmp1.vcf.gz:HG00096 gzip: stdout: Broken pipe Using column name 'NA06986' for ~tmp2.vcf.gz:NA06986 Cheers James.

1 0

multiple file input not taken into account in workflows
by Christophe Antoniewski 09 Jan '12

09 Jan '12

Hi We recently deployed a new galaxy instance for production, under Apache2 and postgresql. Although I am not familiar with postgresql, I followed the instructions in the wiki and it seems overall to work. However, there is at least an obvious issue with this new instance : when running a workflow with multiple files in input, the job starts well with the first file and proceeds until completion of the whole workflow, then stops, not taking care of the other files defined in the multiple input, and, more irritating, not reporting any bug (at least in the front). The local galaxy instance we've been running for 1 year now under SQlite and the internal web server, is doing well in contrast, so that I suspect a misconfiguration in postgresql, primarily, (I don't really see how Apache could pervert this aspect, but I am not so much a geek so I may be wrong) I tried to find traces of the same issue in the forum but did not find something yet. Any help / suggestion are welcome Cheers -- Christophe

2 2

Galaxy @ PAG 2012
by Dave Clements 07 Jan '12

07 Jan '12

Hello all, Plant and Animal Genome XX (PAG2012) starts in San Diego on January 14. If you are attending PAG then you will have (at least) a half-dozen opportunities to learn more about Galaxy and how it is being used to support research. There will be workshops introducing Galaxy, and on deploying Galaxy in the cloud, and talks and posters on using Galaxy and several local Galaxy deployments. See the Galaxy @ PAG2012 page (http://wiki.g2.bx.psu.edu/Events/PAG2012) and the conference website (http://www.intlpag.org/) for more information. Dannon Baker and Dave Clements from the Galaxy Team ( http://wiki.g2.bx.psu.edu/Galaxy%20Team) will also be there throughout the meeting. Please feel free to ask us questions, or just introduce yourself. Hope to see you in San Diego, Dave Clements and Dannon Baker -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

1 0