July 2011 - galaxy-dev - lists.galaxyproject.org

Error moving tool-data dir ... bowite indexes missing
by Bossers, Alex 26 Oct '11

26 Oct '11

On our up-to-date galaxy_central version I have successfully moved the database/files directory to mounted storage. After adapting the universe_wsgi.ini it works fine after galaxy restart. Now the tool-data dir becomes bigger and bigger and I wanted to move that to mounted storage as well. I moved the tool-data dir to the other location, made sure file permissions were ok and adapted the universe_wsgi.ini tool_data_path to match the new tool-data location. Galaxy restarts and the loc files are read (since blast recognises its databases). However, the bowtie indexes are not read since they do not show up in the NGS bowtie mapper for illumina..... what could be wrong? When I restore the universe_wsgi.ini it works all fine. I am puzzled. Alex

2 1

Re: [galaxy-dev] [galaxy-user] problem with displaying tracks from Galaxy
by Nate Coraor 03 Oct '11

03 Oct '11

Sergei Ryazansky wrote: > Hello all, > > we have the UCSC genome browser mirror as well as Galaxy mirror. The > Galaxy has a feature enabling a user to display the data at UCSC genome > browser as custom tracks. I have configured the galaxy to display the data > to our UCSC browser mirror but it doesn't work properly: after the > redirecting to genome browser page the "redirected to non-http(s): /root" > error message is appeared. At the same time displaying Galaxy data at > official UCSC works excellent. What are the possible reasons of it? > Thank you in advance! Hi Sergei, If your Galaxy server is behind a proxy server serving via https, have you set the following header: RequestHeader set X-URL-SCHEME https Please see the "SSL" section of the ApacheProxy page for more information: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ApacheProxy --nate > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

4 7

optional integer value in tool config
by Sarah Diehl 15 Sep '11

15 Sep '11

Hello, in the config of a tool I want to integrate into Galaxy I would like to have a parameter with the following properties: <param name="var" type="integer" label="Var" value="50" optional="true" min="10" max="100" /> So it should be possible to leave this field empty, but when you choose to enter a number it should be between 10 and 100. The problem now is that Galaxy refuses to accept it when this field is empty. After encountering this problem I tried to solve the issue within Cheetah and tried the following: #if $var.value < 10 or $var.value > 100 #set $var.value = "" #end if Galaxy however throws the following error when loading the tool: galaxy.tools ERROR 2011-07-04 14:04:14,231 error reading tool from path: path/to/tool.xml Traceback (most recent call last): File "/galaxy/galaxy_server/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( os.path.join( self.tool_root_dir, path ) ) File "/galaxy/galaxy_server/lib/galaxy/tools/__init__.py", line 131, in load_tool tree = util.parse_xml( config_file ) File "/galaxy/galaxy_server/lib/galaxy/util/__init__.py", line 104, in parse_xml tree = ElementTree.parse(fname) File "/galaxy/galaxy_server/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/galaxy/galaxy_server/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 583, in parse parser.feed(data) File "/galaxy/galaxy_server/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 1242, in feed self._parser.Parse(data, 0) ExpatError: not well-formed (invalid token): line 230, column 21 The token it is complaining about is the <. I searched the Cheetah documentation and also Google but was not able to come up with a solution. Could anyone of you knowledgeable guys help me find a solution to this, please? I would be very grateful! Thanks in advance! Best regards, Sarah

3 3

Installing tools with dependencies using mi-deployment scripts
by Mattias de Hollander 12 Sep '11

12 Sep '11

Hello, I am trying to add tools (PyCogent, Qiime) to a Cloudman image using the mi-deployment scripts. Qiime depends on PyCogent and it needs to be accessible on the python path, otherwise I get an error: PyCogent not installed but required. (Is it installed? Is it in the current user's $PYTHONPATH or site-packages?) Is there a way to add the paths in env.sh to the sys.path during installation using tools_fabfile.py? All the tools are in /mnt/galaxyTools/tools. I tried the DependencyManager from galaxy but then I only receive the path to the env.sh file: In [43]: dependency_manager.find_dep('blast') Out[43]: ('/mnt/galaxyTools/tools/blast/2.2.25/env.sh', '/mnt/galaxyTools/tools/blast/2.2.25', '2.2.25') Is there a smarter way to add files to the PYTHONPATH during the execution of tools_fabfile.py? Thanks, Mattias -- Bioinformatician Netherlands Institute of Ecology (NIOO-KNAW) Wageningen, the Netherlands

2 6

Automatically removing items from history
by Kempenaar, M (med) 08 Sep '11

08 Sep '11

Hello, Is it possible to let a tool in a workflow manage the users history? For instance, I have a tool that takes a variable number of user-uploaded CSV files as input and merges them into one file. Before running this tool, the user thus has a number of CSV files in the history which I would like to 'replace' with the single merged file. I was hoping that when I remove a file from disk that it would also be removed from the history, but that only results in a 'The resource could not be found.' error. Alternatively, is it possible to create a composite datafile/history item from uploaded files? In any case, I would like to remove the separately uploaded files from the users history which makes selecting files for the next step(s) easier. (this is a bit of a followup for: http://gmod.827538.n3.nabble.com/Creating-new-datatype-with-variable-number…) One other question, is there (any) documentation available on the Galaxy API or a set of example files? Thanks for your input! Regards, Marcel. De inhoud van dit bericht is vertrouwelijk en alleen bestemd voor de geadresseerde(n). Anderen dan de geadresseerde(n) mogen geen gebruik maken van dit bericht, het niet openbaar maken of op enige wijze verspreiden of vermenigvuldigen. Het UMCG kan niet aansprakelijk gesteld worden voor een incomplete aankomst of vertraging van dit verzonden bericht. The contents of this message are confidential and only intended for the eyes of the addressee(s). Others than the addressee(s) are not allowed to use this message, to make it public or to distribute or multiply this message in any way. The UMCG cannot be held responsible for incomplete reception or delay of this transferred message.

4 4

value_label tool input param attribute doesn't work as documented
by Leandro Hermida 07 Sep '11

07 Sep '11

Hi, Looking at the example on the Tool Config Syntax wiki page: Example Assume that the tool includes an input parameter named "database" which is a select list ( e.g., assume the following inputs ): Toggle line numbers<http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#> 1 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <inputs> 2 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <param format="tabular" name="input" type="data" label="Input stuff"/> 3 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <param type="select" name="database" label="Database"> 4 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <option value="alignseq.loc">*Human (hg18)*</option> 5 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <option value="faseq.loc">Fly (dm3)</option> 6 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> </param> 7 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> </inputs> Assume that the user selects the first option in the $database select list. Then the following will ensure that the tool produces a tabular data set whose associated history item has the label "Blat on *Human (hg18)*". Toggle line numbers<http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#> 1 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> <outputs> 2 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> <data format="input" name="output" label="Blat on ${database.value_label}" /> 3 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> </outputs> It shows that .value_label attribute should produce the string in between the <option> tags, not the actual value. Why does this not work? When I use it it produces the value, or am I doing something wrong? regards, Leandro

2 2

Re: [galaxy-dev] [galaxy-bugs] Error message when trying to dowload Manhattan plot using the web version
by Ross 06 Sep '11

06 Sep '11

Hi, Alison I've made some fixes to the Manhattan/QQ tool (galaxy-central revision #dacf97a6c663) so it now handles your data - thanks again for discovering these edge cases - the revised code is now available and working on http://test.g2.bx.psu.edu/ where your data produces plots. In particular X in your data, and for good measure Y and Mt are now translated into 23,24 and 25 respectively so the code can now cope with those inputs and they should appear in linear order in the Manhattan x axis The floppydisk link (download) problem remains to be fixed as it's more complex. We only update the main server every few weeks so the changes I just made will take a while to appear there. Meanwhile, please try the revised tool by making yourself an account and then uploading your data to http://test.g2.bx.psu.edu/ - please let me know how you get on? Of course, the (?FDR adjusted) p values in your data are a challenge - typically, the Manhattan and QQ plots are run with unadjusted p values which are expected to be distributed uniformly 0-1 under the null... FYI, when I use the grouping tool on your data, I see that chromosomes 20,21 and 22 are absent from the file - is this expected? Group with a count gives: 1 22910 10 9990 11 12764 12 13413 13 13068 14 11319 15 11175 16 8483 17 12017 18 10073 19 6927 2 18332 3 16173 4 15401 5 15259 6 15487 7 15217 8 12485 9 14560 X 3951 Thanks again for helping us improve Galaxy... On Mon, Jul 18, 2011 at 10:52 PM, Alison Harrill <AHarrill(a)thehamner.org> wrote: > The first time I uploaded the file, I loaded in the -logP values > instead. After that, I changed the file to raw p values and reuploaded. > Do you think it would help to upload the file again, but change the > filename? I just wonder if that original file didn't get cleared out. > > Alison > > -----Original Message----- > From: Ross [mailto:ross.lazarus@gmail.com] > Sent: Monday, July 18, 2011 8:51 AM > To: Alison Harrill > Subject: Re: [galaxy-bugs] Error message when trying to dowload > Manhattan plot using the web version > > ok - thanks. These are not raw pvalues and your data is definitely > causing the Manhattan plot to fail. > > This will take me a little time to figure out - I'll get back to you > in a day or two - thanks again for reporting this problem > > On Mon, Jul 18, 2011 at 10:16 PM, Alison Harrill > <AHarrill(a)thehamner.org> wrote: >> Good morning, >> >> Sorry for taking a while to get back to you (I was out of town for the >> weekend). I set permissions to share the dataset with you. Please let > me >> know if you see an obvious workaround. >> >> Thank you, >> Alison >> >> >> Alison Harrill, Ph.D. >> Research Investigator >> Institute for Drug Safety Sciences >> Hamner Institutes for Health Sciences >> phone: (919) 226-3142 >> fax: (919) 226-3150 >> >> >> >> -----Original Message----- >> From: Ross [mailto:ross.lazarus@gmail.com] >> Sent: Sunday, July 17, 2011 7:25 AM >> To: Alison Harrill >> Subject: Re: [galaxy-bugs] Error message when trying to dowload >> Manhattan plot using the web version >> >> Can you please share a history with me (use this email address) so I >> can take a look please? >> >> I'm talking about links on the results page where you (should!) see >> some graphs - when you click the history item display ('eye') icon - >> not the history item save ('floppy disk') icon >> >> On Sun, Jul 17, 2011 at 9:15 PM, Alison Harrill > <AHarrill(a)thehamner.org> >> wrote: >>> Hi, Ross, >>> >>> Thanks so much for kindly following up with me. I tried the > workaround >>> you suggested, but it appears to not be working for me. > Right-clicking >>> prompts me to save a file called display.htm and when I open the > file, >>> it just reads "Unable to remove temporary library download archive > and >>> directory". >>> >>> Best wishes, >>> Alison >>> >>> -----Original Message----- >>> From: Ross [mailto:ross.lazarus@gmail.com] >>> Sent: Sunday, July 17, 2011 4:09 AM >>> To: Alison Harrill >>> Subject: Re: [galaxy-bugs] Error message when trying to dowload >>> Manhattan plot using the web version >>> >>> Hi, Alison - I wanted to be sure you are able to get your images from >>> the tool - the proper fix is going to take a while so I hope those >>> links are working for you? >>> >>> cheers... >>> >>> >>> On Sat, Jul 16, 2011 at 9:39 AM, Ross <ross.lazarus(a)gmail.com> wrote: >>>> Hi, Alison, >>>> >>>> Thanks for reporting this error - I just confirmed that you have >> found >>>> a bug - the download function is currently not working correctly for >>>> composite datatypes in the history. >>>> >>>> Fortunately, the images themselves have download links under each > one >>>> and can be saved - eg on a PC or linux workstation, while hovering >>>> over the '(Click to download image Armitagep_manhattan.png)' press >> the >>>> right side mouse button and choose "save link as.." - you will be >> able >>>> to choose a location to save the image file on your local >> workstation. >>>> I hope this "work around" allows you to get on with your work - > sorry >>>> for the inconvenience and thanks again for finding and reporting > this >>>> error - we will fix it as soon as we can. >>>> >>>> >>>> >>>> On Sat, Jul 16, 2011 at 6:46 AM, Alison Harrill >>> <AHarrill(a)thehamner.org> wrote: >>>>> Greetings, >>>>> >>>>> I have a question about generating a Manhattan/QQ plot using the > web >>> tool. I >>>>> was able to successfully run the plot (the result box is green), > but >>> when I >>>>> try to download the results, I get a message that says "Unable to >>> remove >>>>> temporary library download archive and directory." Is there another >>> way to >>>>> retrieve the data? >>>>> >>>>> Thank you, >>>>> >

2 1

Galaxy megablastdb indexer script proposal
by Jacob Israel Cervantes Luevano (LANGEBIO) 06 Sep '11

06 Sep '11

Hi Galaxy Dev, Nice to meet you. My name is Jacob and I am from México , Actually I'm working in LANGEBIO (Laboratorio Nacional de Genomica para la Biodiversidad) at Bioinformatics. I would like to contribute with galaxy dev team, I developed shell script to set up reference genomes by megablast. the linux shell script that I prepared uses your current prepdb.py script. the mini pipeline flow-work: 1. read input values: * fasta filename (nucleotide) * blast format: blast format: F->nucleotide, T->protein * database output name * fasta sequence IDs format (ncbi-gbk , other) where: ncbi-gbk means something like ">gi|gi-number|gb|accesión|locus" other means something like ">my own IDs" 2. if fasta sequence IDs are like ncbi format(ncbi-gk) then execute prepdb.py python script and generate new fasta file then go step 4 example: >gi|gi-number|gb|accesión|locus 3. if fasta sequence IDs has diferent (other) identifiers set to galaxy format first then execute prepdb.py python script then go step 4 >my own identifier 4. to format database using formatdb blast utility 5. setting up Megablast Galaxy Loc File 5. delete temp files 6. done Usage example: query help: ./set_galaxy_blastdb.sh -help format blast nucleotide dabatabase yeast, set galaxy loc file ./set_galaxy_blastdb.sh -fasta /media/xlinux/genomes/galaxy/palomero/megablast/yeast_nt.fasta -p F -n yeastdb -fastadesc ncbi-gk when script finished we have following results: $ls /media/xlinux/genomes/galaxy/palomero/megablast/ blastdb.loc new.yeast_nt.fasta yeastdb.nhr yeastdb.nin yeastdb.nsq yeast_nt.fasta How can I contribute with this shell script ? Thanks in advance -- Jacob http://www.langebio.cinvestav.mx/bioinformatica/jacob -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

2 1

TEMP variable in cluster install
by Shantanu Pavgi 06 Sep '11

06 Sep '11

We have configured galaxy to work with our SGE cluster using drmaa job runner interface. We are using 'unified method' for this install and both TEMP environment variable and new_file_path in universe_wsgi.ini file have been configured correctly. However, we are seeing some errors where local /tmp space on compute nodes is being referenced by the galaxy tools. Specifically we saw it mentioned in error messages from following tools: * bwa_wrapper and upload tools: 'No space left on device: /tmp...' * sam_to_bam tool: 'Error sorting alignments from /tmp/filename..' Shouldn't it be referencing TEMP environment variable or new_file_path configuration value? Is it getting overridden by TMP or TMPDIR variables in python wrapper code? Has anyone else experienced similar issue? -- Thanks, Shantanu.

5 6

[API] managing histories
by Louise-Amélie Schmitt 31 Aug '11

31 Aug '11

Hello everyone, I'm currently trying to automate data loading and pre-processing through Galaxy's API, and I will need to delete and share histories at some point. I know the API is still very new but is there a way to do that by any chance? Thanks, L-A

2 2