- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] interval format from pileup
by Anton Nekrutenko 12 May '11

12 May '11

Thanks for letting me know. I'm also forwarding this galaxy-dev, so the thread in complete. Thanks, anton Anton Nekrutenko http://nekrut.bx.psu.edu http://usegalaxy.org On May 12, 2011, at 2:30 PM, Katie Hyma wrote: > Hi Anton, > > You're right, sorry for the false alarm. After a more careful inspection of my data I'm pleased to say that I was wrong and that everything with galaxy is fine, it turns out that the company that generated the previous data had generated the error; good think I checked those results, I had just assumed I that my analysis had the error! > > Katie > > On Thu, May 12, 2011 at 10:54 AM, Anton Nekrutenko <anton(a)bx.psu.edu> wrote: > Katie: > "Filter pileup" outputs 0-based coordinates if you use the " Convert coordinates to intervals" option. So the issue may be related to aaChanges tools. Can you give us a specific example, so we can track things down. > > Thanks! > > anton > galaxy team > > > > Anton Nekrutenko > http://nekrut.bx.psu.edu > http://usegalaxy.org > > > > On May 12, 2011, at 10:37 AM, Katie Hyma wrote: > > > Hi all, > > > > I ran into this problem, and thought others might run in to it as well: > > When using bowtie and samtools, positions are formatted with a 1-base offset, whereas interval format requires 0-base offset. > > Recently, I used the 'Filter pileup on coverage and SNPs' tool with the 'convert coordinates to intervals' option, and used the output with the 'aaChanges' tool to detect amino acid changes. > > In the aaChanges output there was an off-by-one discrepancy with a previous analysis, which brought this problem to my attention. > > Should the 'Filter pileup' tool be changed to produce output with 0-base offset when interval format is selected, since interval format is defined as having 0-base output? It seems that others might run into the same problem, and it would be an easy one to miss. > > Although the pileup format is now deprecated, the same problem might occur if/when the change to VCF format occurs, as it also uses 1-base offset. It also may be a problem for other tools that use interval format as an input, expecting a 0-based position. > > > > Best, > > Katie > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > >

1 0

Filter data on any column and missing values
by Peter Cock 12 May '11

12 May '11

Hi all, I have just found a problem using the "Filter data on any column using simple expressions" tool, i.e. files tools/stats/filters.xml and tools/stats/filters.py I have some six column tabular like this, where I have used \t for a tab, and \n for the new lines: #ID\tHMM_Sprob_score\tSP_len\tRXLR_start\tEER_start\tRXLR?\n gi|301087619|ref|XP_002894699.1|\t0.990\t21\t54\t64\tY\n gi|301087623|ref|XP_002894700.1|\t0.997\t23\t\t\tN\n gi|301087628|ref|XP_002894701.1|\t0.000\t24\t\t\tN\n Breakdown of my data: Column 1 - ID, mandatory string Column 2 - HMM_Sprob_score, mandatory float Column 3 - SP_len, mandatory integer Column 4 - RXLR_start, optional integer Column 5 - EER_start, optional integer Column 6 - RXLR?, mandatory string (Y or N) Notice that in my output columns 4 and 5 can be empty or an integer. I'm trying to filter this file using c6=='Y', i.e. column six is a yes. This works (one row output) but Galaxy tells me: Info: Filtering with c6=='Y', kept 100.00% of 4 lines. Skipped 3 invalid lines starting at line #1: "#ID HMM_Sprob_score SP_len RXLR_start EER_start RXLR?" Then if I try to filter using c6=='N', i.e. column six is a no, it fails to work (zero rows of output instead of three) and tells me: kept 0.00% of 4 lines. Skipped 3 invalid lines starting at line #1: "#ID HMM_Sprob_score SP_len RXLR_start EER_start RXLR?" Digging into the code, tools/stats/filters.py gets given the list of column types from Galaxy and (regardless of which columns are to be used) attempts to cast them to integers, floats, etc. It looks like Galaxy has decided that my columns 4 and 5 are integers (based on the first row), and therefore filters.py blindly tries to using int(...) on all these entries and that fails on the empty cells. I see several issues, (a) The filters.py tool only really needs to cast those columns being used for the filter (fairly easy to fix) (b) The galaxy column type detection seems a bit fragile (hard to really fix without looking at all the data). (c) Are there other tools that would break in a similar way to filter.py? Peter

1 2

interval format from pileup
by Katie Hyma 12 May '11

12 May '11

Hi all, I ran into this problem, and thought others might run in to it as well: When using bowtie and samtools, positions are formatted with a 1-base offset, whereas interval format requires 0-base offset. Recently, I used the 'Filter pileup on coverage and SNPs' tool with the 'convert coordinates to intervals' option, and used the output with the 'aaChanges' tool to detect amino acid changes. In the aaChanges output there was an off-by-one discrepancy with a previous analysis, which brought this problem to my attention. Should the 'Filter pileup' tool be changed to produce output with 0-base offset when interval format is selected, since interval format is defined as having 0-base output? It seems that others might run into the same problem, and it would be an easy one to miss. Although the pileup format is now deprecated, the same problem might occur if/when the change to VCF format occurs, as it also uses 1-base offset. It also may be a problem for other tools that use interval format as an input, expecting a 0-based position. Best, Katie

2 1

Upload fails, webapp and job runner running on different machines
by Louise-Amélie Schmitt 12 May '11

12 May '11

Hi everyone, I'm currently trying a new galaxy install with the webapp and the job runner running on different machines, sharing a nfs volume where the galaxy files are, and another one where the data is supposed to be stored, as specified in the file_path and new_file_path values in the universe_wsgi files. (one for the webapp and one for the runner, as stated in the doc) Both instances run properly with no error message but when I want to upload a file from wherever in a library, here's what happens: The file's row appears in the library with the "uploading" note. The job is registered in the database. And that's all. Nothing changes afterwards. Nothing is saved anywhere. This can't be a file-too-big issue since the test file I'm working with weighs 3.5kB. I probably stupidly missed something but I really don't see where the problem might be. Any idea? Cheers L-A

3 3

Using wget/curl with Tool Shed?
by Peter Cock 11 May '11

11 May '11

Hi all, I'm trying to download and install something from the Galaxy Tool Shed http://community.g2.bx.psu.edu/ on to our sever. To do this, I'd like to copy/paste the URL of a tool's tar ball (or individual XML file etc) from my browser running on my desktop to a terminal window logged in to our server to give to wget or curl. e.g. This URL will give me bam_to_bigwig_0.0.1.tar.gz if run in a web browser: http://community.g2.bx.psu.edu/common/download_tool?cntrller=tool&id=3b1686… However, the URL actually goes to an HTML page then does some javascript magic to redirect to the real file. I can't easily use this URL with wget or curl. Is this something that you could fix in the website code? For now I can just download the file on my desktop, then scp it to the server (or ssh -X into the server and run a browser from there), but this is an unnecessary hurdle I feel. Regards, Peter

2 1

Community server bug and request
by Assaf Gordon 11 May '11

11 May '11

Tiny bug: When viewing a file inside a tarball, the content-type is forced to "text/plain". Works for most files, but not with image files (example: the "Sequence Logo" tool has a jpg image in the tarball, the browser will display it as text/plain with binary characters). This is done in "./lib/galaxy/webapps/community/controllers/tool.py", function "view_tool_file", line 206: trans.response.set_content_type( 'text/plain' ) regardless of the file type. The complicated solution would be to use the "mimetype" library (http://docs.python.org/library/mimetypes.html) or python-magic (https://github.com/ahupp/python-magic) but a simpler workaround would be to just check the file name for "jpg/png/gif" extensions and change the content-type appropriately. Feature request: A way to easily link to tools, so I can send to them to other people or point them in the right direction. Example: Ross's Web logo tool (great tool, BTW), If I want to tell where it is, the link is: http://community.g2.bx.psu.edu/tool/browse_tools?sort=name&f-state=approved… or direct downloading the tarball is: http://community.g2.bx.psu.edu/common/download_tool?cntrller=tool&id=c2f60f… Both are probably not stable links (if a new version is uploaded). -gordon

2 1

External representation of workflow
by Frederick van Staden 11 May '11

11 May '11

Hello everyone, I need some advice on how to represent the output of all the different tools inside a workflow in an external application. I am looking into using galaxy as a digital evidence processing tool. It is very important that i can access the intermediate sets of data created by every tool's output. Some have suggested the actual tools should populate a database and that the secondary(viewer) application should just retrieve the applicable data form there. Any suggestions? Any input would be greatly appreciated. Regards Fred

2 1

"Hide Datasets" bug in workflow
by Assaf Gordon 11 May '11

11 May '11

Hi, There's a small bug with the "hide dataset" button in the workflow editor - once any dataset is marked as "output" (by clicking on the star icon), there's no way to show all the datasets, even if the user manually un-checked every star icon for every workflow step. meaning: If I have a workflow that *used* to have one step marked as "output", unmarking it (so that no dataset is marked is "output", implying all datasets should be retained in the history) doesn't work - galaxy insists on hiding all the other datasets. Steps to reproduce: 1. Import this workflow: http://main.g2.bx.psu.edu/u/foobar/w/hiddendatasetstest2 2. Edit this workflow. Notice that only the last two steps are marked as "output". don't change anything, close with (or without) saving. 3. Run this workflow. Notice that Steps 1,2,3,5,6 have "Action: Hide this dataset." - as expected. Only steps 4,7 are marked as "output". No need to actually execute the workflow. 4. Edit the workflow again. Remove the "output" mark from steps 4,7 ("Add Column" and "compute"). No dataset is marked as output - at least from the user POV, it's implied that all datasets should be kept in the history (just like when editing a new workflow). Save the workflow, close the workflow. 5. Run this workflow. Notice that steps 1,2,3,5,6 STILL have "Action: hide this dataset" - this is the bug. So the only workaround is to run the workflow, then "extract workflow from history" to get a "clean" copy of the workflow without any datasets hidden. Thanks to Marek Kudla for meticulously experimenting with this issue. -gordon

2 1

Email notification - is workflow is the only way?
by Paul, Rohit (NIH/NCI) [C] 11 May '11

11 May '11

>From what I understand about Galaxy so far, in order to have a user notified that their job has completed, a workflow has to be created in which email notification is chosen as a "Step Action". For the project that I'm currently working on, I need to have a link to one particular workflow appear in the Tools left-hand side panel/frame for every user that logs in to Galaxy. They shouldn't have to import the specific workflow — it should automatically be listed. I know that I can create links in the Tools panel, but it seems that in order to access a workflow (via direct URL such as: '/workflow/run?id=12345') the logged in user has to either be the creator of the workflow or the creator has to have explicitly "shared" the workflow with the user. Is there a way to grant "share" permission on a workflow to all logged-in users, so that I can list it as a Tool link and it shows up for all users, current and those who may register in the future? Or is there some other approach I can/should take to enable job completion notification for all logged-in users? -Rohit

2 1

fasta width formatter
by Owen Hardy 11 May '11

11 May '11

Hi, I set up a local instance on OS 10.6 and was using some fasta tools but noticed fasta width formatter does not execute. Although it shows in my local web interface I do not see width formatter in the tools.conf.xml, nor does the python script appear in /tools/fasta_tools with the others. I also browsed source from central and dist in fasta_tools and do not see it there either...Thanks for any help. An error occurred running this job:/bin/sh: fasta_formatter: command not found zcat: /Users/Owen/Galaxy/galaxy-dist/database/files/000/dataset_190.dat.Z: No such file or directory

3 2