- galaxy-dev - lists.galaxyproject.org

attributes associted with xml inputs and outputs
by Chorny, Ilya 20 Oct '11

20 Oct '11

Is there a list of associated attributes such as $input.ext, $input.dbkey and $input.id? Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 1

Trouble getting setting dbkey of output data
by Chorny, Ilya 19 Oct '11

19 Oct '11

I am having trouble setting the dbkey of an output data set. The input and output, respectively, is shown below. I copied this from the bowtie_wrapper.xml but the difference is that I do not have a condition on selecting the reference genome. My tool requires a .loc file. The tools completes correctly but the dbkey is not set. Any thoughts on what I may be doing wrong? Thanks, Ilya <param name="input2" type="select" label="Select a reference genome"> <options from_data_table="RNASEQ_contams"> <filter type="sort_by" column="2" /> <validator type="no_options" message="No indexes are available for the selected input dataset" /> </options> </param> <data format="sam" name="output2" label="${tool.name} on ${on_string}:Output Sam File" /> <actions> <action type="metadata" name="dbkey"> <option type="from_data_table" name="RNASEQ_contams" column="1" offset="0"> <filter type="param_value" column="0" value="#" compare="startswith" keep="False"/> <filter type="param_value" ref="input2" column="0"/> </option> </action> </actions> Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 3

Re: [galaxy-dev] Accept-Ranges: bytes for
by Kip Bodi 19 Oct '11

19 Oct '11

Hi Manuel, Thanks! I've already taken the easy route and switched to nginx, but if I need to switch back to Apache I'll build from source as you've described below. Best, -Kip On Mon, Oct 17, 2011 at 1:14 PM, Manuel Kohler <manuel.kohler(a)bsse.ethz.ch>wrote: > Hi Kip, > I had exactly the same problem you had, but I finally got it working. So > there is some hope :-) I am running Galaxy behind an Apache with RHEL 6.1. > The default Apache provided by RH was 2.2.15 which gave me exactly the > same error. So I upgraded to 2.2.21 using the src rpms from Fedora16 > which did not help. But with being on 2.2.21 I patched the Apache server > with the following file which finally solved the byte range issue: > > http://svn.apache.org/viewvc?view=revision&revision=1177080 > > The bug is described here: > https://issues.apache.org/bugzilla/show_bug.cgi?id=51878 > Galt Barber from UCSC Genome Browser already commented on this. > > To be able to build Apache on your own you can do something like this: > > # rpm -i /root/httpd-2.2.21-1.fc16.src.rpm > # cd /root/rpmbuild/SPECS > # rpmbuild -ba httpd.spec > > We edited the httpd.spec file and removed the dependency to > systemd-units so the following should work then. > > # rpm -U > /root/rpmbuild/RPMS/x86_64/httpd-2.2.21-1.el6.x86_64.rpm > /root/rpmbuild/RPMS/x86_64/httpd-devel-2.2.21-1.el6.x86_64.rpm > /root/rpmbuild/RPMS/x86_64/httpd-tools-2.2.21-1.el6.x86_64.rpm > /root/rpmbuild/RPMS/x86_64/mod_ssl-2.2.21-1.el6.x86_64.rpm > /root/rpmbuild/RPMS/noarch/httpd-manual-2.2.21-1.el6.noarch.rpm > > It might be helpful to work together with a Sysadmin. > > Cheers > Manuel > > -- > Manuel Kohler > Center for Information Sciences and Databases (C-ISD) > Department of Biosystems Science & Engineering (D-BSSE) > ETH Zurich, Mattenstr. 26 (1058), CH-4058 Basel, +41 61 387 3132 > Switzerland > > > > On 14. October 2011 6:56 PM, Kip Bodi wrote: > > Hi Brad, > > > > They're binary BAM files. I followed the section below, but perhaps I > > missed something. Thanks for the clarification on the Paste server, and > > I think I'll try nginx next if I can't get forwarding through Apache to > > work. > > > > Cheers, > > > > -Kip > > > > On Thu, Oct 13, 2011 at 1:17 PM, Brad Chapman <chapmanb(a)50mail.com > > <mailto:chapmanb@50mail.com>> wrote: > > > > > > Kip; > > Are these binary BAM or BigWig type files you are having trouble > > displaying at UCSC? I don't believe the Paste server has support for > the > > byte-range requests that UCSC needs, and you need to send files > through > > the proxy server. I've done this with nginx so don't have direct > Apache > > experience here, but the 'Sending files using Apache' section here > > should get it working: > > > > http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy > > > > Brad > > > > > > > Perhaps this is a distribution issue. For those who have gotten > > the "Display > > > at UCSC" link to work on a local server or workstation, what linux > > > distribution and version did you use? Has anyone used this > > successfully with > > > Red Hat Enterprise Linux 6.1? > > > > > > Best, > > > > > > -Kip > > > > > > On Fri, Oct 7, 2011 at 6:28 PM, Bodi, Kip <Kip.Bodi(a)tufts.edu > > <mailto:Kip.Bodi@tufts.edu>> wrote: > > > > > > > Hi all, > > > > > > > > I have a local galaxy instance, proxied by Apache, and am > > trying to get the > > > > 'display at UCSC' link to work. I've followed all of the > > instructions and am > > > > nearly there, but now I'm getting this response from UCSC when > > it tries to > > > > load the track: > > > > > > > > "Byte-range request was ignored by server. Expected Partial > > Content 206." > > > > etc. > > > > > > > > I think the error is because the PasteWSGIServer isn't allowing > > byte-range > > > > requests. When I look at the header, "Accept-Ranges: bytes" is > > not present: > > > > [galaxy@galaxy galaxy_dist]$ curl --head localhost > > > > HTTP/1.1 200 OK > > > > Date: Fri, 07 Oct 2011 22:23:49 GMT > > > > Server: PasteWSGIServer/0.5 Python/2.6.6 > > > > content-type: text/html; charset=UTF-8 > > > > Set-Cookie: > > > > > > > galaxysession=c6ca0ddb55be603a4e98b4f13a4db8bfb8635f0aa576697114d40b13a8355efe2cf7a075ef81d769; > > > > expires=Thu, 05-Jan-2012 17:23:49 GMT; Max-Age=7776000; Path=/; > > Version=1 > > > > Vary: Accept-Encoding > > > > Connection: close > > > > > > > > Whereas on another server, "Accept-Ranges: bytes" appears in > > the header: > > > > [galaxy@galaxy galaxy_dist]$ curl --head apollo.med.tufts.edu > > <http://apollo.med.tufts.edu>< > > > > http://apollo.med.tufts.edu> > > > > HTTP/1.1 403 Forbidden > > > > Date: Fri, 07 Oct 2011 22:22:42 GMT > > > > Server: Apache/2.2.21 (Fedora) > > > > Accept-Ranges: bytes > > > > Content-Length: 4609 > > > > Connection: close > > > > Content-Type: text/html; charset=UTF-8 > > > > > > > > I'm running RHEL6 with the latest version of galaxy-dist. I've > > tried > > > > manually enabling 'accept-ranges' but haven't had any luck. Any > > suggestions? > > > > Can I enable it somehow in universe_wsgi.ini? > > > > > > > > Best, > > > > > > > > -Kip > > > > > > > > > > -- > > Kip Lord Bodi Jr. > > Tufts University School of Medicine > > 136 Harrison Avenue, South Cove 502 > > Boston, MA 02118 > > 617-636-3763 <tel:617-636-3763> > > kip.bodi(a)tufts.edu <mailto:kip.bodi@tufts.edu> > > www.tucf.org <http://www.tucf.org> | genomics.med.tufts.edu > > <http://genomics.med.tufts.edu> > > > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > > -- Kip Lord Bodi Jr. Tufts University School of Medicine 136 Harrison Avenue, South Cove 502 Boston, MA 02118 617-636-3763 kip.bodi(a)tufts.edu www.tucf.org | genomics.med.tufts.edu

1 0

Re: [galaxy-dev] Possible to browse local filesystem
by Greg Von Kuster 19 Oct '11

19 Oct '11

Hello Alessandro, I misunderstood the issue you were raising in your initial email, but now understand based on the information you've supplied below. When you use the "Upload a directory of files" option to upload files to a data library, all of the files in the directory are uploaded (there is currently no way to choose which files in the directory to upload and which files to ignore). So in order to keep from uploading files more than once, the files that have been previously uploaded will need to be removed from the subdirectory. This same behavior applies to the "Upload files from filesystem paths" option if the path is pointing to a subdirectory of files. The ability to select which files to upload and which to ignore is an enhancement that we'll add to our development roadmap, but it may be a while before we can get to it. Sorry for my initial misunderstanding on this, as well as the inconvenience this may cause. Greg Von Kuster On Oct 14, 2011, at 11:21 AM, alessandro albiero wrote: > Dear Greg, > > many thank for your help. I follow all your instruction but my problem is always the same. > Attache you can find 3 images where you can see the problem. > > In the figure 1, I'm uploading the files from my folder using sym link. As you can see in the figure 2 there is my file (all_homo.fa). > After that I added a new file in my folder and when I upload again this folder, all the files are reuploaded. See the figure 3, here you can see that the file all_homo.fa is present 2 times and there is also a new sam file. > > I'd like to avoid the duplications and upload only the new file not previously uploaded. > > Probably I'm doing something wrong but I am not able to understand what. > > All the best > > Alessandro > > 2011/10/12 Greg Von Kuster <greg(a)bx.psu.edu> > Allesandro, > > Are your users making the correct selection in the Copy data into Galaxy? select list? If they choose "Link to files without copying into Galaxy" as shown below, you should see the behavior you want. The default selection is "Copy files into Galaxy", so if they leave the default selection, copies of the files will be made. > > <upload.tiff> > > On Oct 12, 2011, at 12:10 PM, alessandro albiero wrote: > >> Dear Greg, >> >> thank you for your answer. I followed the wiki but I did not solve my problem. >> I setted these parameters on the universe_wsgi.ini: >> >> user_library_import_dir = /u/galaxy/userimport >> >> Then I created the /u/galaxy/userimport directory with a series of sub-dirs named as the login users. >> I done this to allow our user to load into galaxy their files. In this case every user put its data in its dir on /u/galaxy/userimport (or using symlinks) and then load them on galaxy. >> >> My problem is that galaxy duplicate the files; for example: >> >> 1) User "A" place a file "FILE1" on its dir and it load the dir on galaxy. On galaxy he sees "FILE1" >> 2) The user "A" place another file in its dir ("FILE2") and re-load the dir on galaxy. On galaxy he sees "FILE1", "FILE1", "FILE2" >> >> It seems that galaxy re-loads all the files of the user dir without checking the duplications. >> We would like to avoid this duplications. >> >> We found that removing the old files from the user directory solves the problem. Is there a simpler way to do this (without removing the files) ? >> >> >> All the best >> >> Alessandro >> >> >> >> 2011/10/11 Greg Von Kuster <greg(a)bx.psu.edu> >> Hello Alessandro, >> >> You can upload files from filesystem paths to a Galaxy data library without copying the files into Galaxy's configured file directory. This approach should provide what you're looking to do. See our wiki at http://wiki.g2.bx.psu.edu/Admin/Data%20Libraries/Uploading%20Library%20Files for all of the details. >> >> Greg Von Kuster >> >> On Oct 11, 2011, at 6:45 AM, alessandro albiero wrote: >> >>> Dear all, >>> >>> I have a problem with galaxy and I have already tried to write on ther user list.... >>> My problem concern the upload on galaxy of new files. >>> >>> We are setting up a galaxy istance on our server (HPC Cluster with torque and PBS) for internal use. >>> Our problem is that we manage only NGS data (SOLiD and Illumina Data) that are already stored on the server. >>> We tried to set up the user directory in the universe_wgsi.ini file to allow to any user tu upload their local files, but adding new file and re-uploading the dir we found duplicated files (it seems that galaxy re-upload every time all the files ignoring if some are already present). This method is OK for us but how can we avoid the file duplications ? >>> >>> In a presentation found on web (blankenberg_pittsburg_2001), we saw that there is the possibility to automatically upload the sequenced reads on Galaxy (and let them available to all our user). Where can we find information for doing this ? >>> >>> >>> Thank in advance >>> >>> All the best >>> >>> Alessandro >>> >>> >>> >>> >>> -- >>> ****************************** >>> Alessandro Albiero >>> Bioinformatics Lab >>> CRIBI-Padua University >>> Viale G. Colombo, 3 >>> 35121 Padova-Italy >>> >>> http://genomics.cribi.unipd.it >>> http://genomics.cribi.unipd.it/index.php/User:Albiero >>> Phone. +39. 049 827-6165 >>> Fax. +39. 049 827-6159 >>> ******************************** >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> >> > > Greg Von Kuster > Galaxy Development Team > greg(a)bx.psu.edu > > > > > <Fig_1.png><Fig_2.png><Fig_3.png> Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

how do I pass the URL of the input and output files to my tool wrapper?
by Chorny, Ilya 18 Oct '11

18 Oct '11

Anyone know if it possible to pass the url (shown in the eye ball) to a tool wrapper? Thanks, Ilya Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny(a)illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>

2 4

Unaligned BAM as a (sub) format in Galaxy?
by Peter Cock 18 Oct '11

18 Oct '11

Hello all, Further to recent discussions about dealing with large FASTQ files, and the short term advantages of gzipped FASTQ as a filetype in Galaxy (or gzipped files in general, https://bitbucket.org/galaxy/galaxy-central/issue/666/ ), we also discussed the merits of unaligned BAM files. It occurred to me this could be defined in Galaxy as subclass of the existing BAM definition, with a simple sniffer based on running "samtools idxstats" to find out if there are any mapped reads or not. This would then make it easy for tools capable of aligning SAM/BAM to list unaligned-bam as a Galaxy input format (avoiding complications of realigning previously aligned reads). You could also have a matching aligned BAM subclass, then tools like SNP detectors should insist on an aligned BAM file for input. Does this sound reasonable? Peter

1 0

Re: [galaxy-dev] [galaxy-user] Problem starting galaxy on Cygwin
by Dannon Baker 18 Oct '11

18 Oct '11

Oren, I've moved this thread over to the galaxy-dev mailing list since it deals with a local installation. Users on this list are more likely to have an answer for this type of question. That said, we don't officially support running Galaxy under Cygwin (or Windows at all), and I don't know of anyone running such an instance. If you *do* manage to get things working on Windows, however, it'd certainly be useful to know what you had to go through to make it happen. One relatively painless option for running galaxy under windows, and what I'd suggest, would be to use the virtual machine we provide at usegalaxy.org/vm. Let me know if you have any other questions, and good luck! -Dannon On Oct 12, 2011, at 02:50 PM, Oren Livne <livne(a)uchicago.edu> wrote: Dear All, There are related posts but it is unclear from them how to resolve the following. I'm trying to start a Galaxy server on Cygwin and it fails when trying to fetch an egg relying on NumPy. Could you kindly advise how to resolve the error? This uses the Cygwin python 2.6.5. I tried to point Cygwin to the Windows python (2.7.2 in my case) but that was even worse, the Mako egg threw a KeyError: 'Mako' exception up front. Thank you so much, Oren -------- $ python --version Python 2.6.5 oren@oren-PC /cygdrive/c/web/galaxy-central $ sh run.sh --reload Some eggs are out of date, attempting to fetch... Fetched http://eggs.g2bx.psu.edu/Mako/Mako-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Whoosh/Whoosh-0.3.18-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Tempita/Tempita-01-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/lrucache/lrucache-0.2-py26.egg Fetched http://eggs.g2.bx.psu.edu/NoseHTML/NoseHTML-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pexpect/pexpect-2.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/amqplib/amqplib-0.6.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/PasteDeploy/PasteDeploy-1.3.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebHelpers/WebHelpers-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 30, in <module> c.resolve() # Only fetch eggs required by the config File "/cygdrive/c/web/galaxy-central/lib/galaxy/eggs/__init__.py", line 345, i n resolve egg.resolve() File "/cygdrive/c/web/galaxy-central/lib/galaxy/eggs/__init__.py", line 195, i n resolve return self.version_conflict( e.args[0], e.args[1] ) File "/cygdrive/c/web/galaxy-central/lib/galaxy/eggs/__init__.py", line 226, i n version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg. fetch ) File "/cygdrive/c/web/galaxy-central/lib/pkg_resources.py", line 565, in resol ve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: numpy==1.6.0 Fetch failed. ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

3 3

What's causing this error?
by Luobin Yang 18 Oct '11

18 Oct '11

Hi, Recently I moved my locally installed Galaxy from a local hard drive to an NFS mounted hard drive, when I run some tools, I go the following error from the log file: Job output not returned by PBS: the output datasets were deleted while the job was running, the job was manually dequeued or there was a cluster error. I am pretty sure the job was not manually dequeued. Any idea how this happened and how this can be fixed? Thanks, Luobin

2 2

jquery script error
by Iry Witham 18 Oct '11

18 Oct '11

For some strange reason since merging our local Galaxy instance with the latest distribution we seem to be getting a lot of jquery errors. The error message is: "Warning: Unresponsive script A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete. Script: http://galaxy/static/scripts/jquery.js?v=8:23" I had noticed this especially when I attempt to link more then 128 files to a single history. However, it appears to be occurring on a regular basis even with the simple creation of a folder. Most often it does not impact anything, but with the large number of files it will not complete the process. I have had to limit my link uploads to less then 128 at a time, but the error occurs regardless. Has anyone else experienced this issue and found a solution? Thanks, Iry

1 0

Re: [galaxy-dev] Accept-Ranges: bytes for
by Kip Bodi 18 Oct '11

18 Oct '11

Hi Brad, They're binary BAM files. I followed the section below, but perhaps I missed something. Thanks for the clarification on the Paste server, and I think I'll try nginx next if I can't get forwarding through Apache to work. Cheers, -Kip On Thu, Oct 13, 2011 at 1:17 PM, Brad Chapman <chapmanb(a)50mail.com> wrote: > > Kip; > Are these binary BAM or BigWig type files you are having trouble > displaying at UCSC? I don't believe the Paste server has support for the > byte-range requests that UCSC needs, and you need to send files through > the proxy server. I've done this with nginx so don't have direct Apache > experience here, but the 'Sending files using Apache' section here > should get it working: > > http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy > > Brad > > > > Perhaps this is a distribution issue. For those who have gotten the > "Display > > at UCSC" link to work on a local server or workstation, what linux > > distribution and version did you use? Has anyone used this successfully > with > > Red Hat Enterprise Linux 6.1? > > > > Best, > > > > -Kip > > > > On Fri, Oct 7, 2011 at 6:28 PM, Bodi, Kip <Kip.Bodi(a)tufts.edu> wrote: > > > > > Hi all, > > > > > > I have a local galaxy instance, proxied by Apache, and am trying to get > the > > > 'display at UCSC' link to work. I've followed all of the instructions > and am > > > nearly there, but now I'm getting this response from UCSC when it tries > to > > > load the track: > > > > > > "Byte-range request was ignored by server. Expected Partial Content > 206." > > > etc. > > > > > > I think the error is because the PasteWSGIServer isn't allowing > byte-range > > > requests. When I look at the header, "Accept-Ranges: bytes" is not > present: > > > [galaxy@galaxy galaxy_dist]$ curl --head localhost > > > HTTP/1.1 200 OK > > > Date: Fri, 07 Oct 2011 22:23:49 GMT > > > Server: PasteWSGIServer/0.5 Python/2.6.6 > > > content-type: text/html; charset=UTF-8 > > > Set-Cookie: > > > > galaxysession=c6ca0ddb55be603a4e98b4f13a4db8bfb8635f0aa576697114d40b13a8355efe2cf7a075ef81d769; > > > expires=Thu, 05-Jan-2012 17:23:49 GMT; Max-Age=7776000; Path=/; > Version=1 > > > Vary: Accept-Encoding > > > Connection: close > > > > > > Whereas on another server, "Accept-Ranges: bytes" appears in the > header: > > > [galaxy@galaxy galaxy_dist]$ curl --head apollo.med.tufts.edu< > > > http://apollo.med.tufts.edu> > > > HTTP/1.1 403 Forbidden > > > Date: Fri, 07 Oct 2011 22:22:42 GMT > > > Server: Apache/2.2.21 (Fedora) > > > Accept-Ranges: bytes > > > Content-Length: 4609 > > > Connection: close > > > Content-Type: text/html; charset=UTF-8 > > > > > > I'm running RHEL6 with the latest version of galaxy-dist. I've tried > > > manually enabling 'accept-ranges' but haven't had any luck. Any > suggestions? > > > Can I enable it somehow in universe_wsgi.ini? > > > > > > Best, > > > > > > -Kip > -- Kip Lord Bodi Jr. Tufts University School of Medicine 136 Harrison Avenue, South Cove 502 Boston, MA 02118 617-636-3763 kip.bodi(a)tufts.edu www.tucf.org | genomics.med.tufts.edu

2 1