April 2013 - galaxy-dev - lists.galaxyproject.org

Step 3 of "get Microbial Data" install
by Sarah Maman 08 May '13

08 May '13

Hello, I try to set "Get Microbial Data" in my local instance of Galaxy. So, as explain in /path/to//src/galaxy/galaxy-dist/scripts/microbes/README.txt , step 1 and 2 or OK, binary 'faToNib' is available on our cluster, but step 3 generate empty files : drwxr-xr-x 4 galaxy wbioinfo 100 18 avril 14:26 .. -rw-r--r-- 1 galaxy wbioinfo 203 18 avril 14:26 harvest.txt -rw-r--r-- 1 galaxy wbioinfo 69 18 avril 14:34 ncbi_to_ucsc.txt -rw-r--r-- 1 galaxy wbioinfo 0 19 avril 15:47 sequence.txt -rw-r--r-- 1 galaxy wbioinfo 0 19 avril 15:47 seq.loc drwxr-xr-x 2 galaxy wbioinfo 118 19 avril 15:47 . Could you please, hel me ? Thnaks in advance, Sarah Maman -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

2 1

Galaxy Keeps Freezing
by Kim Spradling 07 May '13

07 May '13

I have been trying to join the intervals of two datasets side-by-side using the public Main instance, but the job never finishes. I've gotten a message stating that Galaxy is unavailable a couple of times, but I have tried running this job multiple times with no success. I also tried to upload a text file to see if it was an issue with using that specific tool, but it won't complete either. Any help will be greatly appreciated. Thanks, Kim

2 1

Galaxy Jobs Waiting to Run for Several Days
by Elnagheeb, Marwa 07 May '13

07 May '13

Hello, I am a student, and I am using the psu galaxy server for an assignment. My workflow for each of my 5 data sets has been waiting to run 3 jobs (bwa alignment, sam to bam, and bam statistics). One of my datasets reached completion two days ago, but it has been 5 days for the others. Several others in my class have had the same issue. Is there something going on with the server this week? I really need to meet a deadline. Any chance I can get my jobs to run quicker without cloud? Thanks, Marwa

2 1

Strange issue where the dataset reported in the bug report doesn't correspond to the current dataset
by Jean-Francois Payotte 07 May '13

07 May '13

Dear Galaxy mailing-list, We are currently facing a strange issue with our local Galaxy installation (distribution from Jan. 11 2013). I'm going to try to describe the problem as much as I can, and feel free to ask for more details if it can help in solving this issue. So basically, we have a situation where (about half the time), when a job ends in error, the error message (shown when clicking on the bug icon), doesn't seem to be related to the dataset in error. As this particular problem seem to be present only when datasets ends in error, I created a simple python script which simply does 2 things: It outputs the hostname to stdout (in order to investigate if the problem is node-dependant) It outputs the following message "This is an error message printed to stderr", to stderr. (This is done in order to get only error datasets and to investigate the before-mentioned issue.) Here is the code of my python script: import sys import socket def main(): print socket.gethostname() sys.stderr.write('This is an error message printed to stderr\n') if __name__ == '__main__': main() I ran this tool about 50 times and I looked at the error messages displayed both in the preview section of the dataset (in the history), and the error message displayed when clicking on the bug icon "view or report this error". About half the time, the results were as expected: the dataset stdout would read the name of the node where the job was executed the dataset stderr would read: "This is an error message printed to stderr" and the error message displayed when clicking on the bug icon would be: Dataset generation errors Dataset 54: Test error Tool execution generated the following error message: This is an error message printed to stderr The tool produced the following additional output: some.server.name However, the other half of the time, the error message shown in stderr doesn't correspond to the error message displayed when clicking on the bug icon: the dataset stdout is still reading the name of the node the dataset stderr is still reading "This is an error message printed to stderr" but the error message displayed when clicking on the bug icon would be something like: Dataset generation errors Dataset 3: chrM.bed Tool execution did not generate any error messages. We can clearly see a discrepancy between the error message in stderr and the error message of the bug report. Actually, the bug report is saying that there is not any error and it makes reference to some "Dataset 3: chrM.bed", even when the actual dataset is "Dataset 53: Test error". There is absolutely no .bed file in my history and the dataset 3 actually reads "Dataset 3: Test error". Some of the datasets mentioned in the faulty bug reports seems to be really old datasets (like one year old). So my question to you is, could this be related to some mix-up between datasets ID? And how can I look this up. I must say that at the moment, I have absolutely no idea how to solve this issue. Many thanks for your help! Best regards, Jean-François

3 9

Re: [galaxy-dev] Running Galaxy through Apache
by Jeffrey Long 06 May '13

06 May '13

1) Any idea which proxy settings in the universe_wsgi file need to be set other than the ones on the website? Those are the only ones I changed. I scrolled down to the 'Advanced Proxy' settings but didn't see anything that looked relevant to me. Probably I should have sent this earlier, but this is the traceback I get; it looks like there's a url somewhere that's not getting passed along or re-assembled properly. Exception happened during processing of request from ('127.0.0.1', 47424) Traceback (most recent call last): File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1053, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.7/SocketServer.py", line 638, in __init__ self.handle() File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 432, in handle BaseHTTPRequestHandler.handle(self) File "/usr/lib/python2.7/BaseHTTPServer.py", line 340, in handle self.handle_one_request() File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 427, in handle_one_request self.wsgi_execute() File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 287, in wsgi_execute self.wsgi_start_response) File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/config.py", line 285, in __call__ return self.app(environ, start_response) File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py", line 193, in __call__ path_info = self.normalize_url(path_info, False)[1] File "/mnt2/birl/MAVEN/bin/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py", line 117, in normalize_url or self.domain_url_re.search(url)), "URL fragments must start with / or http:// (you gave %r)" % url AssertionError: URL fragments must start with / or http:// (you gave '.') 2) Are the static subdirectories strictly necessary, or can I worry about one problem at a time? I've tried both commenting out the proxy lines (in apache2.conf) and leaving them in, and I get the same behaviour. Thanks, I'll try to remember to 'reply-all' here! -Jeff On Mon, Apr 29, 2013 at 1:23 PM, Adam Brenner <aebrenne(a)uci.edu> wrote: > Hia Jeff, > > I think this is related to the proxy settings in Galaxy (universe_wsgi.ini) > > What is happening is that if you look at the source code for your > galaxy website, it is trying to load css/images from: > http://trove.usask.ca/galaxy/static/style/base.css?v=1367254127 But if > you visit that link it offers you a 404 not found. Make sure the proxy > settings in your universe_wsgi.ini is set correctly. > > If it is, it could be that you need to setup proxies for the > subdirectories as well. If you look at the nginx guide here: > http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy > you will notice that we provide alias for /static/*. > > In terms of doing this in Apache, you may need to add more proxy pass > lines that correspond to the subdirectories in question. > > > Be sure to post back to the Galaxy listsrv so that future people may > benefit from this! > -Adam > > -- > Adam Brenner > Computer Science, Undergraduate Student > Donald Bren School of Information and Computer Sciences > > Research Computing Support > Office of Information Technology > http://www.oit.uci.edu/rcs/ > > University of California, Irvine > www.ics.uci.edu/~aebrenne/ > aebrenne(a)uci.edu > > > On Mon, Apr 29, 2013 at 9:51 AM, Jeffrey Long <jlong1(a)ualberta.ca> wrote: > > Hi Adam thanks for the response, > > I'm not using VirtualHosts. I just copied the lines shown into > > /etc/apache2/apache.conf (this is on Ubuntu Linux). > > > > I copied the lines you list (with appropriate modification) into this > same > > apache.conf. That seems to work - as far as Apache goes! However, when > I > > load galaxy in a browser now, I get a bunch of gobbledigook (it looks > like > > the frames aren't being built properly or something). You can take a > look; > > I've temporarily enabled the site to be world-viewable. > > > > http://trove.usask.ca/galaxy > > > > Thanks for your help! > > > > -Jeff > > > > > > On Fri, Apr 26, 2013 at 10:11 PM, Adam Brenner <aebrenne(a)uci.edu> wrote: > >> > >> Jeff, > >> > >> How / Where did you add the apache changes? Are you using VirutalHosts > >> or ... ? Quick glance, everything appears correct. Its been a while > >> since I used Apache (switched everything to nginx) however I believe > >> if you want to setup a Reverse Proxy, you need to add / use the proxy > >> pass lines which does not appear in the Galaxy Wiki. > >> > >> Something like this should work: (untested...) > >> > >> <IfModule mod_proxy.c> > >> ProxyRequests Off > >> <Proxy *> > >> AddDefaultCharset off > >> Order Deny,Allow > >> Deny from all > >> Allow from *.ualberta.ca # Change / Remove as needed > >> </Proxy> > >> > >> ProxyPass /galaxy http://localhost:8080/ > >> ProxyPassReverse /galaxy http://localhost:8080/ > >> # Line below is optional > >> Redirect permanent /galaxy http://mymachine.com/galaxy > >> > >> ProxyVia On > >> </IfModule> > >> > >> > >> Let us know, > >> -Adam > >> > >> -- > >> Adam Brenner > >> Computer Science, Undergraduate Student > >> Donald Bren School of Information and Computer Sciences > >> > >> Research Computing Support > >> Office of Information Technology > >> http://www.oit.uci.edu/rcs/ > >> > >> University of California, Irvine > >> www.ics.uci.edu/~aebrenne/ > >> aebrenne(a)uci.edu > >> > >> > >> On Fri, Apr 26, 2013 at 2:56 PM, Jeffrey Long <jlong1(a)ualberta.ca> > wrote: > >> > Hi folks, > >> > I am wanting to run a galaxy instance and serve it at a location other > >> > than > >> > the webserver root, i.e. at http://mymachine.com/galaxy rather than > >> > just at > >> > http://mymachine. > >> > > >> > To do this, I've followed the instructions about proxying Galaxy > through > >> > Apache at http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy. > >> > I'm > >> > pretty sure I've followed the instructions exactly. I enabled the > >> > necessary > >> > apache proxy modules and then I added these lines to my apache.conf > >> > file: > >> > > >> > > >> > RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule > >> > ^/galaxy/static/style/(.*) > >> > /myplace/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule > >> > ^/galaxy/static/scripts/(.*) > >> > /myplace/galaxy-dist/static/scripts/packed/$1 > >> > [L] RewriteRule ^/galaxy/static/(.*) /myplace/galaxy-dist/static/$1 > [L] > >> > RewriteRule ^/galaxy/favicon.ico > /myplace/galaxy-dist/static/favicon.ico > >> > [L] > >> > RewriteRule ^/galaxy/robots.txt /myplace/galaxy-dist/static/robots.txt > >> > [L] > >> > RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] > >> > > >> > ...and re-started the apache server without a problem (note: I also > >> > tried it > >> > commenting out everything but the first, second and last line in case > >> > there > >> > was an issue with the static stuff). > >> > > >> > I then made sure the following lines were present and uncommented in > my > >> > universe_wsgi.ini file > >> > > >> > [filter:proxy-prefix] > >> > use = egg:PasteDeploy#prefix > >> > prefix = /galaxy > >> > > >> > [app:main] > >> > > >> > filter-with = proxy-prefix > >> > cookie_path = /galaxy > >> > > >> > > >> > I start up Galaxy. And still, I can only reach it on > >> > http://mymachine:8080, > >> > not at http://mymachine/galaxy as I intend. > >> > > >> > Any help would be appreciated! Thanks! > >> > > >> > -Jeff > >> > > >> > > >> > ___________________________________________________________ > >> > Please keep all replies on the list by using "reply all" > >> > in your mail client. To manage your subscriptions to this > >> > and other Galaxy lists, please use the interface at: > >> > http://lists.bx.psu.edu/ > >> > > >> > To search Galaxy mailing lists use the unified search at: > >> > http://galaxyproject.org/search/mailinglists/ > >> > > > >

3 4

Conditional tool options based on input file format?
by Peter Cock 06 May '13

06 May '13

Dear all, I've looked over the wiki and as far as I can see, 'when' tags used inside 'conditional' tags only work on another input variable's values. I would like to be able to do is a conditional based on the file format of an input file. For instance, a tool might take XML or tabular files, and need to show additional parameters for one input type but not the other. Does this enhancement idea seem useful? Peter

4 6

Any hint on how to use Galaxy within virtualenv?
by Jean-Francois Payotte 06 May '13

06 May '13

Dear Galaxy mailing list, We ran into the same issue described in this post: http://dev.list.galaxyproject.org/markupsafe-module-conflicts-tc4656214.html The exact error message we are getting, with a few different tools is: /Galaxy/galaxy_dist/lib/galaxy/__init__.py:79: UserWarning: Module markupsafe was already imported from /usr/lib64/python2.6/site-packages/markupsafe/__init__.pyc, but /Galaxy/galaxy_dist/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg is being added to sys.path self.check_version_conflict() >From the solution given in the above-mentioned post, I guess we would have to run Galaxy inside virtualenv. (Details of the procedure to install virtualenv, are given here: http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer) However, I am not sure to understand completely how it works. Can we use our already existing Galaxy instance with virtualenv? And how can we achieve this? For the installation of "virtualenv" part, I think we are ok. It is the next step: how to tell Galaxy to run in this virtualenv, that I am not sure about. Thank you in advance for your help! Jean-Francois

2 4

user creation using API
by Hagai Cohen 06 May '13

06 May '13

Hi, I would like to auto create users on my local Galaxy instance. I saw that in the "create user" on the API, I should enable use_remote_user (which as I understand means that I have to use Apache or nginx). Is there another way besides using Apache to create users automatically? Thanks, Hagai

3 3

Inconsistent menus in Galaxy Tool Shed
by Peter Cock 02 May '13

02 May '13

Hi all, This is just a usability bug, but I think it should be a simple fix that would make working with the Tool Shed a bit easier as the current menus seem randomised. --------------- If I click on "Repositories I own", then each tool has a drop down with only one action: * Mark as deprecated [image: Inline image 1] (This puzzles me - why just that rarely used option? Why not say upload, or view tests?) --------------- If I click on a tool, I go to the management page. Here the top right menu offers: * View tool functional test results * Upload files to repository * View change log * Browse or delete repository tip files * Reset all repository metadata * Mark repository as deprecated * Download as a .tar.gz file * Download as a .tar.bz2 file * Download as a .zip file [image: Inline image 2] --------------- If I go to the functional test results, the menu changes: * Manage repository * Upload files to repository * View change log * Browse or delete repository tip files * Download as a .tar.gz file * Download as a .tar.bz2 file * Download as a .zip file [image: Inline image 3] i.e. Aside the obvious change given which page we are on, why are "Reset all repository metadata" and "Mark repository as deprecated" missing? --------------- If I goto browse/delete tip files: * Manage repository * Upload files to repository * Browse or delete repository tip files * Download as a .tar.gz file * Download as a .tar.bz2 file * Download as a .zip file [image: Inline image 5] Notice "View change log" is now missing (as well as functional tests etc). --------------- If I go to the change log page, the menu changes again: * Upload files to repository * Manage repository * Browse or delete repository tip files * Download as a .tar.gz file * Download as a .tar.bz2 file * Download as a .zip file [image: Inline image 4] Again, several things are missing - and in particular the functional tests entry. To me that is highly relevant when looking at the change log. Also note the ordering of "Upload files to repository" and "Manage repository" has flipped which is confusing. --------------- My suggested fix is quite simple - use the same drop down menu for the repository actions on every page, either with the current page still on the list but disabled (and visually distinct as marker) is possible, or simply removed. Thanks, Peter

2 3

ASN.1 (text and binary) formats in Galaxy & Tool Shed
by Peter Cock 02 May '13

02 May '13

Hello all, Although they are these days also offering XML for many tools, the NCBI still make heavy use of the older ASN.1 file format (both as plain text and binary). This crops up in BLAST (e.g. as the BLAST archive format, or as dustmasker output), in the Entrez Utilities (e.g. for sequence data as an alternative to GenBank for FASTA format etc, or pubmed, etc) and also for 3D structures. I think it could make sense to define generic 'asn1' and 'asn1-binary' formats in the Galaxy core (name suggestions welcome), and even perhaps 'ncbi-asn1' and 'ncbi-asn1-binary' too. Then ToolShed entries can define domain specific subclasses. For instance, the BLAST+ wrapper could include definitions for the dustmasker output, and perhaps the BLAST archive format too. Separately anyone working with 3D structures as ASN.1 could define another sub-format, etc. I see this as a clear analogy to the assorted XML file formats in existence - defined in Galaxy as subclasses of the core XML format included with the Galaxy core. Would a pull request implementing this be acceptable? Peter P.S. Does anyone know an authoritative source for the MIME types used by the NCBI? Using the BLAST website they offer plain text ASN.1 just as text/plain, likewise efetch also seems to use text/plain for ASN.1 downloads. However I've seen references to chemical/ncbi-asn1-ascii and chemical/ncbi-asn1-binary mime-types mentioned, e.g. http://www.ncbi.nlm.nih.gov/data_specs/asn/NCBI_all.asn i.e. It appears that 3D structure NCBI ASN.1 files use a well defined MIME type, while most NCBI ASN.1 text files default to text/plain - which we can handle nicely in Galaxy as subclasses.

4 10