April 2013 - galaxy-dev - lists.galaxyproject.org

Bug report and patch for <version_command> whitespace
by Peter Cock 17 Apr '13

17 Apr '13

Hello all, I recently tried adding a <version_command> tag as follows, <version_command interpreter="python"> blastxml_to_top_descr.py --version </version_command> This kind of white space is tolerated in the <version> tag, so should work here too I think. However, it resulting the Galaxy attempting to run 'python' (which stalled as there was no user input), and then 'blastxml_to_top_descr.py --version'. The following simple change fixes this bug. In the meantime, I have used the workaround to removing the white space in the XML file: <version_command interpreter="python">blastxml_to_top_descr.py --version</version_command> Could someone please test this and commit it to the trunk, Regards, Peter -- $ hg diff diff -r 94e890fea3a4 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py Wed Apr 17 15:03:12 2013 +0100 +++ b/lib/galaxy/tools/__init__.py Wed Apr 17 15:45:53 2013 +0100 @@ -1094,7 +1094,7 @@ self.version_string_cmd = None version_cmd = root.find("version_command") if version_cmd is not None: - self.version_string_cmd = version_cmd.text + self.version_string_cmd = version_cmd.text.strip() version_cmd_interpreter = version_cmd.get( "interpreter", None ) if version_cmd_interpreter: executable = self.version_string_cmd.split()[0]

2 2

Unable to finish job
by Ramon Tiburski 17 Apr '13

17 Apr '13

I,m getting this error when I try to convert some data, can someone help me? Traceback (most recent call last): File "/home/ramao/galaxy/lib/galaxy/jobs/runners/local.py", line 156, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/ramao/galaxy/lib/galaxy/jobs/__init__.py", line 412, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/home/ramao/shed_tools/web.uniparthenope.it/repos/montella/galaxy_es_core/a8f8a904173c/galaxy_es_core/earthsystem.py", line 318, in set_meta NetCDF.set_meta(self,dataset) File "/home/ramao/shed_tools/web.uniparthenope.it/repos/montella/galaxy_es_core/a8f8a904173c/galaxy_es_core/earthsystem.py", line 261, in set_meta ncml= execAndGetStdout("ncdump -x "+dataset.file_name) File "/home/ramao/shed_tools/web.uniparthenope.it/repos/montella/galaxy_es_core/a8f8a904173c/galaxy_es_core/earthsystem.py", line 48, in execAndGetStdout output=local(cmd, capture=True) File "/home/ramao/galaxy/eggs/Fabric-1.4.2-py2.7.egg/fabric/operations.py", line 1012, in local error(message=msg, stdout=out, stderr=err) File "/home/ramao/galaxy/eggs/Fabric-1.4.2-py2.7.egg/fabric/utils.py", line 305, in error return func(message) File "/home/ramao/galaxy/eggs/Fabric-1.4.2-py2.7.egg/fabric/utils.py", line 26, in abort sys.exit(1) SystemExit: 1

2 3

Re: [galaxy-dev] [galaxy-user] Put an apache password on galaxy?
by Nate Coraor 17 Apr '13

17 Apr '13

On Apr 16, 2013, at 4:30 PM, Lee Katz wrote: > Thanks Jen. I tried a few more things before posting to galaxy-dev, and it turns out that I had the default virtual host still running, while I tried to have a galaxy virtualhost running alongside it. I think it should be possible to do that still, but it simplified and corrected it when I removed the default virtual host from apache/sites-enabled > > I have a second question now. Where would I put the .htaccess file for it to be read correctly? I tried this line in my new virtualhost file, but no password prompt appears > > AccessFileName /home/galaxy/bin/galaxy-dist/static/.htaccess > > More info: > All directories are chmod 775 for user galaxy starting with /home/galaxy/ > > $ cat static/.htaccess > AuthType Basic > AuthName "Yay Bioinformatics" > AuthUserFile /home/galaxy/bin/galaxy-dist/htpasswd > require valid-user > $ cat htpasswd > EDLB:xxxxxxxxxxxxx > $ ll static/.htaccess > -rw-r--r-- 1 galaxy galaxy 117 Apr 8 17:16 static/.htaccess > ll htpasswd > -rw-r--r-- 1 root root 19 Apr 8 17:32 htpasswd Hi Lee, It's best to simply put the authentication configuration directly in the Apache config file where you're defining the VirtualHost. Note that Galaxy also has functionality to restrict access itself, if you prefer. The relevant configuration option is 'require_login'. --nate > > > > > > On Mon, Apr 15, 2013 at 4:28 PM, Jennifer Jackson <jen(a)bx.psu.edu> wrote: > Hello Lee, > > Please see the section " External user authentication" on this wiki: > http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy > > If you still need help, posting this question over on the galaxy-dev(a)bx.psu.edu mailing list (but not cross-posting) with more details about what you have tried (how this solution was not a fit, etc.) will give you question the best exposure to the development community. > http://wiki.galaxyproject.org/MailingLists > > Best, > > Jen > Galaxy team > > > On 4/15/13 1:06 PM, Lee Katz wrote: >> Hi, I am relatively new to Galaxy. I would like to set it up as an apache password protected site. I have tried to do it, but I realize now that it has its own server built-in and that it is ineffective to set up .htaccess because it isn't reading .htaccess (or so I think). >> >> More information: Apache works (test page loads, and I can change it), and Galaxy works (I even made a new tool that is working). I have Ubuntu 12. >> >> My basic, basic need is to set up a server-level password like what .htaccess would provide. Could anyone help? Thank you. >> >> -- >> Lee Katz, Ph.D. >> >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at >> usegalaxy.org >> . Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> >> http://lists.bx.psu.edu/ >> >> >> To search Galaxy mailing lists use the unified search at: >> >> >> http://galaxyproject.org/search/mailinglists/ > > -- > Jennifer Hillman-Jackson > Galaxy Support and Training > > http://galaxyproject.org > > > > -- > Lee Katz, Ph.D. > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/

2 2

Local Galaxy Install - Multiple Job Handlers - Uploads waiting to run endlessly
by Adam Brenner 17 Apr '13

17 Apr '13

Howdy Galaxy Admins, I am installing Galaxy on our HPC cluster and running into a strange issue that I can no longer debug further. I am running the latest galaxy build as of Tuesday April 9th. hg clone https://bitbucket.org/galaxy/galaxy-dist/ hg update stable When a user tries to upload a file via the web interface, nginx reverse proxy setup as described here http://wiki.galaxyproject.org/Admin/Config/Performance/nginx%20Proxy, the upload stays in a state of "Job is waiting to run." The job stays like this for both FTP uploads, and uploads via browser. The dataset is a 319 bytes (both FTP and upload via browser) My universe_wsgi.ini variables are like so: .. under [app:main] .... new_file_path = /som/galaxy/tmp file_path = /som/galaxy/datasets/files track_jobs_in_database = True job_manager = manager job_handlers = handler0,handler1 ... before [app:main] in [server:main] .... [server:main] use = egg:Paste#http port = 8080 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:manager] use = egg:Paste#http port = 8079 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler0] use = egg:Paste#http port = 8090 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 [server:handler1] use = egg:Paste#http port = 8091 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Galaxy, for now, is running under the root user and is installed in /data/apps/galaxy (NFS mount) and the tmp files are stored on a GlusterFS system under /som/galaxy/. I have tried removing the extra job managers and handlers so I just have server:main and this still continues to happen. The paster.log files (with DEBUG = True) does not show any errors from python or issues writing to file. All POST and GET responses are recorded and nothing else. What I find strange is that if I look in new_file_path I do see my dataset as upload_file_data_Zx_1rK and cat'ing the file shows the correct contents. The items listed in my file_path (the dataset_*.dat) files are empty/blank..but are created. Any ideas as to why its failing? Is it nginx? I only have it redirect 8080, and none of the other job managers / handlers...is that an issue? Do you think its permissions on the my file_path or new_file_path directories? A very stumped Adam..... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

1 3

Re: [galaxy-dev] (no subject)
by Dave Bouvier 16 Apr '13

16 Apr '13

David, The reason shed_tools should be outside the Galaxy path is that tool shed mercurial repositories can interfere with the Galaxy mercurial repository, since mercurial will find more than one .hg directory. As long as the data you want to store is not a mercurial repository, you should be safe, but my advice would still be to locate it outside the Galaxy path to avoid any possibility of conflicts. --Dave B. On 4/16/13 16:28:01.000, David Bonsall wrote: > Thanks Dave > > A quick update from /galaxy-dist .... > > % hg update stable > > > .....did the trick! > > > Btw, I have another question I have since added sub-directories to /galaxy-dist to store blast databases etc. Is this bad? Will mercurial be able to update. I remember a warning to not install shed-tools directory in the main galaxy stem for this reason. Does this rule apply to all directories? > > > Best Wishes > > David > > > > On Tue, Apr 16, 2013 at 9:16 PM, Dave Bouvier <dave(a)bx.psu.edu > <mailto:dave@bx.psu.edu>> wrote: > > David, > > Thank you for reporting this error, which was due to a new tool shed > feature being introduced in -central, which had not yet been wrapped > in a backwards compatibility layer for -dist. As of > 9440:bbb229927c68, the issue should be resolved. > > --Dave B. > > On 4/11/13 12:51:03.000, David Bonsall wrote: > > Dear Galaxy team, > > I'm trying to install NCBI + tools on a local instance of galaxy > (rep: > 8.1546099212f), but I get the following error, which I do not > get when > trying to install other tools. > > Thank you for all your hard work. It's all much appreciated > > David > > > > Error Traceback: > View as: Interactive | Text | XML (full) > ⇝ HTTPError: HTTP Error 500: Internal Server Error > URL: > http://127.0.0.1:8080/admin___toolshed/prepare_for_install?__tool_shed_url=… > <http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=http…> > Module weberror.evalexception.__middleware:364 in respond > view > >> app_iter = self.application(environ, detect_start_response) > Module paste.recursive:84 in __call__ view > >> return self.application(environ, start_response) > Module paste.httpexceptions:633 in __call__ view > >> return self.application(environ, start_response) > Module galaxy.web.framework.base:125 in __call__ view > >> return self.handle_request( environ, start_response ) > Module galaxy.web.framework.base:182 in handle_request view > >> body = method( trans, **kwargs ) > Module galaxy.web.framework:228 in decorator view > >> return func( self, trans, *args, **kwargs ) > Module galaxy.webapps.galaxy.__controllers.admin_toolshed:894 in > prepare_for_install view > >> common_install_util.get___dependencies_for_repository( trans, > tool_shed_url, repo_info_dict, includes_tool_dependencies ) > Module tool_shed.util.common_install___util:90 in > get_dependencies_for___repository view > >> required_repo_info_dicts = get_required_repo_info_dicts( > tool_shed_url, util.listify( repo_info_dict ) ) > Module tool_shed.util.common_install___util:283 in > get_required_repo_info_dicts view > >> response = urllib2.urlopen( url ) > Module urllib2:126 in urlopen view > >> return _opener.open(url, data, timeout) > Module urllib2:400 in open view > >> response = meth(req, response) > Module urllib2:513 in http_response view > >> 'http', request, response, code, msg, hdrs) > Module urllib2:438 in error view > >> return self._call_chain(*args) > Module urllib2:372 in _call_chain view > >> result = func(*args) > Module urllib2:521 in http_error_default view > >> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) > HTTPError: HTTP Error 500: Internal Server Error > > > _____________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/__search/mailinglists/ > <http://galaxyproject.org/search/mailinglists/> > >

1 0

Re: [galaxy-dev] Toolshed errors
by Greg Von Kuster 16 Apr '13

16 Apr '13

Hello Björn, On Apr 6, 2013, at 9:06 AM, Björn Grüning wrote: > Hi Greg and Dave, > > good to know that it is at least installing :) > I also tried without apache, fresh checkout and get the same timeouts. > Maybe my computer is just so slow that its triggers the timeout. Also my > Memory is very limited and nearly swapped during compilation. > > But with the new API I'm able to install all of them just fine. Is that > explainable? > During my experiments I found that im not able to install the tools if a > new-panel-section is supplied: > > galaxy.web.framework ERROR 2013-04-06 14:29:05,411 Uncaught exception in > exposed API method: > > Traceback (most recent call last): > File > "/home/bag/projects/galaxy/galaxy-central/lib/galaxy/web/framework/__init__.py", line 196, in decorator > rval = func( self, trans, *args, **kwargs) > File > "/home/bag/projects/galaxy/galaxy-central/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py", line 254, in install_repository_revision > reinstalling=False ) > File > "/home/bag/projects/galaxy/galaxy-central/lib/tool_shed/galaxy_install/repository_util.py", line 358, in install_tool_shed_repository > if tool_panel_section_key: > File "/usr/lib/python2.7/UserDict.py", line 23, in __getitem__ > raise KeyError(key) > KeyError: 'section_chemicaltoolbox' This issue should be fixed in revision 3a07307 - thanks for reporting this! > > Without that option it works. I tried to understand the code-path but > found no section where a new entry is created under the toolbox. What I > found is that: tool_panel_section_keys in repository_dependecy_util.py > line 235 is updated every time and you will end up with several > identical instances of 'section_chemicaltoolbox'. Hope that helps a > little bit. I'm not able to reproduce this (assuming it is not related to the issue I fixed above). Is this issue now resolved for your? > > Also please see a pull-request for a small typo: > https://bitbucket.org/galaxy/galaxy-central/pull-request/150/change-error-m… This pull request has been accepted - thanks for the fixes! > > Thanks for your work! > Bjoern > > > > >> Hello Björn, >> >> Dave Bouvier (cc'd on this) spent some time yesterday inspecting the logs for out test tool shed and found no timeouts in either our paster log or our nginx log. Dave was also successful in installing your repository and dependencies. If I remember correctly, this timeout issue may be intermittent, but again, Dave found no logged timeout events at our end. If you uncover any additional information, please let us know, and we'll continue to keep an eye on this. >> >> Thanks very much, >> >> Greg Von Kuster >> >> >> On Apr 4, 2013, at 5:32 AM, Björn Grüning wrote: >> >>> Hi Greg, >>> >>> sorry forgot to attach the new traceback. Its now attached. >>> If the grepping with the IP address did not work, maybe you can grep for >>> the chemicaltoolbox/chemfp/openbabel/osra repositories. Im probably the >>> only one working with it. >>> >>> Have a nice day! >>> Bjoern >>> >>>> Hello Björn, >>>> >>>> Thanks very much for your message! Please see my inline comments. >>>> >>>> >>>> On Apr 2, 2013, at 4:33 PM, Björn Grüning wrote: >>>> >>>>> I have tested a few things regarding to the toolshed and prepared a >>>>> complex module-dependency-beast, with meta-package and >>>>> orhphan-tool-dependencies and so on. A few problems I encountered are >>>>> summarized below. You can try it by yourself, if you install the >>>>> chemicaltoolbox package from the testtoolshed. >>>>> >>>>> - three crashes with traceback attached, during tool-installation >>>>> >>>>> - one 'name' not defined error, with attached patch >>>> >>>> >>>> Thanks very much for finding this issue and for the patch. We've applied it in changeset 9252:da2d6ee29f30 in our stable branch, so it is running on both public Galaxy tool sheds. >>>> >>>> >>>>> >>>>> - For me the the installation of tools with really complex and long >>>>> compiling steps are stalled in 'Installing' forever. Try to install the >>>>> osra package (it often hangs, but not every time). >>>>> Its not possible to uninstall/reinstall these dependencies >>>>> without touching the database. And the error says, tool is already >>>>> installed. Would be nice if the admin is able to reset that repository >>>>> easily, even if it is limbo ... (is there a timeout somewhere?) >>>>> Error: paste.httpserver.ThreadPool WARNING 2013-04-02 22:02:41,256 Thread 140246687659776 hung (working on task for 1964 seconds) >>>>> (The SystemExit: traceback attached) >>>> >>>> I've create the following Trello card for this issue: >>>> >>>> https://trello.com/card/toolshed-enable-galaxy-admin-to-reset-tool-dependen… >>>> >>>> There are 4 places where this timeout may be occurring: Galaxy tool shed server, Galaxy tool shed nginx, a proxy server at your end, or your local Galaxy server. There is not enough information in your attached traceback log to determine where the timeout is occurring. Can you provide me your Galaxy server's IP address so I can look in logs at our end? >>>> >>>> >>>>> >>>>> - It would be great if the tool-developer or the admin has more control >>>>> over the location of the tools in the toolbar. Installing meta package >>>>> leaves you with only one choice, regardless how many tools are bundled. >>>> >>>> >>>> Support for this will be available in the next Galaxy release. There are Trello cards for this here: >>>> >>>> https://trello.com/card/toolshed-enhance-the-current-galaxy-api-for-install… >>>> https://trello.com/card/toolshed-add-features-that-leverage-the-galaxy-api-… >>>> >>>> >>>>> >>>>> - Installing dependencies will leave me with empty folders under >>>>> galaxy-root with the name of the dependency. My guess is, >>>>> that if I manually download a file/folder with wget, the folder is not purged, but the content is. >>>> >>>> >>>> Our intent is that installing repositories from the tool shed is completely automated with no manual intervention required. Tool dependency definitions should not use utilities like wget to download dependencies because these utilities may not be available in the environments (e.g. vanilla mac os) into which the repositories are being installed. I'll enhance the tool dependency tag sets as necessary to accommodate additional download and install requirements. Can you let me know if the current download_by_url needs enhancements? >>>> >>>> >>>>> >>>>> - Often after upgrading the tool_dependencies.xml file, I was not able >>>>> to 'get updates' from the toolshed. The version was up-to-data, but it >>>>> wasn't. Deinstalling and Installing fixed it. But I think its ab bug, >>>>> isn't it? >>>> >>>> This very well may be a bug. Can you please provide additional details so that I can reproduce this behavior? I'll get a fix asap. >>>> >>>> >>>>> >>>>> - Is there a way, to just download a python file, not zipped and add it to PYTHONPATH? >>>>> If I download it with <action type="download_by_url"> I have problems to copy that file with move. >>>>> I would like to download it with wget, but one download by URL is required. My current solution is, >>>>> to include a dummy download_by_url and than fetch with wget, or download a archive version. >>>> >>>> I've created the following Trello card for this issue: >>>> >>>> https://trello.com/card/toolshed-enhance-the-download-by-url-process-for-in… >>>> >>>> >>>>> >>>>> - Is it correct that a repository referenced only in >>>>> tool_dependencies.xml is not enough to install? numpy was only >>>>> referenced as tool_dependency and I got the error that the tool (with >>>>> owner and revision) was not found. After adding to repository_dependency >>>>> it was ok. >>>> >>>> Can you provide more details about your question above (e.g., the tool_dependencies.xml fie would help). I want to make sure i understand the issue. >>>> >>>> Thanks very much Björn, >>>> >>>> Greg Von Kuster >>>> >>>> >>>> >>> >>> <trackback.txt> >> > >

1 0

(no subject)
by David Bonsall 16 Apr '13

16 Apr '13

Dear Galaxy team, I'm trying to install NCBI + tools on a local instance of galaxy (rep: 8.1546099212f), but I get the following error, which I do not get when trying to install other tools. Thank you for all your hard work. It's all much appreciated David Error Traceback: View as: Interactive | Text | XML (full) ⇝ HTTPError: HTTP Error 500: Internal Server Error URL: http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=http… Module weberror.evalexception.middleware:364 in respond view >> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view >> return self.application(environ, start_response) Module galaxy.web.framework.base:125 in __call__ view >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:182 in handle_request view >> body = method( trans, **kwargs ) Module galaxy.web.framework:228 in decorator view >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:894 in prepare_for_install view >> common_install_util.get_dependencies_for_repository( trans, tool_shed_url, repo_info_dict, includes_tool_dependencies ) Module tool_shed.util.common_install_util:90 in get_dependencies_for_repository view >> required_repo_info_dicts = get_required_repo_info_dicts( tool_shed_url, util.listify( repo_info_dict ) ) Module tool_shed.util.common_install_util:283 in get_required_repo_info_dicts view >> response = urllib2.urlopen( url ) Module urllib2:126 in urlopen view >> return _opener.open(url, data, timeout) Module urllib2:400 in open view >> response = meth(req, response) Module urllib2:513 in http_response view >> 'http', request, response, code, msg, hdrs) Module urllib2:438 in error view >> return self._call_chain(*args) Module urllib2:372 in _call_chain view >> result = func(*args) Module urllib2:521 in http_error_default view >> raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error

2 2

the release_2013.04.01 update crashes our galaxy server
by Christophe Antoniewski 16 Apr '13

16 Apr '13

Hi, After a crash recovery and a second trial I must admit that I cannot upgrade our galaxy instance. I need some help... the facts: initial instance: $ hg head changeset: 8796:961ae35ba612 branch: stable tag: tip parent: 8794:1c7174911392 user: Nate Coraor <nate(a)bx.psu.edu> date: Fri Feb 08 11:23:04 2013 -0500 summary: Added tag release_2013.02.08 for changeset 1c7174911392 changeset: 8795:9fd7fe0c5712 parent: 8785:25bd9d387135 parent: 8794:1c7174911392 user: Nate Coraor <nate(a)bx.psu.edu> date: Fri Feb 08 11:22:54 2013 -0500 summary: merge from stable Then I pull the upgrade using mercurial and update the postgresql database as follows: $ hg pull pulling from https://bitbucket.org/galaxy/galaxy-dist/ searching for changes adding changesets adding manifests adding file changes added 516 changesets with 1568 changes to 710 files (-1 heads) (run 'hg update' to get a working copy) galaxy@GED-Server:~/galaxy-dist$ hg update release_2013.04.01 merging lib/galaxy/web/form_builder.py merging tools/ngs_rna/tophat_wrapper.xml 750 files updated, 2 files merged, 433 files removed, 0 files unresolved galaxy@GED-Server:~/galaxy-dist$ hg branch stable galaxy@GED-Server:~/galaxy-dist$ sh manage_db.sh upgrade 110 -> 111... Add support for job destinations to the job table done 111 -> 112... Migration script to add the data_manager_history_association table and data_manager_job_association. 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,412 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,412 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,428 Created data_manager_job_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-04-16 00:59:22,428 Created data_manager_job_association table done 112 -> 113... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 113 -> 114... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done When I restarts the server, I have $ sh run.sh --reload Initializing data_manager_conf.xml from data_manager_conf.xml.sample Initializing shed_data_manager_conf.xml from shed_data_manager_conf.xml.sample Some eggs are out of date, attempting to fetch... Fetched http://eggs.galaxyproject.org/PasteDeploy/PasteDeploy-1.5.0-py2.7.egg Removed conflicting egg: /home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg Fetched http://eggs.galaxyproject.org/raven/raven-3.1.8-py2.7.egg Fetch successful. > I suspect that some new python eggs may cause my troubles... and then, ... some usual bla bla... and finishes by .... galaxy.webapps.galaxy.buildapp DEBUG 2013-04-16 01:00:30,077 Enabling 'Request ID' middleware Traceback (most recent call last): File "./scripts/paster.py", line 33, in <module> serve.run() File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 1056, in run invoke(command, command_name, options, args[1:]) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 1062, in invoke exit_code = runner.run(args) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 227, in run result = self.command() File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/serve.py", line 650, in command app = loadapp( app_spec, name=app_name, relative_to=base, global_conf=vars) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 350, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 375, in loadobj return context.create() File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 813, in create return self.object_type.invoke(self) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 332, in invoke filtered = context.next_context.create() File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 813, in create return self.object_type.invoke(self) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 249, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/home/galaxy/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py", line 97, in fix_call val = callable(*args, **kw) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 172, in app_factory webapp = wrap_in_static( webapp, global_conf, **kwargs ) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 327, in wrap_in_static urlmap["/static"] = Static( conf.get( "static_dir" ), cache_time ) File "/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/static.py", line 16, in __init__ StaticURLParser.__init__( self, directory ) File "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py", line 430, in __init__ self.directory = self.normpath(directory) File "/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/urlparser.py", line 435, in normpath return os.path.normcase(os.path.abspath(path)) File "/usr/lib/python2.7/posixpath.py", line 343, in abspath if not isabs(path): File "/usr/lib/python2.7/posixpath.py", line 53, in isabs return s.startswith('/') *AttributeError: 'NoneType' object has no attribute 'startswith'* *galaxy.jobs.handler INFO 2013-04-16 01:00:30,112 sending stop signal to worker thread* *galaxy.jobs.handler INFO 2013-04-16 01:00:30,112 job handler queue stopped* *galaxy.jobs.runners INFO 2013-04-16 01:00:30,112 LWRRunner: Sending stop signal to monitor thread* *galaxy.jobs.runners INFO 2013-04-16 01:00:30,112 LWRRunner: Sending stop signal to 3 worker threads* *galaxy.jobs.runners INFO 2013-04-16 01:00:30,113 LocalRunner: Sending stop signal to 5 worker threads* *galaxy.jobs.handler INFO 2013-04-16 01:00:30,115 sending stop signal to worker thread* *galaxy.jobs.handler INFO 2013-04-16 01:00:30,115 job handler stop queue stopped* does anyone has a clue about what is going wrong (eggs or something...) Thanks thanks thanks in advance.... -- Christophe Antoniewski Drosophila Genetics and Epigenetics Laboratoire de Biolologie du Développement 9, Quai St Bernard, Boîte courrier 24 75252 Paris Cedex 05 Tel +33 1 44 27 34 39 Fax +33 1 44 27 34 45 Mobile +33 6 68 60 51 50 http://drosophile.org

2 3

where the log in information saved?
by shenwiyn 16 Apr '13

16 Apr '13

Hi everyone, I installed my local galaxy successfully.But I have a problem about how to find out the log in information of each user ,such as the register mail and the password .And which files do this informations saved? Thank you! shenwiyn

4 3

Re: [galaxy-dev] Tophat didn't run
by Jennifer Jackson 16 Apr '13

16 Apr '13

Hi Sarah, On 4/11/13 8:02 AM, Sarah Maman wrote: > Thnaks Jennifer, > > Excuse me, my previous mail contains an error : In fact, the reference > genome from my history was a fasta format (the name was GTF file but > the format was fasta...). > So, when I run tophat with a reference genome from your history, here > is the error message (my reference genome is a FASTA file) : OK, now this looks like a tool/index mismatch problem. Most likely rooted in a binary path issue. > Error in tophat: > > [2013-04-11 14:57:12] Beginning TopHat run (v2.0.5) > ----------------------------------------------- > [2013-04-11 14:57:12] Checking for Bowtie > Bowtie version: 2.0.0.7 > [2013-04-11 14:57:12] Checking for Samtools > Samtools version: 0.1.19.0 > [2013-04-11 14:57:13] Checking for Bowtie index files > Error: Could not find Bowtie 2 index files (/tmp/1078173.1.workq/tmpzxEFNK/dataset_6485.*.bt2) > > Settings: > blablabla OK..... > Total time for backward call to driver() for mirror index: 00:00:57 > TopHat v2.0.5 > tophat -p 4 /tmp/1078173.1.workq/tmpzxEFNK/dataset_6485 /work/galaxy/database/files/006/dataset_6528.dat > [bam_header_read] EOF marker is absent. The input is probably truncated. > [bam_header_read] invalid BAM binary header (this is not a BAM file). > [bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM file. > Epilog : job finished at jeu. avril 11 14:57:18 CEST 2013 > > And here is my bowtie and tophat versions : > > $ which bowtie > bowtie -v > 0.12.8 This is good > $ which tophat > tophat -v > 2.0.5 This is most likely the problem. There is probably a symbolic link pointing from tophat -> tophat2. You will want to remove that. The tool wrappers will be looking for the correct binary/indexes for the version they are each dependent on. This means that if you are running Tophat2 for Illumina, you want both tophat2 and bowtie2 to be used, along with the bowtie2 indexes. This is detailed in the wikis I sent in the Tophat/Bowtie sections, for both dependencies and the index set up. My guess at this point, without seeing your exact files, is that you need to add the index path to the bowtie2 loc file, and remove/adjust the symbolic link as I stated above, then restart, and test again to see if that fixes the problem. > But we have also, available on our cluster : > bowtie2 --version is 2.0.0-beta7 This is also good, if "which bowtie" == v0.12.8 and "which bowtie2" == v2. If "bowtie" is pointing to v2 on your cluster nodes, then remove that symbolic link, so that this is instead pointing to the correct binary (v0.12.8). same for bowtie2, should point to the v2 binary. bowtie/tophat and bowtie2/tophat2 are not the same executable and use different indexes - this is most likely why you had to use the bowtie v0.12.8 loc to get tophat2 going to begin with. Hope it works this time! Please keep replies on the list to help us with tracking, Jen Galaxy team > > Could you please tell me how to point to the v2 binaries (how to > change symbolic links) ? > > Thanks in advance, > Sarah > > > Jennifer Jackson a écrit : >> Hi Sarah, >> >> It still sounds like there is a path problem - this is why the tools >> are looking in the wrong loc file. When bowtie2/tophat2 installs, it >> will create a symbolic link that names itself as just "bowtie" or >> "tophat", pointing to the v2 binaries. >> >> When you run these, what do you get? >> >> $ which bowtie >> >> $ which tophat >> >> My guess is that these are symbolic links pointing to the v2 >> binaries. You will want to remove these. This is noted in the NGS >> set-up wiki, but easy to miss. >> >> For the custom _reference genome _portion below, there is a mix-up >> here. A custom _reference genome_ is in fasta format, not GTF format. >> I think what you are doing is using a _reference annotation_ file >> with the process. Both can be used with RNA-seq tools, but the >> _reference genome_ is the one with the indexes. The link I sent about >> _custom reference genomes_ explains how to use one of these, if you >> still what want to try. >> >> I think it is worth reviewing the path and loc info, plus the binary >> commands above. Unless this helps you to solve the problem on your >> own now. >> >> Thanks! >> >> Jen >> Galaxy team >> >> On 4/11/13 6:16 AM, Sarah Maman wrote: >>> Thanks a lot Jennifer, >>> >>> Restart, full paths were OK. >>> >>> I don't know why but the 2nd version of Tophat (so the tophat tool >>> available from Galaxy) search indexs in bowtie-index.loc file and >>> not in bowtie2-index.loc >>> So, I've added my bowtie2 index paths in bowtie-index. loc file and >>> tophat run... >>> >>> >>> But when I want to run tophat with a reference genome from your >>> history, here is the error message (my reference genome is a GFT file) : >>> Error in tophat: >>> >>> [2013-04-11 14:57:12] Beginning TopHat run (v2.0.5) >>> ----------------------------------------------- >>> [2013-04-11 14:57:12] Checking for Bowtie >>> Bowtie version: 2.0.0.7 >>> [2013-04-11 14:57:12] Checking for Samtools >>> Samtools version: 0.1.19.0 >>> [2013-04-11 14:57:13] Checking for Bowtie index files >>> Error: Could not find Bowtie 2 index files (/tmp/1078173.1.workq/tmpzxEFNK/dataset_6485.*.bt2) >>> >>> >>> Settings: >>> blablabla OK..... >>> Total time for backward call to driver() for mirror index: 00:00:57 >>> TopHat v2.0.5 >>> tophat -p 4 /tmp/1078173.1.workq/tmpzxEFNK/dataset_6485 /work/galaxy/database/files/006/dataset_6528.dat >>> [bam_header_read] EOF marker is absent. The input is probably truncated. >>> [bam_header_read] invalid BAM binary header (this is not a BAM file). >>> [bam_index_core] Invalid BAM header.[bam_index_build2] fail to index the BAM file. >>> Epilog : job finished at jeu. avril 11 14:57:18 CEST 2013 >>> Thanks in advance, >>> Sarah >>> >>> >>> Jennifer Jackson a écrit : >>>> Hi Sarah, >>>> >>>> Let's try to sort this out. Your problem does not seem to be the >>>> same as in the question referenced, but we can see. First - just to >>>> double check - since setting up the genome, you have restarted the >>>> server? If not, do that first and check to see if that fixes the >>>> problem. Basically, you want to follow this checklist and >>>> restarting is the final step: >>>> http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup >>>> >>>> If the problem persists, then would you please send a few more details: >>>> >>>> 1 - full paths* on you system where you keep the .bt2 indexes, sam >>>> index, and .fa file. Maybe do an "ls -l" on these dirs so we can >>>> check the symbolic links are in place and named correctly. >>>> >>>> * as a note, these should be "hard paths" and not symbolic (except >>>> for the .fa links), and must have permissions set to be accessible >>>> to the "galaxy user" >>>> >>>> 2 - lines from your bowtie2_indices.loc and sam_fa_indices.loc file >>>> for this genome. I may have you double check your builds.txt file >>>> later. if this doesn't sounds familiar, it could be the problem, >>>> the genome must be in there, too. - see this wiki: >>>> http://wiki.galaxyproject.org/Admin/Data%20Integration >>>> >>>> 3 - full error message you get when you try to run this using a >>>> genome in fasta format from your history. It really shouldn't be >>>> the same error - something is not right with the settings and a >>>> custom genome is not actually being used if that is the case. Give >>>> it another try and see what happens, then send that info. This is a >>>> bit of a side case, we should get your basic install correct, but >>>> knowing how to do this is a good thing and easy to learn. >>>> http://wiki.galaxyproject.org/Support#Custom_reference_genome >>>> >>>> It is OK to masked out anything like user names/groups you don't >>>> want to share. Please keep on the list in case we need other feedback. >>>> >>>> Thanks! >>>> >>>> Jen >>>> Galaxy team >>>> >>>> On 4/10/13 3:15 AM, Sarah Maman wrote: >>>>> Hello, >>>>> >>>>> >>>>> When I run tophat ("Tophat for Illumina Find splice junctions >>>>> using RNA-seq data ), the job failed with truncated files. >>>>> However, index files are available and I get exactly the same >>>>> error message using built-in index or one of my history. >>>>> >>>>> / >>>>> Tool execution generated the following error message: >>>>> >>>>> Error in tophat: >>>>> >>>>> [2013-04-10 09:17:07] Beginning TopHat run (v2.0.5) >>>>> ----------------------------------------------- >>>>> [2013-04-10 09:17:07] Checking for Bowtie >>>>> Bowtie version: 2.0.0.7 >>>>> [2013-04-10 09:17:07] Checking for Samtools >>>>> Samtools version: 0.1.19.0 >>>>> [2013-04-10 09:17:07] Checking for Bowtie index files >>>>> Error: Could not find Bowtie 2 index files >>>>> (/work/galaxy/Danio_rerio.Zv9.62.dna.chromosome.22.fa.*.bt2) >>>>> >>>>> >>>>> The tool produced the following additional output: >>>>> >>>>> TopHat v2.0.5 >>>>> tophat -p 4 /work/galaxy/Danio_rerio.Zv9.62.dna.chromosome.22.fa >>>>> /work/galaxy/database/files/006/dataset_6528.dat >>>>> [bam_header_read] EOF marker is absent. The input is probably >>>>> truncated. >>>>> [bam_header_read] invalid BAM binary header (this is not a BAM file). >>>>> [bam_index_core] Invalid BAM header.[bam_index_build2] fail to >>>>> index the BAM file. >>>>> Epilog : job finished at mer. avril 10 09:17:12 CEST 2013 / >>>>> >>>>> In this post >>>>> (http://dev.list.galaxyproject.org/tophat-for-illumina-looking-in-wrong-dire…) >>>>> the solution isn't found. >>>>> >>>>> Do you have any idea, >>>>> Sarah Maman >>>>> -- >>>>> --*-- >>>>> Sarah Maman >>>>> INRA - LGC - SIGENAE >>>>> http://www.sigenae.org/ >>>>> Chemin de Borde-Rouge - Auzeville - BP 52627 >>>>> 31326 Castanet-Tolosan cedex - FRANCE >>>>> Tel: +33(0)5.61.28.57.08 >>>>> Fax: +33(0)5.61.28.57.53 >>>>> --*-- >>>>> >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> http://lists.bx.psu.edu/ >>>>> >>>>> To search Galaxy mailing lists use the unified search at: >>>>> http://galaxyproject.org/search/mailinglists/ >>>> >>>> -- >>>> Jennifer Hillman-Jackson >>>> Galaxy Support and Training >>>> http://galaxyproject.org >>> >>> >>> -- >>> --*-- >>> Sarah Maman >>> INRA - LGC - SIGENAE >>> http://www.sigenae.org/ >>> Chemin de Borde-Rouge - Auzeville - BP 52627 >>> 31326 Castanet-Tolosan cedex - FRANCE >>> Tel: +33(0)5.61.28.57.08 >>> Fax: +33(0)5.61.28.57.53 >>> --*-- >> >> -- >> Jennifer Hillman-Jackson >> Galaxy Support and Training >> http://galaxyproject.org > > > -- > --*-- > Sarah Maman > INRA - LGC - SIGENAE > http://www.sigenae.org/ > Chemin de Borde-Rouge - Auzeville - BP 52627 > 31326 Castanet-Tolosan cedex - FRANCE > Tel: +33(0)5.61.28.57.08 > Fax: +33(0)5.61.28.57.53 > --*-- -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org

2 1