September 2008 - galaxy-user - lists.galaxyproject.org

accessing data from the filesystem
by James Casbon 29 Sep '08

29 Sep '08

Hi, Is it possible to access data on the filesystem without uploading it. i.e. I install galaxy locally, and then want to refer to some huge fasta file on disk. At the moment, I can upload it, which is slow and creates two copies. Can I just point galaxy at it somehow? thanks, James

3 4

patch: use xml includes when parsing xml
by James Casbon 29 Sep '08

29 Sep '08

Hi, I have inlined a patch that allows xinclude statements in xml files. I presume I cannot send attachments to the list? Motivation: I have my own tools I am developing managed in tools/my_tools. I want to version that directory, etc, but want to be able to include new tools without editing the tools_conf.xml in galaxy_dist each time which would not get versioned. Solution: use xinclude to include an xml file that contains tool definitions. Example tool_conf.xml: <toolbox xmlns:xi="http://www.w3.org/2001/XInclude"> .... <xi:include href="tools/my_tools/tool_conf.xml" parse="xml" /> </toolbox> Patch below, cheers, James ============== diff -r 3b6771227a43 lib/galaxy/util/__init__.py --- a/lib/galaxy/util/__init__.py Fri Sep 26 09:10:04 2008 -0400 +++ b/lib/galaxy/util/__init__.py Mon Sep 29 15:50:20 2008 +0100 @@ -12,7 +12,7 @@ from galaxy.util.docutils_ext.htmlfrag import Writer as HTMLFragWriter pkg_resources.require( 'elementtree' ) -from elementtree import ElementTree +from elementtree import ElementTree, ElementInclude log = logging.getLogger(__name__) _lock = threading.RLock() @@ -64,6 +64,8 @@ def parse_xml(fname): """Returns an parsed xml tree""" tree = ElementTree.parse(fname) + root = tree.getroot() + ElementInclude.include(root) return tree def xml_to_string(elem): ================

1 0

LaunchDaemon for OS X
by James Casbon 26 Sep '08

26 Sep '08

Here is a LaunchDaemon for OS X, maybe useful. Need to change the paths for your install. <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>edu.psu.galaxy</string> <key>OnDemand</key> <false/> <key>Program</key> <string>/Users/james/galaxy-dist/run.sh<string> <key>RunAtLoad</key> <true/> <key>ServiceDescription</key> <string>Galaxy</string> <key>StandardErrorPath</key> <string>/Users/james/galaxy-dist/logs/launchd.stderr</string> <key>StandardOutPath</key> <string>/Users/james/galaxy-dist/logs/launchd.stdout</string> <key>UserName</key> <string>james</string> </dict> </plist>

1 0

several questions from newbie
by Xianjun Dong 24 Sep '08

24 Sep '08

3 2

Re: [galaxy-user] Install problems on OS X with python from macports
by James Casbon 16 Sep '08

16 Sep '08

Hi Nate, Your fix works for the scramble fine, but not for the run.sh which now fails. On removing the -ES from run.sh, I get an error importing simplejson (see below). simplejson.egg is in the eggs/py2.5-noplatform dir so I dont' see why it doesn't pick this up if it manages to pick up WebHelpers. Traceback (most recent call last): File "./scripts/paster.py", line 26, in <module> command.run() File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py", line 78, in run File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py", line 117, in invoke File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py", line 212, in run File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/serve.py", line 227, in command File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/serve.py", line 250, in loadapp File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 193, in loadapp File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 213, in loadobj File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 237, in loadcontext File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 267, in _loadconfig File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 397, in get_context File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 439, in _context_from_explicit File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py", line 18, in import_string File "/Users/james/Src/galaxy_dist/lib/pkg_resources.py", line 1912, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/Users/james/Src/galaxy_dist/lib/galaxy/web/__init__.py", line 5, in <module> from framework import expose, json, require_login, url_for, error, form, FormBuilder File "/Users/james/Src/galaxy_dist/lib/galaxy/web/framework/__init__.py", line 17, in <module> import webhelpers File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/__init__.py", line 1, in <module> File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/rails/__init__.py", line 7, in <module> File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/rails/scriptaculous.py", line 16, in <module> ImportError: No module named simplejson 2008/9/12 Nate Coraor <nate(a)bx.psu.edu>: > James Casbon wrote: > >> Here is the traceback. If you read hashlib, it tries to import >> _hashlib and if that fails and so it tries to get _md5. The MacPorts >> python should have _hashlib but not _md5 (as below). _hashlib cannot >> be found with the path for some reason. >> >> Traceback (most recent call last): >> File "scramble.py", line 11, in <module> >> use_setuptools( download_delay=8, to_dir=scramble_lib ) >> File >> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", >> line 91, in use_setuptools >> return do_download() >> File >> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", >> line 85, in do_download >> egg = download_setuptools(version, download_base, to_dir, >> download_delay) >> File >> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", >> line 120, in download_setuptools >> import urllib2, shutil >> File >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", >> line 91, in <module> >> import hashlib >> File >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py", >> line 133, in <module> >> md5 = __get_builtin_constructor('md5') >> File >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py", >> line 60, in __get_builtin_constructor >> import _md5 >> ImportError: No module named _md5 > > Okay, what's happening here is that in the egg's source directory, scramble > creates a script which it then executes in a clean environment using python > -ES. Since your hashlib/md5 isn't part of the base install, you'll need to > remove the -ES from line 132 of lib/galaxy/eggs/__init__.py: > > cmd = "%s -ES %s" % ( self.python, "scramble.py" ) > > becomes: > > cmd = "%s %s" % ( self.python, "scramble.py" ) > > Try this out and let me know if it works. > > --nate >

2 3

Re: [galaxy-user] workflow question
by Michael Rusch 12 Sep '08

12 Sep '08

Thanks for checking me, I was looking at our local instance which I hadn't updated for awhile. My bad. I just updated, and it works great. Thanks, Michael > -----Original Message----- > From: James Taylor [mailto:james@bx.psu.edu] > Sent: Friday, September 12, 2008 10:59 AM > To: Michael Rusch > Subject: Re: [galaxy-user] workflow question > > Did this happen recently? The problem was that connectors were > capturing the mouse, and preventing other things under them from being > dragged. However this should be completely fixed. Just checked and I > can't reproduce in Firefox. His problem was with IE6, which is > unsupported. > > On Sep 12, 2008, at 11:27 AM, Michael Rusch wrote: > > > I've had a similar problem, using Firefox 3.0.1. > > > > In my case, you can generally drag stuff just fine at first, but as > > time > > goes on, fewer and fewer of the boxes will drag, until finally, > > there is no > > response to any clicks in the canvas. Last time I did it, I was > > able to add > > two tools, and once I connected them, I could suddenly only get the > > mouse > > pointer to change by hovering over a small area on one of the > > tools. I was > > able to move it one more time, and then everything was non-responsive. > > > > Michael > > > >> -----Original Message----- > >> From: galaxy-user-bounces(a)bx.psu.edu [mailto:galaxy-user- > >> bounces(a)bx.psu.edu] On Behalf Of James Taylor > >> Sent: Friday, September 12, 2008 3:37 AM > >> To: Jose Jorge Galan > >> Cc: galaxy-user(a)bx.psu.edu > >> Subject: Re: [galaxy-user] workflow question > >> > >> Hello, > >> > >> I'm sorry you are having trouble. Can you let us know which galaxy > >> server are you using, and what browser. > >> > >> Thanks, > >> James > >> > >> On Sep 12, 2008, at 2:23 AM, Jose Jorge Galan wrote: > >> > >>> Dear Sirs, > >>> > >>> I am trying to create a workflow from scratch although it has been > >>> impossible for me. > >>> The simple problem is that I cannot move the boxes through the > >>> canvas. > >>> Each time I add a new box it is sent to the same position in the > >>> canvas, all the boxes overlap and I am not able to move or connect > >>> them. > >>> According to the screencast "Workflow from scratch" when you put the > >>> cursor on the top of each box, a pam icon appears and you are then > >>> able to move the boxes representing both, input datasets and tools. > >>> However, when I try to do so, a cross icon instead of a pam icon > >>> appears and it does not allow me to move to boxes. > >>> > >>> Any help in this topic is welcome. > >>> > >>> Yours Faithfully, > >>> > >>> Jose Jorge Galan, PhD > >>> > >>> Connect to the next generation of MSN Messenger Get it now! > >>> _______________________________________________ > >>> galaxy-user mailing list > >>> galaxy-user(a)bx.psu.edu > >>> http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > >> > >> _______________________________________________ > >> galaxy-user mailing list > >> galaxy-user(a)bx.psu.edu > >> http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > > > > _______________________________________________ > > galaxy-user mailing list > > galaxy-user(a)bx.psu.edu > > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user

1 0

Fwd: Install problems on OS X with python from macports
by James Casbon 12 Sep '08

12 Sep '08

Oops, missed the mailing list on this reply... ---------- Forwarded message ---------- From: James Casbon <casbon(a)gmail.com> Date: 2008/9/12 Subject: Re: [galaxy-user] Install problems on OS X with python from macports To: Nate Coraor <nate(a)bx.psu.edu> 2008/9/12 Nate Coraor <nate(a)bx.psu.edu>: > Hi James, > > James Casbon wrote: > >> I'm trying to do a local install of Galaxy. My python version is from >> MacPorts. I realize that you recommend using MacPython, but since all >> the dev environment I have uses MacPorts, I thought I'd give it a go. >> BTW, it does work with the shipped python 2.5 with OS X. > > Thanks! Good to know. > >> The problem I have is that the egg downloading part (scramble.py, >> etc.) seems to monkey with the pythonpath such that _hashlib cannot be >> imported. I tried to track what happens, but it is not obvious where >> and why the path is being changed. > > I'm not sure what's trying to import _hashlib... but what's stranger is that > it should be part of the base python install, and nothing we do affects the > base install. We do start Galaxy with these flags: It's the egg downloader doing md5s on the files. I think there are two ways hashlib can work, and I think the macports way is to use the _hashlib c implementation, which goes missing from the path. This is probably because MacPorts choose to split out hashlib from the base install. > > -E Ignore environment variables like PYTHONPATH and PYTHONHOME that > modify the behavior of the interpreter. > > -S Disable the import of the module site and the site-dependent > manipulations of sys.path that it entails. > > This is so we can limit conflicts between our provided dependencies, and any > you may have installed locally. > > If you start the interactive interpreter, are you able to import _hashlib? > How about with -E and/or -S? Any output you can provide from where it > fails (I assume in scramble.py?) would be helpful. I can get hashlib from the interactive shell. Is there a way to get a shell with the galaxy environment? Here is the traceback. If you read hashlib, it tries to import _hashlib and if that fails and so it tries to get _md5. The MacPorts python should have _hashlib but not _md5 (as below). _hashlib cannot be found with the path for some reason. Traceback (most recent call last): File "scramble.py", line 11, in <module> use_setuptools( download_delay=8, to_dir=scramble_lib ) File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", line 91, in use_setuptools return do_download() File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", line 85, in do_download egg = download_setuptools(version, download_base, to_dir, download_delay) File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py", line 120, in download_setuptools import urllib2, shutil File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py", line 91, in <module> import hashlib File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py", line 133, in <module> md5 = __get_builtin_constructor('md5') File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor import _md5 ImportError: No module named _md5 > >> How can I safely add something to galaxy's pythonpath and ensure it is >> not removed? > > Remove -E from run.sh, if necessary, but this may cause other problems. > Tools do run without these flags so they can find other site-installed > dependencies (such as numpy, etc.). OK I will try these flags out and see what happens. Thanks for the quick reply, James

1 0

Re: [galaxy-user] local sequence storage]
by Greg Von Kuster 12 Sep '08

12 Sep '08

4 3

Install problems on OS X with python from macports
by James Casbon 12 Sep '08

12 Sep '08

Hi Everyone, I'm trying to do a local install of Galaxy. My python version is from MacPorts. I realize that you recommend using MacPython, but since all the dev environment I have uses MacPorts, I thought I'd give it a go. BTW, it does work with the shipped python 2.5 with OS X. The problem I have is that the egg downloading part (scramble.py, etc.) seems to monkey with the pythonpath such that _hashlib cannot be imported. I tried to track what happens, but it is not obvious where and why the path is being changed. How can I safely add something to galaxy's pythonpath and ensure it is not removed? thanks, James

2 1

linking into galaxy
by Michael Rusch 05 Sep '08

05 Sep '08

I want to construct a link into galaxy that will show a particular dataset. The use case is that I have an external resource that references a galaxy dataset, and I want to provide a way to link to it. I will need to create these links fairly frequently to various datasets. I'm somewhat open as to how this could work. I'm running a local instance, so I have the ability to peek into the db to get any internal IDs that I might need for this purpose. I'm thinking a minimal implementation would be to get a history that includes the dataset and link to it based on that, though I'm not sure how I would build the URL. The next step up would be if there was some way for it to somehow highlight the particular dataset in the history. Or maybe scroll to it. Best thing would be a link that only required information about the dataset and not what histories it's a part of and could pick a history, or create a new history with only that dataset, something like that. Thanks, Michael

2 3