accessing data from the filesystem
by James Casbon
Hi,
Is it possible to access data on the filesystem without uploading it.
i.e. I install galaxy locally, and then want to refer to some huge
fasta file on disk. At the moment, I can upload it, which is slow and
creates two copies. Can I just point galaxy at it somehow?
thanks,
James
12 years, 3 months
patch: use xml includes when parsing xml
by James Casbon
Hi,
I have inlined a patch that allows xinclude statements in xml files.
I presume I cannot send attachments to the list?
Motivation: I have my own tools I am developing managed in
tools/my_tools. I want to version that directory, etc, but want to be
able to include new tools without editing the tools_conf.xml in
galaxy_dist each time which would not get versioned.
Solution: use xinclude to include an xml file that contains tool definitions.
Example tool_conf.xml:
<toolbox xmlns:xi="http://www.w3.org/2001/XInclude">
....
<xi:include href="tools/my_tools/tool_conf.xml" parse="xml" />
</toolbox>
Patch below,
cheers,
James
==============
diff -r 3b6771227a43 lib/galaxy/util/__init__.py
--- a/lib/galaxy/util/__init__.py Fri Sep 26 09:10:04 2008 -0400
+++ b/lib/galaxy/util/__init__.py Mon Sep 29 15:50:20 2008 +0100
@@ -12,7 +12,7 @@
from galaxy.util.docutils_ext.htmlfrag import Writer as HTMLFragWriter
pkg_resources.require( 'elementtree' )
-from elementtree import ElementTree
+from elementtree import ElementTree, ElementInclude
log = logging.getLogger(__name__)
_lock = threading.RLock()
@@ -64,6 +64,8 @@
def parse_xml(fname):
"""Returns an parsed xml tree"""
tree = ElementTree.parse(fname)
+ root = tree.getroot()
+ ElementInclude.include(root)
return tree
def xml_to_string(elem):
================
12 years, 3 months
LaunchDaemon for OS X
by James Casbon
Here is a LaunchDaemon for OS X, maybe useful. Need to change the
paths for your install.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>edu.psu.galaxy</string>
<key>OnDemand</key>
<false/>
<key>Program</key>
<string>/Users/james/galaxy-dist/run.sh<string>
<key>RunAtLoad</key>
<true/>
<key>ServiceDescription</key>
<string>Galaxy</string>
<key>StandardErrorPath</key>
<string>/Users/james/galaxy-dist/logs/launchd.stderr</string>
<key>StandardOutPath</key>
<string>/Users/james/galaxy-dist/logs/launchd.stdout</string>
<key>UserName</key>
<string>james</string>
</dict>
</plist>
12 years, 3 months
Re: [galaxy-user] Install problems on OS X with python from macports
by James Casbon
Hi Nate,
Your fix works for the scramble fine, but not for the run.sh which now
fails. On removing the -ES from run.sh, I get an error importing
simplejson (see below). simplejson.egg is in the
eggs/py2.5-noplatform dir so I dont' see why it doesn't pick this up
if it manages to pick up WebHelpers.
Traceback (most recent call last):
File "./scripts/paster.py", line 26, in <module>
command.run()
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py",
line 78, in run
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py",
line 117, in invoke
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/command.py",
line 212, in run
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/serve.py",
line 227, in command
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteScript-1.3.6-py2.5.egg/paste/script/serve.py",
line 250, in loadapp
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 193, in loadapp
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 213, in loadobj
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 237, in loadcontext
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 267, in _loadconfig
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 397, in get_context
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 439, in _context_from_explicit
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/PasteDeploy-1.3.1-py2.5.egg/paste/deploy/loadwsgi.py",
line 18, in import_string
File "/Users/james/Src/galaxy_dist/lib/pkg_resources.py", line 1912, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/Users/james/Src/galaxy_dist/lib/galaxy/web/__init__.py", line
5, in <module>
from framework import expose, json, require_login, url_for, error,
form, FormBuilder
File "/Users/james/Src/galaxy_dist/lib/galaxy/web/framework/__init__.py",
line 17, in <module>
import webhelpers
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/__init__.py",
line 1, in <module>
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/rails/__init__.py",
line 7, in <module>
File "/Users/james/Src/galaxy_dist/eggs/py2.5-noplatform/WebHelpers-0.2-py2.5.egg/webhelpers/rails/scriptaculous.py",
line 16, in <module>
ImportError: No module named simplejson
2008/9/12 Nate Coraor <nate(a)bx.psu.edu>:
> James Casbon wrote:
>
>> Here is the traceback. If you read hashlib, it tries to import
>> _hashlib and if that fails and so it tries to get _md5. The MacPorts
>> python should have _hashlib but not _md5 (as below). _hashlib cannot
>> be found with the path for some reason.
>>
>> Traceback (most recent call last):
>> File "scramble.py", line 11, in <module>
>> use_setuptools( download_delay=8, to_dir=scramble_lib )
>> File
>> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
>> line 91, in use_setuptools
>> return do_download()
>> File
>> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
>> line 85, in do_download
>> egg = download_setuptools(version, download_base, to_dir,
>> download_delay)
>> File
>> "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
>> line 120, in download_setuptools
>> import urllib2, shutil
>> File
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py",
>> line 91, in <module>
>> import hashlib
>> File
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py",
>> line 133, in <module>
>> md5 = __get_builtin_constructor('md5')
>> File
>> "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py",
>> line 60, in __get_builtin_constructor
>> import _md5
>> ImportError: No module named _md5
>
> Okay, what's happening here is that in the egg's source directory, scramble
> creates a script which it then executes in a clean environment using python
> -ES. Since your hashlib/md5 isn't part of the base install, you'll need to
> remove the -ES from line 132 of lib/galaxy/eggs/__init__.py:
>
> cmd = "%s -ES %s" % ( self.python, "scramble.py" )
>
> becomes:
>
> cmd = "%s %s" % ( self.python, "scramble.py" )
>
> Try this out and let me know if it works.
>
> --nate
>
12 years, 4 months
Re: [galaxy-user] workflow question
by Michael Rusch
Thanks for checking me, I was looking at our local instance which I hadn't
updated for awhile. My bad.
I just updated, and it works great.
Thanks,
Michael
> -----Original Message-----
> From: James Taylor [mailto:james@bx.psu.edu]
> Sent: Friday, September 12, 2008 10:59 AM
> To: Michael Rusch
> Subject: Re: [galaxy-user] workflow question
>
> Did this happen recently? The problem was that connectors were
> capturing the mouse, and preventing other things under them from being
> dragged. However this should be completely fixed. Just checked and I
> can't reproduce in Firefox. His problem was with IE6, which is
> unsupported.
>
> On Sep 12, 2008, at 11:27 AM, Michael Rusch wrote:
>
> > I've had a similar problem, using Firefox 3.0.1.
> >
> > In my case, you can generally drag stuff just fine at first, but as
> > time
> > goes on, fewer and fewer of the boxes will drag, until finally,
> > there is no
> > response to any clicks in the canvas. Last time I did it, I was
> > able to add
> > two tools, and once I connected them, I could suddenly only get the
> > mouse
> > pointer to change by hovering over a small area on one of the
> > tools. I was
> > able to move it one more time, and then everything was non-responsive.
> >
> > Michael
> >
> >> -----Original Message-----
> >> From: galaxy-user-bounces(a)bx.psu.edu [mailto:galaxy-user-
> >> bounces(a)bx.psu.edu] On Behalf Of James Taylor
> >> Sent: Friday, September 12, 2008 3:37 AM
> >> To: Jose Jorge Galan
> >> Cc: galaxy-user(a)bx.psu.edu
> >> Subject: Re: [galaxy-user] workflow question
> >>
> >> Hello,
> >>
> >> I'm sorry you are having trouble. Can you let us know which galaxy
> >> server are you using, and what browser.
> >>
> >> Thanks,
> >> James
> >>
> >> On Sep 12, 2008, at 2:23 AM, Jose Jorge Galan wrote:
> >>
> >>> Dear Sirs,
> >>>
> >>> I am trying to create a workflow from scratch although it has been
> >>> impossible for me.
> >>> The simple problem is that I cannot move the boxes through the
> >>> canvas.
> >>> Each time I add a new box it is sent to the same position in the
> >>> canvas, all the boxes overlap and I am not able to move or connect
> >>> them.
> >>> According to the screencast "Workflow from scratch" when you put the
> >>> cursor on the top of each box, a pam icon appears and you are then
> >>> able to move the boxes representing both, input datasets and tools.
> >>> However, when I try to do so, a cross icon instead of a pam icon
> >>> appears and it does not allow me to move to boxes.
> >>>
> >>> Any help in this topic is welcome.
> >>>
> >>> Yours Faithfully,
> >>>
> >>> Jose Jorge Galan, PhD
> >>>
> >>> Connect to the next generation of MSN Messenger Get it now!
> >>> _______________________________________________
> >>> galaxy-user mailing list
> >>> galaxy-user(a)bx.psu.edu
> >>> http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user
> >>
> >> _______________________________________________
> >> galaxy-user mailing list
> >> galaxy-user(a)bx.psu.edu
> >> http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user
> >
> > _______________________________________________
> > galaxy-user mailing list
> > galaxy-user(a)bx.psu.edu
> > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user
12 years, 4 months
Fwd: Install problems on OS X with python from macports
by James Casbon
Oops, missed the mailing list on this reply...
---------- Forwarded message ----------
From: James Casbon <casbon(a)gmail.com>
Date: 2008/9/12
Subject: Re: [galaxy-user] Install problems on OS X with python from macports
To: Nate Coraor <nate(a)bx.psu.edu>
2008/9/12 Nate Coraor <nate(a)bx.psu.edu>:
> Hi James,
>
> James Casbon wrote:
>
>> I'm trying to do a local install of Galaxy. My python version is from
>> MacPorts. I realize that you recommend using MacPython, but since all
>> the dev environment I have uses MacPorts, I thought I'd give it a go.
>> BTW, it does work with the shipped python 2.5 with OS X.
>
> Thanks! Good to know.
>
>> The problem I have is that the egg downloading part (scramble.py,
>> etc.) seems to monkey with the pythonpath such that _hashlib cannot be
>> imported. I tried to track what happens, but it is not obvious where
>> and why the path is being changed.
>
> I'm not sure what's trying to import _hashlib... but what's stranger is that
> it should be part of the base python install, and nothing we do affects the
> base install. We do start Galaxy with these flags:
It's the egg downloader doing md5s on the files. I think there are
two ways hashlib can work, and I think the macports way is to use the
_hashlib c implementation, which goes missing from the path. This is
probably because MacPorts choose to split out hashlib from the base
install.
>
> -E Ignore environment variables like PYTHONPATH and PYTHONHOME that
> modify the behavior of the interpreter.
>
> -S Disable the import of the module site and the site-dependent
> manipulations of sys.path that it entails.
>
> This is so we can limit conflicts between our provided dependencies, and any
> you may have installed locally.
>
> If you start the interactive interpreter, are you able to import _hashlib?
> How about with -E and/or -S? Any output you can provide from where it
> fails (I assume in scramble.py?) would be helpful.
I can get hashlib from the interactive shell. Is there a way to get a
shell with the galaxy environment?
Here is the traceback. If you read hashlib, it tries to import
_hashlib and if that fails and so it tries to get _md5. The MacPorts
python should have _hashlib but not _md5 (as below). _hashlib cannot
be found with the path for some reason.
Traceback (most recent call last):
File "scramble.py", line 11, in <module>
use_setuptools( download_delay=8, to_dir=scramble_lib )
File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
line 91, in use_setuptools
return do_download()
File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
line 85, in do_download
egg = download_setuptools(version, download_base, to_dir, download_delay)
File "/Users/james/Src/galaxy_dist/scripts/scramble/build/py2.5-macosx-10.3-i386-ucs2/Cheetah/ez_setup.py",
line 120, in download_setuptools
import urllib2, shutil
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib2.py",
line 91, in <module>
import hashlib
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py",
line 133, in <module>
md5 = __get_builtin_constructor('md5')
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/hashlib.py",
line 60, in __get_builtin_constructor
import _md5
ImportError: No module named _md5
>
>> How can I safely add something to galaxy's pythonpath and ensure it is
>> not removed?
>
> Remove -E from run.sh, if necessary, but this may cause other problems.
> Tools do run without these flags so they can find other site-installed
> dependencies (such as numpy, etc.).
OK I will try these flags out and see what happens.
Thanks for the quick reply,
James
12 years, 4 months
Install problems on OS X with python from macports
by James Casbon
Hi Everyone,
I'm trying to do a local install of Galaxy. My python version is from
MacPorts. I realize that you recommend using MacPython, but since all
the dev environment I have uses MacPorts, I thought I'd give it a go.
BTW, it does work with the shipped python 2.5 with OS X.
The problem I have is that the egg downloading part (scramble.py,
etc.) seems to monkey with the pythonpath such that _hashlib cannot be
imported. I tried to track what happens, but it is not obvious where
and why the path is being changed.
How can I safely add something to galaxy's pythonpath and ensure it is
not removed?
thanks,
James
12 years, 4 months
linking into galaxy
by Michael Rusch
I want to construct a link into galaxy that will show a particular dataset.
The use case is that I have an external resource that references a galaxy
dataset, and I want to provide a way to link to it. I will need to create
these links fairly frequently to various datasets.
I'm somewhat open as to how this could work. I'm running a local instance,
so I have the ability to peek into the db to get any internal IDs that I
might need for this purpose.
I'm thinking a minimal implementation would be to get a history that
includes the dataset and link to it based on that, though I'm not sure how I
would build the URL.
The next step up would be if there was some way for it to somehow highlight
the particular dataset in the history. Or maybe scroll to it.
Best thing would be a link that only required information about the dataset
and not what histories it's a part of and could pick a history, or create a
new history with only that dataset, something like that.
Thanks,
Michael
12 years, 4 months
