- galaxy-dev - lists.galaxyproject.org

Python issue on cluster
by michael xu 31 Aug '12

31 Aug '12

Hi, Recently, I tried to run galaxy on our cluster and got the following error messages when I was trying to run samtools. Everything is running well when I run it on a local machine. Once I switch it to cluster, the conversion could not be successfully executed. I checked the version of python, they are all v2.6.6 on both head and nodes machines. Anybody can help me on this issue? Thanks a lot in advance! ================================================== Traceback (most recent call last): File "/home/gxu/galaxy_dist/tools/samtools/bam_to_sam.py", line 10, in <module> from galaxy import eggs File "/home/gxu/galaxy_dist/lib/galaxy/__init__.py", line 5, in <module> __import__( "pkg_resources" ).declare_namespace( __name__ ) File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 698, in <module> class Environment(object): File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 701, in Environment def __init__(self, search_path=None, platform=get_supported_platform(), python=PY_MAJOR): File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 96, in get_supported_platform plat = get_build_platform(); m = macosVersionString.match(plat) File "/home/gxu/galaxy_dist/lib/pkg_resources.py", line 221, in get_build_platform from distutils.util import get_platform ImportError: No module named distutils.util ==================================================

1 0

GTF, GFF and gff_filter_by_attribute
by Björn Grüning 31 Aug '12

31 Aug '12

Hi, i have found a small problem in gff_filter_by_attribute and not sure how to fix it properly. gff_filter_by_attribute.xml accepts gff data types but crashes if you run it with gff. Because metadata.attribute_types is not defined for gff and the dictionary is empty. For gtf files it works, because gtf has metadata.attribute_types defined. Should we extend the gff data type to also set metadata.attribute_types or is it for some reason better to create a gff<->gtf converter and chance gff_filter_by_attribute.xml to only accept gtf files? Thanks! Bjoern -- Björn Grüning Albert-Ludwigs-Universität Freiburg Institute of Pharmaceutical Sciences Pharmaceutical Bioinformatics Hermann-Herder-Strasse 9 D-79104 Freiburg i. Br. Tel.: +49 761 203-4872 Fax.: +49 761 203-97769 E-Mail: bjoern.gruening(a)pharmazie.uni-freiburg.de Web: http://www.pharmaceutical-bioinformatics.org/

2 1

Socket error starting galaxy
by Gaueko Erge 31 Aug '12

31 Aug '12

Hi, On a fresh galaxy installation the server start fine when it looks at port 8080, but as soon as I change to port 80 I get the following error I think that is due to permissions since I have my galaxy inside a non-root galaxy user directory Any ideas on the best way to solve this would be appreciated Thanks --G Starting server in PID 8799. Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 303, in command serve() File "/usr/local/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 287, in serve server(app) File "/usr/local/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 151, in server_wrapper **context.local_conf) File "/usr/local/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1314, in server_runner serve(wsgi_app, **kwargs) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1264, in serve threadpool_options=threadpool_options) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1114, in __init__ RequestHandlerClass, ssl_context) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1094, in __init__ RequestHandlerClass, ssl_context) File "/usr/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 328, in __init__ HTTPServer.__init__(self, server_address, RequestHandlerClass) File "/usr/lib/python2.7/SocketServer.py", line 408, in __init__ self.server_bind() File "/usr/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind SocketServer.TCPServer.server_bind(self) File "/usr/lib/python2.7/SocketServer.py", line 419, in server_bind self.socket.bind(self.server_address) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 13] Permission denied

3 2

generating composite files
by Jorrit Boekel 31 Aug '12

31 Aug '12

Dear list, I'm on galaxy-dist. I have a tool that outputs an unknown amount of files in a directory, all of which I need for input to the next tool. I used to tar them, but after a tip from John Chilton, I tried to fit this into Galaxy's composite datatypes. The tool now works, but I wonder if there is a better way to do it than how I have implemented it. I ran into some trouble when my new datatype's generate_primary_file was never called. (Is this only used after an upload?) My tool now outputs to the output.extra_files_path given to it on the xml command line, and writes an HTML file to the output file it is given. The next tool reads input.extra_files_path, and uses the files there. In other words, no use has been made of add_composite_file, generate_primary_file, etc. I can even set composite_file=None in the datatype, and it will still work. The whole thing feels a bit like a hack, so I wanted to ask if there is a way for tools to generate composite files the same way uploads can. cheers, jorrit boekel

1 0

missing slash when installing from toolshed?
by Dan Sullivan 30 Aug '12

30 Aug '12

Hi, Galaxy Developers, I have a question that is very similar to the following thread; http://dev.list.galaxyproject.org/Problem-fetching-updates-to-toolshed-tool… Basically whenever I try to install a tool from the toolshed, it appears that a trailing slash is not appended to the Galaxy URL, which is causing a DNS lookup failure. For example, the URL specified in the attached screenshot should be https://crigalaxy.uchicago.edu/admin_toolshed, however it is appearing as https://crigalaxy.uchicago.eduadmin_toolshed. Does anybody know if there are any known bugs that would cause this behavior? I might venture down the road of modifying lib/galaxy/webapps/community/controllers/repository.py as suggested by the thread above, although I would like to avoid doing this if possible. I am seeing experiencing this behavior on the following version of Galaxy: > hg parent changeset: 7400:ec29ce8e27a1 In addition to this, I feel that its worthwhile to mention that I am using the apache proxy balancer with web scaling, however based on what I have seen and tested, I do not believe this (apache) to be the root cause of my problem. The URL below is a screenshot of the network debug I am seeing from the firebug extension in firefox. https://webshare.uchicago.edu/users/dansully/Public/Screen%20Shot%202012-08… If somebody has seen this, or could offer a suggestion to bring this issue to resolve, I would definitely be interested in your strategy. Thank-you so much for your help and for your time. Dan Sullivan 312-607-3702

3 4

Setup questions: data locations, MySQL, and cores
by Steven Peckins 30 Aug '12

30 Aug '12

I've been asked to set up a local Galaxy installation specifically for large datasets (tens of terabytes). Is there a list of default locations where Galaxy puts data? As an admin, that would be my first question, but it's not obvious from the documentation, and for large datasets, it's important to know. (During testing, with Galaxy in the default location, /home/galaxy, an attempt to upload and decompress a 50GB file wreaked havoc on the server due to heavy I/O. Changing the tmp directory so it wasn't reading and writing massive amounts of data to the same filesystem at the same time helped, but that would have been nice to know ahead of time.) Is the database used for datasets at all or just user account data? In other words, if users are crunching terabytes of data, do I need to worry about the amount of space on the filesystem that hosts the database? What exactly are the disadvantages of using MySQL over PostgreSQL? Several places in the docs state that it is preferred but not why. Is it a bigger problem than installing another database that I have no experience with over one that is already installed and with which I am already familiar? Is an MPI configuration necessary for getting full use out of a multicore system? The docs seem to indicate that Galaxy will use multiple cores ("Without a cluster, you'll be limited to the number of cores in your server...") but take pains to say that GIL won't allow more than a single thread ("This means that regardless of the number of cores in your server, Galaxy can only use one" and "having a multi-core system will not improve the Galaxy framework's performance out of the box since Galaxy can use (at most) one core at a time"). We have a 48-core machine, and there will only be two or three users. Thanks, --steve

2 1

Feature Request: Nested sections in tool_conf.xml
by Ted Goldstein 30 Aug '12

30 Aug '12

Dear Galaxy Developers, First, thanks for the great work. Galaxy is a amazingly well architected and extraordinary useful system. Of course, I have a feature request to make it even more perfect ;-) The list of tool sections is already very long. It would be nice to organize the sections in nested sections. For example: <toolbox> <section name="Percussive Hammer" id="hammer"> <section name="hand hammers" id="hammer"> <tool file="Air" id="Air hammer"> <tool file="Blacksmith" id="Blacksmith hammer"> <tool file="Body" id="Body hammer"> <tool file="Bossing mallet" id="Bossing mallet"> <tool file="Brass" id="Brass hammer"> <tool file="Brass" id="Brass hammer"> <tool file="Bricklayer" id="Bricklayer hammer"> <tool file="Bumping" id="Bumping hammer"> <tool file="Chipping" id="Chipping hammer"> <tool file="Claw" id="Claw hammer"> <tool file="Copper-faced" id="Copper-faced hammer"> <tool file="Finish" id="Finish hammer"> <tool file="Nail" id="Nail hammer"> <tool file="Nylon" id="Nylon hammer"> <tool file="Panel" id="Panel hammer"> <tool file="Pick" id="Pick hammer"> <tool file="Planishing" id="Planishing hammer"> <tool file="Ripping" id="Ripping hammer"> <tool file="Riveting" id="Riveting hammer"> <tool file="Rotary" id="Rotary hammer"> <tool file="Rubber mallet" id="Rubber mallet"> <tool file="Shingler's" id="Shingler's hammer"> <tool file="Shrinking" id="Shrinking hammer"> <tool file="Sledge" id="Sledge hammer"> <tool file="Slide" id="Slide hammer"> <tool file="Tack" id="Tack hammer"> <tool file="Tinners" id="Tinner's hammer"> <tool file="Wooden_mallet" id="Wooden mallet"> </section> <section name="Power hammer" id="power_hammer"> <tool file="Jack Hammer " id="jackhammer"> <tool file="Electromechanical Hammer " id="e_jackhammer"> <tool file="Pneumatic Hammer " id="p_jackhammer"> <tool file="Hydraulic Hammer " id="h_jackhammer"> </section> .. etc .. </section> <section name="Wrenches" id="wrenches"> <section name="Power wrench" id="power_wrench"> .. etc .. </section> <section name="hand wrench" id="hand_wrench"> .. etc .. </section> </section> </toolbox> BTW, Its great that tools can conveniently be in more than one section! This makes it every useful to group common tasks together. Best regards, Ted

1 0

Re: [galaxy-dev] Blastxml error on Galaxy Cloudman
by Peter Cock 30 Aug '12

30 Aug '12

Hi Mohammad, Don't forget to CC the mailing list ;) It sounds like something has gone wrong with the update... Peter On Wednesday, August 29, 2012, Mohammad Heydarian wrote: > Hi Peter, > Thanks for your response. > > I am not using any of the Blast tools, I have only tried using the "filter > and sort" and "text manipulation" tools and gotten this error. > > Could this be fixed by re-updating my instance of Galaxy Cloudman? Or some > similar fix? I am not able to install tools to my instance of Galaxy > Cloudman, the error I get is: > The *tool_config_file* setting in *universe_wsgi.ini* must include at > least one shed tool configuration file name with a *<toolbox>* tag that > includes a *tool_path* attribute value which is a directory relative to > the Galaxy installation directory in order to automatically install tools > from a Galaxy tool shed (e.g., the file name *shed_tool_conf.xml* whose * > <toolbox>* tag is *<toolbox tool_path="../shed_tools">*). > > > Cheers, > Mo Heydarian > > PhD candidate > The Johns Hopkins School of Medicine > Department of Biological Chemistry > 725 Wolfe Street > 414 Hunterian > Baltimore, MD 21205 > > > > On Wed, Aug 29, 2012 at 3:24 PM, Peter Cock <p.j.a.cock(a)googlemail.com<javascript:_e({}, 'cvml', 'p.j.a.cock(a)googlemail.com');> > > wrote: > >> Hi >> >> The BLAST+ tools and the definition of the blastxml format were moved to >> the tool shed. You should have seen a migration message during the update... >> >> If you are not using these tools, perhaps you have a legacy reference to >> the blastxml file which should be removed? >> >> If you are using BLAST+ they and the blastxml format need to be installed >> from the ToolShed, >> >> Peter >> >> On Wednesday, August 29, 2012, Mohammad Heydarian wrote: >> >>> Hello, >>> I recently updated my instance of Galaxy Cloudman (using Amazon AWS) and >>> I have since not been able to run any jobs. I get the following error: >>> >>> WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml' >>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml' >>> >>> >>> Any help would greatly be appreciated. >>> >>> >>> Cheers, >>> Mo Heydarian >>> >>> >

2 1

work flow tool saving problem
by YuFei Luo 30 Aug '12

30 Aug '12

Hi, I just added a new tool to my existed work flow. And I would like to set the parameter of this tool to 'set at runtime', but I got an error message as following: Server error URL: http://127.0.0.1:8085/workflow/editor_form_post Module weberror.evalexception.middleware:*364* in respond >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>app_iter *=* self*.*application*(*environ*,* detect_start_response*)* Module paste.debug.prints:*98* in __call__ >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>environ*,* self*.*app*)* Module paste.wsgilib:*539* in intercept_output >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>app_iter *=* application*(*environ*,* replacement_start_response*)* Module paste.recursive:*80* in __call__ >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>*return* self*.*application*(*environ*,* start_response*)* Module paste.httpexceptions:*632* in __call__ >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>*return* self*.*application*(*environ*,* start_response*)* Module galaxy.web.framework.base:*160* in __call__ >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>body *=* method*(* trans*,* ****kwargs *)* Module galaxy.web.framework:*67* in decorator >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>*return* simplejson*.*dumps*(* func*(* self*,* trans*,* ***args*,* ****kwargs *)* *)* Module galaxy.web.controllers.workflow:*706* in editor_form_post >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>'tool_state' *:* incoming*.*pop*(*"tool_state"*)* Module galaxy.workflow.modules:*363* in from_dict >> <http://127.0.0.1:8085/workflow/editor?id=d413a19dec13d11e#>*assert* type *in* self*.*module_types *AssertionError: * [] And when I added an annotation/notes, and I wanted to save this work flow, I got the same error. Any idea? Thanks! Yufei

1 0

LSF DRMAA/FedStage config-make errors
by Douglas W Cloud 30 Aug '12

30 Aug '12

Hello all, I am trying to get a Galaxy server configured to submit jobs to an LSF cluster running version 7 update 2, generally CentOS/Redhat 5 systems. I have the Galaxy host as a client in LSF, but when I tried to configure/make/install the FedStage drmaa code, it errors out in the make stage (see stdout below). Don't know what to fix, if it's possible. If anyone has experience configuring/troubleshooting this code to get Galaxy conversant with LSF, I'd appreciate any feedback. Thanks! Regards, Doug _________ util.c: In function 'lsfdrmaa_map_lsberrno': util.c:730: error: 'LSBE_ADRSV_ID_UNIQUE' undeclared (first use in this function) util.c:730: error: (Each undeclared identifier is reported only once util.c:730: error: for each function it appears in.) util.c:731: error: 'LSBE_BAD_RSVNAME' undeclared (first use in this function) util.c:732: error: 'LSBE_BAD_CHKPNTDIR' undeclared (first use in this function) util.c:734: error: 'LSBE_JOB_REQUEUE_BADEXCLUDE' undeclared (first use in this function) util.c:737: error: 'LSBE_ADVRSV_ACTIVESTART' undeclared (first use in this function) util.c:739: error: 'LSBE_ADRSV_ID_USED' undeclared (first use in this function) util.c:740: error: 'LSBE_ADRSV_PREVDISABLED' undeclared (first use in this function) util.c:742: error: 'LSBE_ADRSV_DISABLECURR' undeclared (first use in this function) util.c:744: error: 'LSBE_ADRSV_NOT_RSV_HOST' undeclared (first use in this function) util.c:749: error: 'LSBE_ADRSV_MOD_REMOTE' undeclared (first use in this function) util.c:751: error: 'LSBE_JOB_RUSAGE_EXCEED_LIMIT' undeclared (first use in this function) util.c:752: error: 'LSBE_APP_RUSAGE_EXCEED_LIMIT' undeclared (first use in this function) util.c:755: error: 'LSBE_ADRSV_DISABLE_DATE' undeclared (first use in this function) util.c:759: error: 'LSBE_ADRSV_DETACH_MIX' undeclared (first use in this function) util.c:761: error: 'LSBE_ADRSV_DETACH_ACTIVE' undeclared (first use in this function) util.c:765: error: 'LSBE_MISSING_START_END_TIME' undeclared (first use in this function) make[3]: *** [libdrmaa_la-util.lo] Error 1 make[3]: Leaving directory `/root/installers/lsf_drmaa-1.0.4/lsf_drmaa' make[2]: *** [all] Error 2 make[2]: Leaving directory `/root/installers/lsf_drmaa-1.0.4/lsf_drmaa' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/installers/lsf_drmaa-1.0.4' (Same output with version 1.0.3.) This communication is for use by the intended recipient and contains information that may be Privileged, confidential or copyrighted under applicable law. If you are not the intended recipient, you are hereby formally notified that any use, copying or distribution of this e-mail, in whole or in part, is strictly prohibited. Please notify the sender by return e-mail and delete this e-mail from your system. Unless explicitly and conspicuously designated as "E-Contract Intended", this e-mail does not constitute a contract offer, a contract amendment, or an acceptance of a contract offer. This e-mail does not constitute a consent to the use of sender's contact information for direct marketing purposes or for transfers of data to third parties. Francais Deutsch Italiano Espanol Portugues Japanese Chinese Korean http://www.DuPont.com/corp/email_disclaimer.html

2 1