galaxy-dev March 2012

galaxy-dev@lists.galaxyproject.org

110 participants
202 discussions

Re: [galaxy-dev] Local Toolshed misbehaviours?
by Greg Von Kuster 23 Mar '12

23 Mar '12

On Mar 23, 2012, at 10:09 AM, Daniel Sobral wrote: > The path I defined on shed_tool_conf.xml should be writable. In fact, I > can install repositories from your toolsheds (galaxy main or galaxy > test) with no problems. I actually used the galaxy test toolshed to add > my own repositories as a workaround of having my own toolshed! The path > on shed_tool_conf is outside the galaxy-dist. > Ok, then could it be your apache rewrite rules? It's certainly something in your proprietary environment, but I have no idea what, in particular. > > The error seems to come from my toolshed. I'm running the galaxy and the > toolshed under the same galaxy clone, but I think this is the intended > usage, no? Yes, this is fine to do, but you should not be using the same database. > > Thanks for the help, > Daniel > > > On 03/23/2012 01:43 PM, Greg Von Kuster wrote: >> Hi Daniel, >> >> >>>> On Mar 22, 2012, at 11:21 AM, Daniel Sobral wrote: >>>> >>>>> Hi Greg, >>>>> >>>>> This is the output I get in the toolshed log: >>>>> " >>>>> Firefox/10.0.2" >>>>> 172.22.50.249 - - [22/Mar/2012:15:11:50 +0100] "GET >>>>> /toolshed/repository/install_repository_revision?repository_id=190cd6004dde15fd&changeset_revision=22619d71daa6&webapp=galaxy >>>>> HTTP/1.1" 302 - >>>>> " >>>>> >>>>> What I read from this is that someone tried to access the repository to >>>>> install the tool, which is correct. >> >> You should be seeing something like this instead, so it looks like the repository is not being cloned. >> >> >>>>>> Clicking the install button when browsing the tool shed from Galaxy will produce something like the following in your tool shed paster log: >>>>>> >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?pairs=0000000000000000000000000000000000000000-0000000000000000000000000000000000000000&cmd=between HTTP/1.1" 200 1 "-" "mercurial/proto-1.0" >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=heads HTTP/1.1" 200 41 "-" "mercurial/proto-1.0" >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=changegroup&roots=0000000000000000000000000000000000000000 HTTP/1.1" 200 - "-" "mercurial/proto-1.0" >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=capabilities HTTP/1.1" 200 79 "-" "mercurial/proto-1.0" >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=listkeys&namespace=bookmarks HTTP/1.1" 200 - "-" "mercurial/proto-1.0" >>>>>> 128.118.200.161 - - [16/Mar/2012:09:48:31 -0400] "GET /get_tool_versions?name=filter&owner=test&changeset_revision=8ab8044881c6&webapp=galaxy HTTP/1.1" 200 - "-" "Python-urllib/2.6" >>>>> And then I get this in the galaxy server log: >>>>> " >>>>> 172.22.50.249 - - [22/Mar/2012:15:11:56 +0100] "GET >>>>> /galaxy/admin_toolshed/browse_repositories?status=error&message=abort%3A+HTTP+Error+404%3A+Not+Found%0A%3Cbr%2F%3E >>>>> HTTP/1.1" 200 - >> >> This confirms that your repository is not cloned. Are you using the default file named shed_tool_conf.xml for this? What is the value of the "tool_path" attribute in the <tool> tag of this file? Is that directory writeable by the user that start up the Galaxy server? >> >

1 0

Re: [galaxy-dev] Local Toolshed misbehaviours?
by Greg Von Kuster 23 Mar '12

23 Mar '12

Hi Daniel, >> >> On Mar 22, 2012, at 11:21 AM, Daniel Sobral wrote: >> >>> Hi Greg, >>> >>> This is the output I get in the toolshed log: >>> " >>> Firefox/10.0.2" >>> 172.22.50.249 - - [22/Mar/2012:15:11:50 +0100] "GET >>> /toolshed/repository/install_repository_revision?repository_id=190cd6004dde15fd&changeset_revision=22619d71daa6&webapp=galaxy >>> HTTP/1.1" 302 - >>> " >>> >>> What I read from this is that someone tried to access the repository to >>> install the tool, which is correct. You should be seeing something like this instead, so it looks like the repository is not being cloned. >>>> Clicking the install button when browsing the tool shed from Galaxy will produce something like the following in your tool shed paster log: >>>> >>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?pairs=0000000000000000000000000000000000000000-0000000000000000000000000000000000000000&cmd=between HTTP/1.1" 200 1 "-" "mercurial/proto-1.0" >>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=heads HTTP/1.1" 200 41 "-" "mercurial/proto-1.0" >>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=changegroup&roots=0000000000000000000000000000000000000000 HTTP/1.1" 200 - "-" "mercurial/proto-1.0" >>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=capabilities HTTP/1.1" 200 79 "-" "mercurial/proto-1.0" >>>> 128.118.200.161 - - [16/Mar/2012:09:48:30 -0400] "GET /repos/test/filter?cmd=listkeys&namespace=bookmarks HTTP/1.1" 200 - "-" "mercurial/proto-1.0" >>>> 128.118.200.161 - - [16/Mar/2012:09:48:31 -0400] "GET /get_tool_versions?name=filter&owner=test&changeset_revision=8ab8044881c6&webapp=galaxy HTTP/1.1" 200 - "-" "Python-urllib/2.6" >>> >>> And then I get this in the galaxy server log: >>> " >>> 172.22.50.249 - - [22/Mar/2012:15:11:56 +0100] "GET >>> /galaxy/admin_toolshed/browse_repositories?status=error&message=abort%3A+HTTP+Error+404%3A+Not+Found%0A%3Cbr%2F%3E >>> HTTP/1.1" 200 - This confirms that your repository is not cloned. Are you using the default file named shed_tool_conf.xml for this? What is the value of the "tool_path" attribute in the <tool> tag of this file? Is that directory writeable by the user that start up the Galaxy server?

1 0

HTML output with Javascript
by Jose Navas 23 Mar '12

23 Mar '12

Hi Galaxy developers, I have a tool that generates an html output which executes some javascript functions that are defined in a *.js file. I put this file under the $output1.files_path but when I display the html file in the galaxy browser, it doesn't executes the Javascript. What can I do to execute Javascripts in the Galaxy browser? Thanks, Jose

2 3

hg19 built-in index is not listed
by Jennifer Jackson 22 Mar '12

22 Mar '12

Hi Jianpeng, For reference, this is the wiki with instructions: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup The first thing to double check is that the server has been restarted. This is necessary after adding new indexes. The next check is to walk through the setup steps again and make sure that the data organization is correct (directory structure), the indexing is valid (can you run Bowtie on the command line with a small sample using this hg19 index?), and the loc files have the correct content. Even minor differences in loc file data rows, such as a space instead of a tab (or the reverse), can prevent an index from appearing in the drop-down menu for a tool. After troubleshooting, please let us know if you continue to have problems. Let's keep the thread on the galaxy-dev(a)bx.psu.edu mailing list (instead of using the galaxy-dev-owner(a)lists.bx.psu.edu email, which is intended for resolving subscription problems). If we do need to share private data at some point, we can move to the galaxy-team-only galaxy-bugs(a)bx.psu.edu list. Thanks! Jen Galaxy team On 3/22/12 1:29 PM, Xu, Jianpeng wrote: > Hi , > > I have installed a local galaxy on our own server. And I have installed > the hg18 built-in index reference genome for bowtie, and it shows up in > the list. Then I installed the hg19 built-in index using the same > method. But the hg19 is not listed on the bowtie. Can you tell me why > the hg19 does not show up? We use the exact same method to do hg18 and > hg19. It seems that hg18 works fine, but hg19 does not work. > > Can you tell me how to fix it ? > > Thanks, > > Jianpeng > > > ------------------------------------------------------------------------ > > This e-mail message (including any attachments) is for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this message (including any attachments) is strictly > prohibited. > > If you have received this message in error, please contact > the sender by reply e-mail message and destroy all copies of the > original message (including attachments).

1 0

setuid and sudo in job submission
by Lance Bailey 22 Mar '12

22 Mar '12

The fellows here working on galaxy, about ready to release it to prime time, came to me asking about a few sudo commands that they needed. looking into it I came across a discussion (http://osdir.com/ml/galaxy-development-source-control/2011-11/msg00067.html) from last November talking about why this was being done and the security issues. Because I'm coming at this from the systems side, I've got a keen interest in the security aspect of things so I've dove into this with hopes of developing what might be a more secure approach. The issues that we have run into center around the sudo approach taken by the three scripts: external_chown_script, drmaa_external_killer.py and drmaa_external_runner.py. I have read the comments from Nov on galaxy-dev on using sudo but have taken a different approach. The issue with the external_chown_script is that is does a "sudo chown" and "sudo chgrp" allowing the galaxy id to chown any file on the system, not just ones involved with galaxy, and because this sudo permission is system wide, the permission is not just via the scripts, but extends to any command line with the galaxy id. Also, because the sudo is on "chown" and not "/bin/chown" there is a potential of PATH manipulation to substitute in a malicious or harmful "chown". This is a bad thing(tm). The issue with the other two scripts (drmaa_external_runner and drmaa_external_killer) scripts is that the sudo is on the entire script which can be edited after the sudo is setup to contain potentially malicious/harmful code. Also a bad thing(tm). But you know all that, and we all know that this is a tricky issue. 1. What I have done is to create a compiled C program, to replace external_chown_script.py. This program is intended to be installed setuid root. I have added checks to ensure that it is used as intended and then do the appropriate changes via the chown(2) system call. As this is to be installed setuid root, there is no longer a need for a sudo setting on chown(1). 2. The other two scripts, because of the midstream setuid to an arbitrary user need to also be setuid root compiled programs. After toying with some python compilers unsuccessfully, I just coded them in C. drama_external_killer.py is done, and I'm mostly finished drama_external_runner.py with the exception of parsing the JT_JSON file containing the job definitions. Because the guys working on the galaxy install haven't quite got it running, I don't have an example of a .jt_json file from which I can see samples of the input to map it into drmaa_set_attribute() and drama_set_vector_attribute() calls. Most of the keys are obvious (remoteCommand, outputPath, ...) but I don't know how to map the following Name - should this be DRMAA_JOB_NAME? Email - should this be DRMAA_V_EMAIL, and if so why is it a vector? Project - no idea where to map this aTdHvAaNnKcSe, I plan on releasing my code, when it hardens, to the galaxy community if there is interest in the setuid approach instead of the sudo approach to job submission/control __ cheersLance | Lance Bailey, Systems | GSC, 100-570 W 7th Ave | 604-707-5900 x5413 | Vancouver BC, V5Z 4S6 Canada

1 0

Cleanup script error - guess_dialect_for_url
by rocco.piazza 22 Mar '12

22 Mar '12

Hello, I'm working on a local instance of Galaxy-dist and I just updated it. Right after the update I tried to run the cleanup scripts. Unfortunately what I'm getting is the following error: # sh delete_datasets.sh -d 0 -r Traceback (most recent call last): File "./scripts/cleanup_datasets/cleanup_datasets.py", line 524, in <module> if __name__ == "__main__": main() File "./scripts/cleanup_datasets/cleanup_datasets.py", line 101, in main app = CleanupDatasetsApplication( config ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 512, in __init__ self.model = galaxy.model.mapping.init( config.file_path, config.database_connection, engine_options={}, create_tables=False, object_store=self.object_store ) File "/illumina/Galaxy/galaxy-dist/lib/galaxy/model/mapping.py", line 1818, in init load_egg_for_url( url ) File "/illumina/Galaxy/galaxy-dist/lib/galaxy/model/mapping.py", line 1798, in load_egg_for_url dialect = guess_dialect_for_url( url ) File "/illumina/Galaxy/galaxy-dist/lib/galaxy/model/mapping.py", line 1794, in guess_dialect_for_url return (url.split(':', 1))[0] AttributeError: 'bool' object has no attribute 'split' The same error occurs if I try to run other cleanup scripts, such as the delete_datasets or purge_datasets. Any idea to solve this issue? Best regards, Rocco

1 0

compare tabular datasets not working
by Mikel Egaña Aranguren 22 Mar '12

22 Mar '12

Hi; I'm getting an error when comparing two datasets (two tabular files) with "Join, substract and group" (Error pasted bellow). If I execute joinWrapper.py manually I don't get the error, and also it doesn't happen if I use the Galaxy version changeset 5743:720455407d1c. Any clues? Any help would be much appreciated, I need this for a paper I'm writting and deadline is too close :-) Regards Traceback (most recent call last): File "/home/megana/galaxy-dist-Erick/lib/galaxy/jobs/runners/local.py", line 125, in run_job job_wrapper.finish( stdout, stderr ) File "/home/megana/galaxy-dist-Erick/lib/galaxy/jobs/__init__.py", line 607, in finish self.sa_session.flush() File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 773, in execute_save_steps self.execute_cyclical_dependencies(trans, task, False) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 789, in execute_cyclical_dependencies self.execute(trans, [t], isdelete) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/megana/galaxy-dist-Erick/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE history_dataset_association SET update_time=?, info=?, blurb=?, peek=? WHERE history_dataset_association.id = ?' ['2012-03-22 16:23:45.441156', 'join (GNU coreutils) 7.4\nCopyright \xc2\xa9 2009 Free Software Foundation, Inc.\nLicencia GPLv3+: GNU GPL versi\xc3\xb3n 3 o superior .\nEsto es software libre: es libre de cambiarlo y redistribuirlo.\nNO HAY NINGUNA GARANT\xc3\x8dA, en la me', 'empty', 'no peek', 330] -- Mikel Egaña Aranguren, PhD http://mikeleganaaranguren.com Marie Curie post-doc at Ontology Engineering Group, UPM http://www.oeg-upm.net/

1 0

Fixed Thread number in certain tools
by Daniel Sobral 22 Mar '12

22 Mar '12

Hello, For certain tools, like bwa or blast, there is a non-configurable number of threads to use inside the tool xml. This is inconvenient for certain cluster configurations, where what we want is division of a job into multiple jobs (I guess this is what is intended by the <parallel> tag), and not creation of threads that may wreak havoc in a cluster. Is there any reason why bwa and blast have --num-threads=8 pre-configured in the xml file? Thanks, Daniel

2 1

22 Mar '12

Hi, For our fresh new local instance I created a user. But whenever I try to login it looks like I end up as anonymous user again. And if I click "Workflows" I'm told that I need to login first. The only thing that looks like an error message is this one: "galaxy.web.framework DEBUG 2012-03-22 16:38:42,570 Error: this request returned None from get_history(): http://bbc.mdc-berlin.de/galaxy/workflow" I'm running behind an Apache, could that be the reason? My Apache config: RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /opt/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /opt/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /opt/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /opt/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /opt/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://database1:8080$1 [P] universe_wsgi.conf ... [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy ... [app:main] ... # Allow unregistered users to create new accounts (otherwise, they will have to # be created by an admin). #allow_user_creation = True # Allow administrators to delete accounts. #allow_user_deletion = False # Allow administrators to log in as other users (useful for debugging) #allow_user_impersonation = False regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

2 1

Importing data from password protected UCSC Genome Browser
by Federico De Masi 22 Mar '12

22 Mar '12

Dear colleagues, I have been struggling with this issue for quite a while now, and I don't seem to find a solution (neither in situ nor in webo). here it is: Because of the confidentiality of our data, I have setup both a Galaxy server and a UCSC Genome Browser locally on my server. Both are run through Apache, using the same .htpasswd authentication method and file (ie: same usernmae/password combo for each). Everything works great except that I do not seem to be able to send/receive data from Galaxy to UCSC and vice versa. When I try to "get data" from UCSC main (my local instance), I get the followin error: "Tool execution generated the following error message: The remote data source application may be off line, please try again later. Error: EOF when reading a line The tool produced the following additional output: Enter username for SG at 127.0.0.1:" where SG is the AuthName in vhost.conf and 127.0.0.1 is UCSC's IP. Whenever I try to to send data to UCSC ("Display at UCSC main" in the history boxes), I get this guy: "Expected 200 http://127.0.0.3/display_application/48916fac0de9a85d/ucsc_bigwig/main/b9ba…: 401 Authorization Required" Where 127.0.0.3 is galaxy's IP. I have: - hardcoded username:passowrd on UCSC's "send to galaxy" routine but to no avail - hardcode username:passowrd on Galaxy's definiton of UCSC's "main" URL, but to no avail - I have removed UCSC authentication, and everything works fine - I have kept UCSC authentication and let Galaxy "open", doesn't work - I remove authentication to both servers, and it all works cool. Somehow, Galaxy doesn't like to talk to a protected UCSC browser. I'll be happy to post my vhost.conf files if needed. Can anyone help, please? Thanks a lot, Fred -- Federico De Masi, PhD, Assistant Professor The Technical University of Denmark - DTU Center for Biological Sequence Analysis - CBS Kemitorvet 208/002 DK-2800 KGS. LYNGBY, DENMARK Telephone: (+45) 45 25 24 21 Fax: (+45) 45 93 15 85 http://rg.cbs.dtu.dk

3 2

← Newer
1
...
7
8
9
10
11
12
13
...
21
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev March 2012