- galaxy-dev - lists.galaxyproject.org

Moving Data to New Instance
by Robert Chase 17 May '12

17 May '12

Hello All, We have a large amount of data on our production server that we would like to make available on our new test server. I tried to move data from our production server to our test server by copying the meta-data files and dataset files from ~/database/files to the new test server but it didn't work. Is there a way to move the datasets en mass? -Rob

2 2

Preview of large files seems not to finish
by Anthonius deBoer 16 May '12

16 May '12

Hi, I noticed in the latest installment of Galaxy-Central that there is a pretty view option for VCF files, in that it tries to show the data in nice columns. But for my VCF files it does not seem to finish loading the first MB of the file..It also no longer shows the SHOW ALL at the top, so now I have to load the file out of my browser, which is a pain... Is this just a bug or a feature? Thanks, Thon

1 0

Idea for user-based dataset subdirectories
by Josh Nielsen 16 May '12

16 May '12

Hello, Please forgive the length of this proposition as I try to explain my reasoning behind this. Let me say first of all that I understand that Galaxy is not meant to be everything to everyone and that requests for features may not suit everyone who uses Galaxy. That being said I have an idea or request that I think would be convenient for dealing with user's datasets from a file-system perspective. Galaxy has the obvious benefit and advantage (compared to manual job-submission for tools on a cluster) of providing an interface for using all the analysis tools, and the history of the operations done on your data, all in one place. However I have found that putting all the output & datasets in one directory (the files/000/ directory) on the file-system causes a problem for the users if they specifically want to interact with it *on the file-system*, and not just through the Web interface - for whatever complicated or diverse reasons. Since Galaxy runs on a cluster of its own in our environment, and we do not allow users to remote connect into it to submit manual jobs (and individually output it to their separate home directories) like we do our main cluster, it is essentially a black box beyond the GUI interface of Galaxy. That is essentially what we want except for how they can interact with the output files. The issue is that our users would like an easy means of copying their files off of the Galaxy cluster to other servers from a command line (possibly even automated by scripts). Even if we allow an FTP share of the output directory for users to do that, the common [galaxy-dist]/database/files/000/ directory clumps all of the files for all users together in one directory and uses a sequential file-naming scheme (dataset_N++) that is not easy to discriminate between as to who the owner is for each file. Is there a way that the dataset output directory locations could be designed (or set optionally?) like the FTP upload feature's expected directory structure: where the files are dropped into the corresponding subdirectory of the user who produced it? For example having under database/files/ subdirectories named according to the user's Galaxy account id (like [galaxy-dist]/database/files/jsmith, [galaxy-dist]/database/files/sparker, etc.). If they could be segregated by user it would be much easier to keep track of what datasets belong to whom on the file-system. Then I could possibly set up a read-only FTP share to the files/ directory on the cluster, from which the users could directly copy the files in their personal subdirectory to other systems, and perhaps batch download them, rather than having to rely solely on the Web interface. I understand that the way Galaxy is currently designed is that the files are just generically named (the "behind-the-scenes" handling of data is a black box) and it is the database that keeps track of which files belong to whom, and which has the metadata for more meaningful dataset/job names, etc. But a file-system hierarchy alternative would also be welcome in a heavily command-line oriented computational environment too. Would setting up a more user-representative output directory hierarchy on the file-system like that be possible? Best Regards, Josh Nielsen

3 5

⇝ IOError: [Errno 13] Permission denied: u'/home/koala2/galaxy-central/database/compiled_templates/base_panels.mako.py'
by Luciano Cosme 16 May '12

16 May '12

Hi Everyone, I was just using my local galaxy instance. I closed it up (from terminal), open IGV to look at some genes, then I restarted it from the terminal and when I tried to access it I got the error message below. I just don't know what happened in these few minutes. Any suggestion how to fix it? Thank you. Luciano URL: http://127.0.0.1:8080/ File '/home/koala2/galaxy-central/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/home/koala2/galaxy-central/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/home/koala2/galaxy-central/lib/galaxy/web/controllers/root.py', line 25 in index m_c=m_c, m_a=m_a ) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/__init__.py', line 733 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/home/koala2/galaxy-central/lib/galaxy/web/framework/__init__.py', line 744 in fill_template_mako return template.render( **data ) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 691 in _render_context (inherit, lclcontext) = _populate_self_namespace(context, tmpl) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 637 in _populate_self_namespace ret = template.module._mako_inherit(template, context) File '/home/koala2/galaxy-central/database/compiled_templates/root/ index.mako.py', line 25 in _mako_inherit return runtime._inherit_from(context, u'/webapps/galaxy/base_panels.mako', _template_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 609 in _inherit_from ret = callable_(template, lclcontext) File '/home/koala2/galaxy-central/database/compiled_templates/webapps/galaxy/ base_panels.mako.py', line 25 in _mako_inherit return runtime._inherit_from(context, u'/base_panels.mako', _template_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 596 in _inherit_from template = _lookup_template(context, uri, calling_uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 626 in _lookup_template return lookup.get_template(uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/lookup.py', line 218 in get_template return self._load(srcfile, uri) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/lookup.py', line 283 in _load **self.template_args) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 218 in __init__ module = self._compile_from_file(path, filename) File '/home/koala2/galaxy-central/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 245 in _compile_from_file module = imp.load_source(self.module_id, path, open(path, 'rb')) IOError: [Errno 13] Permission denied: u'/home/koala2/galaxy-central/database/compiled_templates/ base_panels.mako.py'

3 3

Latest Galaxy build gives me error on WSF extension not found
by Anthonius deBoer 16 May '12

16 May '12

Hi, In the latest Galaxy Central rev I get this error in each tool I run... Any ideas what is going on? Thanks, Thon WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'wsf': 'module' object has no attribute 'wsf'

2 1

Error loading data
by SHAUN WEBB 16 May '12

16 May '12

Hi, I have just upgrade my test version of Galaxy to the latest dist release and I am getting an error trying to load data either as an upload, a pasted entry or from UCSC. I have only altered conf files, any pointers to what could be causing this would be appreciated. The error is below: Traceback (most recent call last): File "/storage/home/galaxy/test/pre-dev/galaxy-dist/tools/data_source/upload.py", line 403, in <module> __main__() File "/storage/home/galaxy/test/pre-dev/galaxy-dist/tools/data_source/upload.py", line 378, in __main__ registry.load_datatypes( root_dir=sys.argv[1], config=sys.argv[2] ) AttributeError: 'Registry' object has no attribute 'load_datatypes' Thanks Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

managing external dependency eggs
by Shantanu Pavgi 16 May '12

16 May '12

Hi, At the risk of exposing of my Python ignorance, here is a doubt regarding galaxy deployment. The galaxy application manages external dependency eggs through it's custom ini specification file and eggs installation scripts. I was wondering if this could be managed through pip using a requirements.txt file. Are there any limitation of the later (pip) approach? Appreciate your comments. -- Thanks, Shantanu

2 1

Serious problems with the new Galaxy update (12 may) / bugs??
by Nikolai Vazov 16 May '12

16 May '12

Hi, I updated Galaxy following the instructions in http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_05_11 Everything went well : the update, the shed_tool (left it unchanged), database migration, reboot as --daemon Yet, when I write the url in the address line, I get : the three empty panels and the text in the middle one: ======= Not Found The resource could not be found. No route for /user/login ======= Already at boot I had an error message of type cannot import validate_username from galaxy.security.validate_user_input where actually this function validate_username is missing. I replaced the line 13 in lib/galaxy/web/controllers/user.py from from galaxy.security.validate_user_input import validate_email, validate_username, validate_password to from galaxy.security.validate_user_input import validate_email, validate_publicname, validate_password and Galaxy started. But then I only got empty panels and the error above Do you have any hints? I am sending you the paster.log at boot and a screenshot Thank you in advance Nikolay -- Nikolay Vazov, PhD Research Computing Centre - http://hpc.uio.no USIT, University of Oslo

1 0

Value error when running run.sh
by julie dubois 16 May '12

16 May '12

Hi, I 've installed Galaxy on Ubuntu 12.04 with apache2 and MySQL I've configured apache 2 with this lines on file /etc/apache2/sites-available/default between <VirtualHost> and </VirtualHost> : <Proxy "http://localhost:8080"> Order Deny,Allow Allow from all </Proxy> RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/chevalier/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/chevalier/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/chevalier/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/chevalier/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/chevalier/galaxy-dist/static/robots.txt [L] RewriteRule ^/galaxy/(.*) http://localhost:8080$1 [P] I've configured my MYSQL with a user who have all privilegies on a database and in the universe_wsgi.ini the connection database appears like this : database_connection = mysql://cistrome:cistrome@localhost :/var/run/mysqld/mysqld.sock/cistromeap After running sh run.sh in my galaxy directory, I obtain this error message : .... File "/home/chevalier/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/url.py", line 49, in __init__ self.port = int(port) ValueError: invalid literal for int() with base 10: ' ' This is the case too when I run this with root privilegies. Is it an error in my configuration of apache 2 or in MySQL ? Or is it an error in my universe_wgsi.ini ? Thank you Julie

2 2

Follow Up
by Daniel Patrick Sullivan 16 May '12

16 May '12

Hi, Galaxy Developers, I sent an email to this list earlier this week regarding an issue I was having with the 'schedule jobs as user' functionality of Galaxy. I am taking a closer look at the documentation on this configuration option, and based on my reading, this feature is only supported with DRMAA (and not PBS). Based on my reading of the cluster URL in the galaxy documentation (http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster) it is my understanding that galaxy must use PBS to interact with TORQUE; the documentation specifically states "although for legacy reasons, pbs_python is used for TORQUE". Does this mean that I am not able to use the schedule jobs as user functionality of galaxy with TORQUE? Is DRMAA not supported with TORQUE at all for Galaxy? Dan Sullivan

2 1