How to manage stderr output from R?
by Carlos Borroto
Hi all,
I'm working on integrating cummeRbund[1] into Galaxy. This is a very
helpful tool for analysis and visualization of cuffdiff output.
My problem is any text output from running R code, puts the galaxy
output in error state. Even if everything is working correctly and the
outputs are generated as intended. I tried this code from
'tools/plotting/xy_plot.xml':
## Setup R error handling to go to stderr
options( show.error.messages=F,
error = function () { cat( geterrmessage(), file=stderr() ); q(
"no", 1, F ) } )
But it didn't help. Even for loading the library I had to go with:
suppressPackageStartupMessages(library("cummeRbund"))
Nothing else would work. Now the tool is outputting some messages
while running R functions, which is putting the output in galaxy in
error state. I'm thinking about asking the upstream developer to
provide a way to run the functions in the library in 'quite' mode. Is
there any other option?
Right now I'm using the path of creating a R script in the XML file
and using r_wrapper.sh to run it. I tried first with rpy, as it would
give me more freedom, but cummeRbund use ggplot2 which seems to be
supported only on rpy2.
[1]http://compbio.mit.edu/cummeRbund/
Thanks in advance,
Carlos
10 years, 3 months
Re: [galaxy-dev] shed_tool_conf.xml not updated
by Greg Von Kuster
Anne,
Thanks for all of your patience and help with this. You uncovered a problem that is very nice to have corrected. The fix is included in change set 6634:ebf72cda61e7 on our central repository. This change set will allow you to have section tags in your tool_conf.xml file that do not include version attributes. Selecting these sections to contain tools installed with a tool shed repository will now be functionally correct.
Thanks again!
Greg
On Jan 30, 2012, at 12:08 PM, Anne Pajon wrote:
> Brilliant! It is working - no more server error and I now have a shed_tool_conf.xml updated:
>
> Let me know when you commit your next change set, so I will be able to test.
> Thanks a lot for your help.
> Anne.
>
Greg Von Kuster
Galaxy Development Team
greg(a)bx.psu.edu
10 years, 3 months
Problem running purge_datasets.sh cleanup scripts
by Leandro Hermida
Hello,
In the latest galaxy-dist version when running the system cleanup commands
in the documented order I get an error when running the purge_datasets.sh
script:
sh delete_userless_histories.sh
sh purge_histories.sh
sh purge_libraries.sh
sh purge_folders.sh
sh delete_datasets.sh
sh purge_datasets.sh
Traceback (most recent call last):
File "./scripts/cleanup_datasets/cleanup_datasets.py", line 518, in
<module>
if __name__ == "__main__": main()
File "./scripts/cleanup_datasets/cleanup_datasets.py", line 116, in main
purge_datasets( app, cutoff_time, options.remove_from_disk, info_only =
options.info_only, force_retry = options.force_retry )
File "./scripts/cleanup_datasets/cleanup_datasets.py", line 353, in
purge_datasets
_purge_dataset( app, dataset, remove_from_disk, info_only = info_only )
File "./scripts/cleanup_datasets/cleanup_datasets.py", line 478, in
_purge_dataset
print "Error attempting to purge data file: ", dataset.file_name, "
error: ", str( exc )
File
"/gpfshpc/software/pmrdcdb/galaxy/galaxy_dist_dev/lib/galaxy/model/__init__.py",
line 651, in get_file_name
assert self.object_store is not None, "Object Store has not been
initialized for dataset %s" % self.id
AssertionError: Object Store has not been initialized for dataset 58
What could be going wrong?
regards,
Leandro
10 years, 3 months
samtools in local instance of galaxy
by Praveen Baskaran
i have installed local instance of galaxy and have a small problem with samtools. when i try to convert sam file to bam, the result pane shows that the job is running (for several hours). but when check the galaxy log file, it shows that the job is finished normally. i can also find the result (bam) file in galaxy/database/files/000/ folder. can anyone help to resolve this issue.
Thanks
Praveen
10 years, 3 months
Some eggs are out of date...Fetch failed (Mac OS X 10.6)
by Lucilla Luzi
Hi,
I'm pretty new to Galaxy, at least to this side.
I'm sorry to bother you with my issue, that, I read, was quite
extensively discussed previously, but even if I think to have followed
correctly the instruction and further suggestions, I'm still not able to
overcome the problem.
I'm trying to install a local instance of Galaxy on my MAC (OS X10.6.8)
My Python version is 2.6.1 and I just installed Mercurial 2.0.2 for
MacOS X 10.6.
FYI
$ python --version
Python 2.6.1
$ echo $PATH
/Users/myName/System/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
When I try to run the run.sh I get the error attached below.
Can you envisage a possible reason why I'm getting this error?
Sorry for the naive admission, but from the error message
I'm not even capable to understand which are the "missing eggs"...
If I'd have ignored a previous post with the solution, I apologize in
advance.
Thank you a lot for your attention.
Lu
------------------
$ sh run.sh
Some eggs are out of date, attempting to fetch...
Traceback (most recent call last):
File "./scripts/fetch_eggs.py", line 30, in <module>
c.resolve() # Only fetch eggs required by the config
File
"/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py",
line 345, in resolve
egg.resolve()
File
"/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py",
line 195, in resolve
return self.version_conflict( e.args[0], e.args[1] )
File
"/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py",
line 226, in version_conflict
r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ),
env, egg.fetch )
File
"/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py",
line 518, in resolve
raise DistributionNotFound(req) # XXX put more info here
pkg_resources.DistributionNotFound: numpy==1.6.0
Fetch failed.
10 years, 3 months
some basic cluster dev questions
by Jaime Frey
Hello, everyone.
I'm working on a Condor job-runner module for Galaxy that can provide better integration than the existing drmaa module. I'm sure I'll have many questions as I work on this module, but I have just a few to start with:
* What versions of Python does Galaxy support (i.e. which newer features of Python do I need to avoid)?
* What is the status of the 'staged' cluster deployment? The wiki says it's not in active use or development. But our Condor cluster doesn't have a shared filesystem, so we need to stage data files to/from the execute nodes.
* The wiki also says that even with the staged cluster deployment, a shared directory (galaxy_dist/database/tmp by default) is required for some tools. These tools write temporary files on the Galaxy server and read them on the execute node. Does anyone know which tools those are? Are these files ever written after the job is handed to the batch system? Do these tools write files into galaxy_dist/database/tmp on the execute ndoes and read the files on the Galaxy server?
Thanks and regards,
Jaime Frey
UW-Madison Condor Team
10 years, 3 months
postgres user change problem
by Holger Klein
Dear all,
due to some work on the user management of our servers we had to rename
the user galaxy runs as. Up to now we used ident as postgres
authentication method, meaning that here postgres expects unix username
"galaxy" to have permissions of "galaxy" postgres user.
The entry in universe_wsgi.ini is simply:
database_connection = postgres:///galaxy?host=/var/run/postgresql
After the renaming, the new user "galaxynew" didn't get access at all at
first. Now I tried two things:
- adding a user "galaxynew" to postgres with permissions for database galaxy
and
- dumping the contents of database "galaxy" into a file and re-reading
this into database "galaxynew", which is owned by the user "galaxynew"
and of course changing the config file line to
database_connection = postgres:///galaxynew?host=/var/run/postgresql
In both cases I end up with an error during startup of galaxy:
[...]
WARNING 2012-01-20 14:46:55,556 Error closing cursor: current
transaction is aborted, commands ignored until end of transaction block
[...]
ProgrammingError: (ProgrammingError) permission denied for relation
migrate_version
'SELECT migrate_version.repository_id, migrate_version.repository_path,
migrate_version.version \nFROM migrate_version \nWHERE
migrate_version.repository_id = %(repository_id_1)s' {'repository_id_1':
'Galaxy'}
(complete traceback below).
Does anyone have a hint on how I could fix this?
Cheers,
Holger
sqlalchemy.pool.QueuePool.0x...8d10 WARNING 2012-01-20 14:46:55,556
Error closing cursor: current transaction is aborted, commands ignored
until end of transaction block
Traceback (most recent call last):
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/web/buildapp.py",
line 82, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/app.py", line
39, in __init__
create_or_verify_database( db_url, kwargs.get( 'global_conf', {}
).get( '__file__', None ), self.config.database_engine_options )
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/model/migrate/check.py",
line 99, in create_or_verify_database
db_schema = schema.ControlledSchema( engine, migrate_repository )
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py",
line 24, in __init__
self._load()
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py",
line 41, in _load
self.table.c.repository_id == str(self.repository.id)))
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 1202, in execute
return connection.execute(statement, *multiparams, **params)
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 824, in execute
return Connection.executors[c](self, object, multiparams, params)
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 874, in _execute_clauseelement
return self.__execute_context(context)
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 896, in __execute_context
self._cursor_execute(context.cursor, context.statement,
context.parameters[0], context=context)
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 950, in _cursor_execute
self._handle_dbapi_exception(e, statement, parameters, cursor, context)
File
"/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py",
line 931, in _handle_dbapi_exception
raise exc.DBAPIError.instance(statement, parameters, e,
connection_invalidated=is_disconnect)
ProgrammingError: (ProgrammingError) permission denied for relation
migrate_version
'SELECT migrate_version.repository_id, migrate_version.repository_path,
migrate_version.version \nFROM migrate_version \nWHERE
migrate_version.repository_id = %(repository_id_1)s' {'repository_id_1':
'Galaxy'}
--
Dr. Holger Klein
Core Facility Bioinformatics
Institute of Molecular Biology gGmbH (IMB)
http://www.imb-mainz.de/
Tel: +49(6131) 39 21511
10 years, 3 months
server error when installing tool from tool shed repository into local instance
by Anne Pajon
Hello,
I have the version 6528:63bc46cc73b7 of galaxy running locally on our server installed from https://bitbucket.org/galaxy/galaxy-dist/.
I've tried to bring tabular2html tool from Galaxy main tool shed repository following the wiki information (http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_to...) without success.
Here is what I've done.
(1) I've edited my universe_wsgi.ini
# -- Files and directories
# Tool config files, defines what tools are available in Galaxy.
# Tools can be locally developed or installed from Galaxy tool sheds.
tool_config_file = tool_conf.xml,shed_tool_conf.xml
# Default path to the directory containing the tools defined in tool_conf.xml.
# Other tool config files must include the tool_path as an attribute in the <toolbox> tag.
tool_path = tools
(2) I've created the directory shed_tools/ above the galaxy-dist/ as recommended
(3) I've restarted the server
(4) I've logged in as "admin" user. I've clicked the "Admin" link in the top Galaxy tool panel to display the Galaxy Administration interface and then "Search and browse tool sheds" under "Tool sheds" from the left blue section to display the Accessible Galaxy tool sheds page, click "Galaxy main tool shed" to display the valid repositories and choose tabular2html and click on Install to local Galaxy, select the panel section and click Install.
I get a page with "Server Error".
Strangely, the tool appears in the list of tools and works in my Galaxy instance. The tool files have been installed on disk at the directory created in (2) but the shed_tool_conf.xml has not been updated. After restarting my Galaxy server, the tool is not visible anymore.
Here is what appears in the log related to the installation of the tool:
galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Installing repository 'tabular2html'
galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Cloning http://toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html
galaxy.util.shed_util DEBUG 2012-01-25 14:20:55,486 Updating cloned repository to revision "a5814dd5a11a"
galaxy.tools DEBUG 2012-01-25 14:20:55,652 Reloading section: My Tools
galaxy.tools DEBUG 2012-01-25 14:20:55,673 Loaded tool id: toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html/tabular2HTML/1.0.0, version: 1.0
.0.
143.65.172.178 - - [25/Jan/2012:14:20:54 +0100] "POST /galaxy/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.ps
u.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c6573
2066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f74616275
6c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True HTTP/1.1" 500 - "http://uk-cri-lbio08/galaxy/admin_toolshed/insta
ll_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03:7b22746162756c61723268746d
6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e
7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True" "Mozilla/5.0 (
Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13"
Error - <type 'exceptions.TypeError'>: cannot serialize None (type NoneType)
URL: http://uk-cri-lbio08/galaxy/admin_toolshed/install_repository?tool_shed_u...
9982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c6172206
46174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c202261353
8313464643561313161225d7d&includes_tools=True
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/exceptions/errormiddleware.py', line 143 in __call__
app_iter = self.application(environ, start_response)
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__
return self.application(environ, start_response)
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__
return self.application(environ, start_response)
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__
body = method( trans, **kwargs )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 173 in decorator
return func( self, trans, *args, **kwargs )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', line 275 in install_repository
new_install=True )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 644 in load_repository_contents
add_to_config=True )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 699 in load_altered_part_of_tool_panel
add_shed_tool_conf_entry( app, shed_tool_conf, elem_entry )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 41 in add_shed_tool_conf_entry
new_shed_tool_conf.write( util.xml_to_string( tool_panel_entry, pretty=True ) )
File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/__init__.py', line 113 in xml_to_string
return ElementTree.tostring( pretty_print_xml( elem ) )
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 1006 in tostring
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 660 in write
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 695 in _write
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 827 in _escape_a
ttrib
File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 774 in _raise_se
rialization_error
TypeError: cannot serialize None (type NoneType)
Any help would be very welcome. Thanks a lot in advance.
Kind regards,
Anne.
--
Anne Pajon, Ph.D.
Cancer Research UK - Cambridge Research Institute
Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE
anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353
NOTICE AND DISCLAIMER
This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose.
We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you.
Cancer Research UK
Registered in England and Wales
Company Registered Number: 4325234.
Registered Charity Number: 1089464 and Scotland SC041666
Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.
10 years, 3 months
Error downloading output file from Galaxy
by Cory Spencer
Hello all -
I'm working on a tool for Galaxy that generates an output file. I am able to view the file without issue in Galaxy itself, but when I click on the "Download" icon, the History sidebar produces the following error message:
"Unable to remove temporary library download archive and directory"
Checking the log files, I see the following entry:
galaxy.web.controllers.dataset ERROR 2012-01-17 23:14:05,599 Unable to remove temporary library download archive and directory
Traceback (most recent call last):
File "/n/galaxy/www/galaxy/lib/galaxy/web/controllers/dataset.py", line 284, in archive_composite_dataset
os.rmdir( tmpd )
OSError: [Errno 39] Directory not empty: './database/tmp/tmprofQCp'
Listing the contents on the referenced directory shows it to be empty:
$ ls -la database/tmp/tmprofQCp
total 16
drwx------ 2 galaxy galaxy_lab 1024 2012-01-17 23:14 .
drwxrwx--- 8 galaxy galaxy_lab 1024 2012-01-17 23:14 ..
Does anyone have any ideas as to what might be happening?
Thank you!
Cory
10 years, 3 months
How to use scripts/api/update.py?
by Carlos Borroto
Hi,
I'm trying to use the update.py script and is not clear for me what to
use for 'url' in the usage line:
usage: create.py key url [key=value ...].
Which I'm guessing is just a carry over from create.py.
I tried passing the direct link to the dataset:
$ ./update.py <my_key>
http://localhost:8080/api/histories/0397e7c5778be5ee/contents/45d854c903d...
name=foobar
HTTP Error 404: Not Found
404 Not Found
The resource could not be found.
No action for /api/histories/0397e7c5778be5ee/contents/45d854c903d27dcf
Just to show you this dataset does exists:
$ ./display.py <my_key>
http://localhost:8080/api/histories/0397e7c5778be5ee/contents/45d854c903d...
Member Information
------------------
misc_blurb: 4.3 Gb
name: BMD01.GAIIx.1.fastq
data_type: fastqillumina
deleted: False
state: ok
download_url: /datasets/45d854c903d27dcf/display?to_ext=fastqillumina
visible: True
genome_build: ?
model_class: HistoryDatasetAssociation
metadata_sequences: None
file_size: 4644313368
metadata_data_lines: None
id: 45d854c903d27dcf
misc_info: uploaded fastq file
metadata_dbkey: ?
Thanks,
Carlos
10 years, 3 months
