February 2015 - galaxy-dev - lists.galaxyproject.org

problem with latest toolshed devteam cuffmerge wrapper multi-select
by Nikhil Joshi 26 Feb '15

26 Feb '15

Hi all, We run our own custom galaxy instance in the Amazon Cloud and I am updating our instance with the latest toolshed files. For cuffmerge, in past revisions, we would just have a button to add another GTF input. However, this latest one (10:b6e3849293b1 <https://toolshed.g2.bx.psu.edu/repository/view_changelog?id=84bbebf45888f374>) that has been replaced with a multi-select. We also have a few built-in genomes that we use with cuffmerge. When I choose the first GTF file (using locally cached sequence data) it will populate the reference genome correctly with our built-in genomes. However, once I choose more than one GTF file, I get one of two errors: No reference genome is available for the build associated with the selected input dataset OR An invalid option was selected, please verify The built-in genomes are definitely there and available for cuffmerge, but for some reason choosing more than one GTF file in the multi-select causes an error. Is this a bug or am I doing something wrong? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

3 4

custom java tools.xml : problem with classpath
by Pierre Lindenbaum 26 Feb '15

26 Feb '15

Hi Galaxy, I'm trying to put one of my java tool in galaxy. I've copied the required files in my tool directory under /tools/jvarkit (I can see it galaxy) $ ls commons-jexl-2.1.1.jar commons-logging-1.1.1.jar htsjdk-1.129.jar snappy-java-1.0.3-rc3.jar vcffilterjs.jar vcffilterjs.xml Here is the content of vcffilterjs.xml <?xml version="1.0"?> <tool hidden="false" id="com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS" version="1.0.0" name="vcffilterjs"> <requirements> <requirement type="binary">java</requirement> </requirements> <command><![CDATA[java -cp $INSTALL_DIR/commons-jexl-2.1.1.jar:$INSTALL_DIR/commons-logging-1.1.1.jar:$INSTALL_DIR/htsjdk-1.129.jar:$INSTALL_DIR/snappy-java-1.0.3-rc3.jar:$INSTALL_DIR/vcffilterjs.jar com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS -e '${expression}' -o ${output} ${input} 2> /tmp/error ]]></command> <inputs> <param format="vcf" name="input" type="data" label="VCF input"/> <param name="expression" type="text" label="javascript expression"/> </inputs> <outputs> <data format="vcf" name="output"/> </outputs> <stdio> <exit_code range="1:"/> <exit_code range=":-1"/> </stdio> <help>(...)</help> </tool> When I run this tool, the log says: NotFound: cannot find 'INSTALL_DIR' If I replace $INSTALL_DIR with \$INSTALL_DIR , the error on stderr is : Error: Could not find or load main class com.github.lindenb.jvarkit.tools.vcffilterjs.VCFFilterJS How can I make this work ? Bonus question: is there a way to catch the error file to display it in galaxy when a process fails ? Thanks; Pierre

3 12

Importing file via FTP can silently fail, [Errno ftp error] 550
by Peter Cock 26 Feb '15

26 Feb '15

Hi all, Using either the legacy upload tool, or the new upload widget, the following FTP URL *appears* to load into Galaxy correctly giving a green history entry. However, on closer inspection it is an empty file with this peep text: Unable to fetch ftp://ftp.sanger.ac.uk/pub/pathogens/Bursaphelenchus/xylophilus/Assembly-v1.2/BurXv1.2.supercontigs.fa.gz [Errno ftp error] 550 pathogens: No such file or directory Clearly this should have been treated as a failure (red history entry). Should I file this as a bug on Trello? It seems there is a problem with the Sanger folder permissions which trips up Galaxy and curl (but does not bother Firefox and wget): $ curl -O ftp://ftp.sanger.ac.uk/pub/pathogens/Bursaphelenchus/xylophilus/Assembly-v1.2/BurXv1.2.supercontigs.fa.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (9) Server denied you to change to the given directory Using curl -v for verbose mode it tries to walk the path (as per the https://www.ietf.org/rfc/rfc1738.txt standard) and fails with the command "CWD pathogens" matching the Galaxy error. The alternative curl FTP methods work: $ curl -O ftp://ftp.sanger.ac.uk/pub/pathogens/Bursaphelenchus/xylophilus/Assembly-v1.2/BurXv1.2.supercontigs.fa.gz --ftp-method nocwd % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 21.8M 100 21.8M 0 0 7656k 0 0:00:02 0:00:02 --:--:-- 7654k and: $ curl -O ftp://ftp.sanger.ac.uk/pub/pathogens/Bursaphelenchus/xylophilus/Assembly-v1.2/BurXv1.2.supercontigs.fa.gz --ftp-method singlecwd % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 21.8M 100 21.8M 0 0 7767k 0 0:00:02 0:00:02 --:--:-- 7766k You can also trick curl by replacing (some of the) slashes with the escaped version %2f which has the result of combining the CWD commands, e.g. $ curl -O ftp://ftp.sanger.ac.uk/pub%2fpathogens%2fBursaphelenchus%2fxylophilus%2fAssembly-v1.2/BurXv1.2.supercontigs.fa.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 21.8M 100 21.8M 0 0 6827k 0 0:00:03 0:00:03 --:--:-- 6829k This encoding trick does not seem to work within Galaxy. I plan to report this to Sanger tomorrow. Should they fix this quickly additional test cases might be useful (does Galaxy have an FTP server which might be used for testing data importing like this?). Is this a common issue? Should Galaxy automatically use an FTP CWD workaround in this situation? Regards, Peter

3 6

Subject: Serving Galaxy at a sub directory - symptoms without "filter-with = proxy-prefix"
by Dooley, Damion 25 Feb '15

25 Feb '15

This info is definitely useful! I ran into this exact same problem last week, trying to set up a new instance of galaxy from the mercurial repo. I was re-using a previous install's apache config; saw the same web page results. Had to go to the galaxy web docs to track down that filter-with setting. Damion

1 0

problems splitting
by Roberto Alonso CIPF 25 Feb '15

25 Feb '15

Hello, I am trying to map a a fastqsacer file and map it with bwa, my bwa tool config file is this: <tool id="bwa_mio" name="map with bwa"> <description>map with bwa</description> <parallelism method="basic" split_size="2" split_mode="number_of_parts"></parallelism> <command> bwa mem /home/ralonso/BiB/Galaxy/data/Cclementina_v1.0_scaffolds.fa $input > $output 2>xx</command> <inputs> <param format="fastqsanger" name="input" type="data" label="fastq"/> </inputs> <outputs> <data format="sam" name="output" /> </outputs> <help> bwa </help> </tool> And when I see the stderr I see this error: type object 'Sequence' has no attribute 'get_split_commands_sequential' It seems that this command that I see in the log is not working galaxy.jobs.runners DEBUG 2015-02-11 16:33:48,738 (74) command is: /home/ralonso/galaxy-dist/extract_dataset_parts.sh /home/ralonso/galaxy-dist/database/job_working_directory/000/74/task_0; bwa mem /home/ralonso/BiB/Galaxy/data/Cclementina_v1.0_scaffolds.fa /home/ralonso/galaxy-dist/database/job_working_directory/000/74/task_0/dataset_8.dat > /home/ralonso/galaxy-dist/database/job_working_directory/000/74/task_0/dataset_75.dat When I go directly to the code, around line 559 of class galaxy.datatypes.sequence I can't find this function get_split_commands_sequential anywhere. Any idea? Thank you very much Regards -- Roberto Alonso Functional Genomics Unit Bioinformatics and Genomics Department Prince Felipe Research Center (CIPF) C./Eduardo Primo Yúfera (Científic), nº 3 (junto Oceanografico) 46012 Valencia, Spain Tel: +34 963289680 Ext. 1021 Fax: +34 963289574 E-Mail: ralonso(a)cipf.es

3 14

Tophat2 Problem
by Lapointe, David 25 Feb '15

25 Feb '15

The Tophat2 entry screen does not list the reference genomes, however Bowtie2 does list them ( the ones I have in the bowtie2_indices.loc). The bowtie2_indices.loc states that this file works for both Bowtie2 and Tophat2. -- David Lapointe Ph.D. Sr. Bioinformatics Specialist Research and Geospatial Technology Services Tufts Technology Services (TTS) 16 Dearborn Road Somerville MA 02144 Phone: 617-627-5319 Fax: 617-627-3667 http://it.tufts.edu

1 0

Serving Galaxy at a sub directory - symptoms without "filter-with = proxy-prefix"
by Peter Cock 25 Feb '15

25 Feb '15

Hi all, TL:DR abstract: Forgetting the "filter-with = proxy-prefix" gives very confusing results. Hopefully this email will help someone else who made the same configuration mistake. -- As part of trying out the new (still in beta) Galaxy via git on GitHub (rather than via hg on BitBucket), I am attempting to recreate my development instance under http://localhost/galaxy-dev via port 8081 using an Apache Proxy: https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Various things have changed, so rather than universe_wsgi.ini we now use config/galaxy.ini and I wanted to try that here, and so began by manually copying over settings I had changed. Key settings include the prefix and cookie_path, here is what I initially changed from the current defaults: $ diff config/galaxy.ini config/galaxy.ini.sample 32d31 < port = 8081 64,65c63 < #prefix = /galaxy < prefix = /galaxy-dev --- > prefix = /galaxy 93d90 < cookie_path = /galaxy-dev 678d674 < library_import_dir = True 693d688 < allow_library_path_paste = True 773d767 < admin_users = p.j.a.cock(a)googlemail.com 787d780 < allow_user_impersonation = True 793d785 < allow_user_dataset_purge = True 896,897d887 < expose_dataset_path = True < Here's my Apache configuration file, $ more /etc/httpd/conf.d/galaxy-dev.conf # mod_rewrite <IfModule mod_rewrite.c> RewriteEngine on RewriteRule ^/galaxy-dev$ /galaxy-dev/ [R] RewriteRule ^/galaxy-dev/static/style/(.*) /mnt/galaxy/repositories/galaxy/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy-dev/static/scripts/(.*) /mnt/galaxy/repositories/galaxy/static/scripts/packed/$1 [L] RewriteRule ^/galaxy-dev/static/(.*) /mnt/galaxy/repositories/galaxy/static/$1 [L] RewriteRule ^/galaxy-dev/favicon.ico /mnt/galaxy/repositories/galaxy/static/favicon.ico [L] RewriteRule ^/galaxy-dev/robots.txt /mnt/galaxy/repositories/galaxy/static/robots.txt [L] RewriteRule ^/galaxy-dev/(.*) http://localhost:8081/$1 [P] </IfModule> The Apache settings look fine, e.g. this URL works for the welcome page: http://localhost/galaxy-dev/static/welcome.html I've confirmed this by editing the default welcome page at /mnt/galaxy/repositories/galaxy/static/welcome.html The observed problem is that http://localhost/galaxy-dev/ gives: <quote> None Tools History Not Found The requested URL /root/welcome was not found on this server. ________________________________ Apache/2.2.15 (CentOS) Server at xxxxxx Port 80 </quote> >From the page source, the wrong URLs are being used, e.g.: <link href="/static/style/base.css?v=1424865275" media="screen" rel="stylesheet" type="text/css" /> <script src="/static/scripts/libs/jquery/jquery.js?v=1424865275" type="text/javascript"></script> and: <div style="position: absolute; width: 100%; height: 100%"> <iframe name="galaxy_main" id="galaxy_main" frameborder="0" style="position: absolute; width: 100%; height: 100%;" src="/root/welcome"></iframe> </div> For some reason the prefix specified was not being used for all the static resources (they should be "/galaxy-dev/static/..." instead of just "/static/...") and the iframe used "/root/welcome" rather then what I expected, "/galaxy-dev/static/welcome". I eventually realised on re-reading the wiki page that I was missing the crucial setting "filter-with = proxy-prefix" which fixed this: https://wiki.galaxyproject.org/Admin/Config/ApacheProxy I'm mainly sending this email in the hope it might show up in future Google searches and help someone else debugging the same mistake one day. Would it be helpful for Galaxy to notice if the configuration file contained a [filter:proxy-prefix] section without actually enabling it with "filter-with = proxy-prefix"? Regards, Peter

2 2

GCC2015 Abstract Submission is Open
by Dave Clements 25 Feb '15

25 Feb '15

Hello all, *We are pleased to announce that abstract submission for Oral and Poster Presentations <http://gcc2015.tsl.ac.uk/Abstracts> at the 2015 Galaxy Community Conference (GCC2015) <http://gcc2015.tsl.ac.uk/> is now open.* Abstract submission <http://gcc2015.tsl.ac.uk/Abstracts> for oral presentations closes 10 April, and poster submission closes 1 May. Poster authors will be notified of acceptance status within two weeks of submission, while oral presentation authors will be notified no later than 24 April. *Please consider presenting your work. If you are dealing with big biological data, then this meeting wants to hear about it.* *About GCC2015* GCC2015 <http://gcc2015.tsl.ac.uk/> is hosted by The Sainsbury Lab <http://www.tsl.ac.uk/>, in Norwich, United Kingdom <http://www.visitnorwich.co.uk/>, in July 2015. GCC has been expanded this year and will now offer two hackathons, two Training Days, and a two day conference <http://gcc2015.tsl.ac.uk/programme/>. The Training Days <http://gcc2015.tsl.ac.uk/training-day/> cover 15 different topics spanning the full Galactic spectrum of topics. *Take a look <http://gcc2015.tsl.ac.uk/programme/>.* Thanks, GCC2015 Organising Committee <http://gcc2015.tsl.ac.uk/organisers/> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

charts
by Michael Thon 24 Feb '15

24 Feb '15

Hi - I tried to install the package charts but it failed on the dependency charts_r_packages with this error: File "/home/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 139, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( tool_shed_repository, tool_dependency, actions_dict ) File "/home/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 97, in install_and_build_package initial_download=True ) File "/home/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step initial_download=initial_download ) File "/home/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 1238, in execute_step self.url_download( work_dir, filename, url, extract=False ) File "/home/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 183, in url_download raise Exception( err_msg ) Error downloading from URL http://cran.r-project.org/src/contrib/Matrix_1.1-4.tar.gz: HTTP Error 404: Not FoundInstalled tool dependencies: charts_r_packages It looks like matrix is now at version: Matrix_1.1-5.tar.gz -Mike

2 1

ValueError: Attribute 'fields' does not accept objects of type <type 'list'>
by Peter Cock 24 Feb '15

24 Feb '15

Hi all, I'm trying the (BETA 2) GitHub repository with a view to switching from hg for my Galaxy development. There are a couple of big scary tracebacks in the stdout which look like SQLalchemy incompatibilities - are these a recent issue? $ git clone $ cd galaxy $ cp config/galaxy.ini.sample config/galaxy.ini $ emacs config/galaxy.ini (set port and a few misc settings like admin users) $ ./run.sh ... galaxy.jobs.handler DEBUG 2015-02-24 14:49:20,223 Loaded job runners plugins: lwr:local galaxy.jobs.handler INFO 2015-02-24 14:49:20,224 job handler stop queue started galaxy.jobs.handler INFO 2015-02-24 14:49:20,238 job handler queue started galaxy.sample_tracking.external_service_types ERROR 2015-02-24 14:49:20,240 error reading external_service_type from path: simple_unknown_sequencer.xml Traceback (most recent call last): File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 32, in load_all external_service_type = self.load_external_service_type( os.path.join( self.root_dir, file_path ), visible ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 43, in load_external_service_type return ExternalServiceType( config_file, root, visible ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 60, in __init__ self.parse( root ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 78, in parse self.form_definition = form_factory.from_elem( root.find( 'form' ) ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/forms/forms.py", line 59, in from_elem return self.new( form_type, name, description=description, fields=fields, layout=layout, form_definition_current=form_definition_current ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/forms/forms.py", line 32, in new rval = FormDefinition( name=name, desc=description, form_type=self.form_types[form_type], form_definition_current=form_definition_current, layout=layout, fields=fields ) File "<string>", line 4, in __init__ File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/state.py", line 260, in _initialize_instance return manager.original_init(*mixed[1:], **kwargs) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/model/__init__.py", line 3417, in __init__ self.fields = fields File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 226, in __set__ instance_dict(instance), value, None) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 693, in set value, old, initiator) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 712, in fire_replace_event state, value, previous, initiator or self._replace_token) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/ext/mutable.py", line 445, in set value = cls.coerce(key, value) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/model/custom_types.py", line 48, in coerce return Mutable.coerce(key, value) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/ext/mutable.py", line 403, in coerce raise ValueError(msg % (key, type(value))) ValueError: Attribute 'fields' does not accept objects of type <type 'list'> galaxy.sample_tracking.external_service_types ERROR 2015-02-24 14:49:20,244 error reading external_service_type from path: applied_biosystems_solid.xml Traceback (most recent call last): File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 32, in load_all external_service_type = self.load_external_service_type( os.path.join( self.root_dir, file_path ), visible ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 43, in load_external_service_type return ExternalServiceType( config_file, root, visible ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 60, in __init__ self.parse( root ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/sample_tracking/external_service_types.py", line 78, in parse self.form_definition = form_factory.from_elem( root.find( 'form' ) ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/forms/forms.py", line 59, in from_elem return self.new( form_type, name, description=description, fields=fields, layout=layout, form_definition_current=form_definition_current ) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/forms/forms.py", line 32, in new rval = FormDefinition( name=name, desc=description, form_type=self.form_types[form_type], form_definition_current=form_definition_current, layout=layout, fields=fields ) File "<string>", line 4, in __init__ File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/state.py", line 260, in _initialize_instance return manager.original_init(*mixed[1:], **kwargs) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/model/__init__.py", line 3417, in __init__ self.fields = fields File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 226, in __set__ instance_dict(instance), value, None) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 693, in set value, old, initiator) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py", line 712, in fire_replace_event state, value, previous, initiator or self._replace_token) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/ext/mutable.py", line 445, in set value = cls.coerce(key, value) File "/mnt/galaxy/repositories/galaxy/lib/galaxy/model/custom_types.py", line 48, in coerce return Mutable.coerce(key, value) File "/mnt/galaxy/repositories/galaxy/eggs/SQLAlchemy-0.9.8-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/ext/mutable.py", line 403, in coerce raise ValueError(msg % (key, type(value))) ValueError: Attribute 'fields' does not accept objects of type <type 'list'> galaxy.workflow.scheduling_manager DEBUG 2015-02-24 14:49:20,247 Starting workflow schedulers ... Starting server in PID 360. serving on http://127.0.0.1:8081 Peter

2 2