- galaxy-dev - lists.galaxyproject.org

GCC2015 Registration is Open!
by Dave Clements 27 Apr '15

27 Apr '15

Hello all, We are tremendously pleased (and more than a little relieved) to announce that *registration for GCC2015 is now open <http://gcc2015.tsl.ac.uk/registration/>*. The 2015 Galaxy Community Conference <http://gcc2015.tsl.ac.uk/> includes 2 days of hackathons, 2 days of training, and a two day meeting <http://gcc2015.tsl.ac.uk/programme/> featuring accepted presentations, keynotes, poster sessions, lightning talks, birds-of-a-feather meetups, and plenty of networking. GCC2015 is hosted by The Sainsbury Laboratory <http://www.tsl.ac.uk/>, in Norwich, UK, and events run 4-8 July. Registration costs <http://gcc2015.tsl.ac.uk/registration/> depend on which events you register for <http://gcc2015.tsl.ac.uk/programme/>, your career stage/affiliation <http://gcc2015.tsl.ac.uk/registration/>, and when you register <http://gcc2015.tsl.ac.uk/registration/>*(register by 22 May - really, it will save you plenty).* Early bird registration starts at less than £40 / day for students and postdocs, and at £55 / day for other attendees from non-profits. You can also sign up for conference housing <http://gcc2015.tsl.ac.uk/logistics/#Lodging> during (and only during) registration. We apologize profusely for the delay in opening registration, and we thank you for your patience and faith. We promise the conference will againbe spectacular <https://wiki.galaxyproject.org/GalaxyProject/Statistics#Events> . Dave Clements <https://wiki.galaxyproject.org/DaveClements> and Martin Page <http://www.tsl.ac.uk/staff/martin-page/>, on behalf of the GCC2015 Organising Committee <http://gcc2015.tsl.ac.uk/organisers/>. PS: *And don't forget the poster abstract <http://gcc2015.tsl.ac.uk/Abstracts> submission deadline is 1 May.* -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

error in Galaxy (lefse)
by Borruso Luigimaria 26 Apr '15

26 Apr '15

Dear Galaxy-project staff, in the last two weeks, I have tried several time to use Lefse but it seems to have some problem. I always get the following message: Internal Server Error Galaxy was unable to successfully complete your request. An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. Thanks a lot Luigi Borruso, PhD Environmental Microbiology Faculty of Science & Technology Building K, Office 1.05 Free University of Bozen/Bolzano Piazza Università 5 I-39100 Bozen/Bolzano Italy Tel. +39 0471 017610 E-mail: luigimaria.borruso(a)unibz.it<mailto:lorenzo.brusetti@unibz.it> Website: www.unibz.it/sciencetechnology<http://www.unibz.it/sciencetechnology>

2 1

Announcement: Galaxy Docker Image 15.03
by Björn Grüning 25 Apr '15

25 Apr '15

Hi all! We are proud to present you the next release of Galaxy Docker. https://github.com/bgruening/docker-galaxy-stable https://registry.hub.docker.com/u/bgruening/galaxy-stable/ A few highlights: * based on Galaxy 15.03, with all it's new awesome features * Docker image is now based on Ubuntu 14.04 * move all required Galaxy dependencies to toolshed/requirements * use Ansible roles to build large parts of the image * export the supervisord web-interface (port 9002) * enable Galaxy reports webapp on GALAXY_URL/reports * enable/disable services like FTP or Galaxy reports during start-up The IPython integration was also updated and has a few new libraries to make it more convenient for R users. A big thanks to John Chilton for his enormous support and constant input. The following new releases are available: * docker pull bgruening/galaxy-stable:latest_2015.01.13 * docker pull bgruening/galaxy-stable:15.03 * docker pull bgruening/galaxy-stable (-> currently 15.03) As usual feedback is more than welcome! Cheers, Bjoern

2 4

InvalidRequestError: kombu.transport.sqlalchemy.Queue
by Asma Riyaz 24 Apr '15

24 Apr '15

Hi, I am receiving this traceback when I do restart Apache and not sure what to make of it: Exception in thread WorkflowRequestMonitor.monitor_thread: Traceback (most recent call last): File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/lib/python2.7/threading.py", line 530, in __bootstrap_inner self.run() File "/broad/software/free/Linux/redhat_6_x86_64/pkgs/python_2.7.1-sqlite3-rtrees/lib/python2.7/threading.py", line 483, in run self.__target(*self.__args, **self.__kwargs) File "/seq/regev_genome_portal/SOFTWARE/galaxy/lib/galaxy/workflow/scheduling_manager.py", line 158, in __monitor self.__schedule( workflow_scheduler_id, workflow_scheduler ) File "/seq/regev_genome_portal/SOFTWARE/galaxy/lib/galaxy/workflow/scheduling_manager.py", line 162, in __schedule invocation_ids = self.__active_invocation_ids( workflow_scheduler_id ) File "/seq/regev_genome_portal/SOFTWARE/galaxy/lib/galaxy/workflow/scheduling_manager.py", line 191, in __active_invocation_ids handler=handler, File "/seq/regev_genome_portal/SOFTWARE/galaxy/lib/galaxy/model/__init__.py", line 3254, in poll_active_workflow_ids WorkflowInvocation File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/scoping.py", line 150, in do return getattr(self.registry(), name)(*args, **kwargs) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py", line 1165, in query return self._query_cls(entities, self, **kwargs) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 108, in __init__ self._set_entities(entities) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 118, in _set_entities self._set_entity_selectables(self._entities) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 151, in _set_entity_selectables ent.setup_entity(*d[entity]) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 3036, in setup_entity self._with_polymorphic = ext_info.with_polymorphic_mappers File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/util/langhelpers.py", line 725, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/mapper.py", line 1877, in _with_polymorphic_mappers configure_mappers() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/mapper.py", line 2589, in configure_mappers mapper._post_configure_properties() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/mapper.py", line 1694, in _post_configure_properties prop.init() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/interfaces.py", line 144, in init self.do_init() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/relationships.py", line 1549, in do_init self._process_dependent_arguments() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/relationships.py", line 1605, in _process_dependent_arguments self.target = self.mapper.mapped_table File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/util/langhelpers.py", line 725, in __get__ obj.__dict__[self.__name__] = result = self.fget(obj) File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/relationships.py", line 1522, in mapper argument = self.argument() File "/seq/regev_genome_portal/SOFTWARE/galaxy/eggs/SQLAlchemy-0.9.8-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/ext/declarative/clsregistry.py", line 283, in __call__ (self.prop.parent, self.arg, n.args[0], self.cls) InvalidRequestError: When initializing mapper Mapper|Queue|kombu_queue, expression 'Message' failed to locate a name ("name 'Message' is not defined"). If this is a class name, consider adding this relationship() to the <class 'kombu.transport.sqlalchemy.Queue'> class after both dependent classes have been defined. This happens sometimes but when I shut down galaxy and re-start I don't get it anymore. Can someone guide me to a more permanent solution to this? Asma

2 1

README.rst missing on Tool Shed display?
by Peter Cock 24 Apr '15

24 Apr '15

Hi all, I don't see tool README files on the main view of a repository any more. Is this a recent regression? e.g. https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools and https://testtoolshed.g2.bx.psu.edu/view/bgruening/deeptools should show the readme.rst file e.g. https://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus and https://testtoolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus should show the README.rst file This might be specific to restructured-text markup - I'd have to trawl the ToolShed looking for a plain text README file to check that. Regards, Peter

1 0

History panel error
by Jose Juan Almagro Armenteros 24 Apr '15

24 Apr '15

Hello all, I have updated my Galaxy distribution today to the latest version 15.03 and the history panel is not being shown. The history seems to be there but Galaxy can't display the datasets. Even if I create a new history and I run a tool the dataset is not shown. It is just displayed an error which says: "An error occurred while getting updates from the server..." and the details shows this: { "agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36", "url": "http://galaxy.bric.ku.dk/history/view_multipl/api/histories/782c6cc7ec4cfd9…", "data": "", "options": { "data": {}, "silent": true, "parse": true, "emulateHTTP": false, "emulateJSON": false }, "xhr": { "readyState": 4, "responseText": "{\"err_msg\": \"Uncaught exception in exposed API method:\", \"err_code\": 0}", "responseJSON": { "err_msg": "Uncaught exception in exposed API method:", "err_code": 0 }, "status": 500, "statusText": "Internal Server Error", "responseHeaders": { "Date": "Fri, 24 Apr 2015 12:09:40 GMT\r", "cache-control": "max-age=0,no-cache,no-store\r", "Transfer-Encoding": "chunked\r", "Server": "PasteWSGIServer/0.5 Python/2.7.3\r", "Connection": "close\r", "x-frame-options": "SAMEORIGIN\r", "content-type": "application/json\r" } }, "source": [], "user": { "username": "jose", "quota_percent": 0, "total_disk_usage": 101352147, "nice_total_disk_usage": "96.7 MB", "email": "jjalmagro(a)gmail.com", "is_admin": true, "tags_used": [], "model_class": "User", "id": "72ad249754f05d26" } } Do you have any idea what could go wrong? Thank you and best regards, Jose

2 2

question about splitting bams
by Roberto Alonso CIPF 24 Apr '15

24 Apr '15

Hello, I ma trying ti write some code in order to give the possibility of parallelize some tasks. Now, I was with the problem of splitting a bam in some parts, for this I create this simple tool <parallelism method="multi" split_size="3" split_mode="number_of_parts" merge_outputs="output" split_inputs="input" ></parallelism> <command> java -jar /home/ralonso/software/GenomeAnalysisTK-3.3-0/GenomeAnalysisTK.jar -T UnifiedGenotyper -R /home/ralonso/BiB/Galaxy/data/chr_19_hg19_ucsc.fa -I $input -o $output 2> /dev/null; </command> <inputs> <param format="bam" name="input" type="data" label="bam"/> </inputs> <outputs> <data format="vcf" name="output" /> </outputs> But I have one problem, when I execute the tool it goes through this part of code (I am working in dev branch): *$galaxy/lib/galaxy/jobs/splitters/multi.py, line 75:* for input in parent_job.input_datasets: if input.name in split_inputs: this_input_files = job_wrapper.get_input_dataset_fnames(input.dataset) if len(this_input_files) > 1: log_error = "The input '%s' is composed of multiple files - splitting is not allowed" % str(input.name) log.error(log_error) raise Exception(log_error) input_datasets.append(input.dataset) So, it is raising the exception because this_input_files=2, concretely: ['/home/ralonso/galaxy/database/files/000/dataset_171.dat', '/home/ralonso/galaxy/database/files/_metadata_files/000/metadata_13.dat'], I guess that: *dataset_171.dat*: It is the bam file. *metadata_13.dat*: It is the bai file. So, Galaxy can't move on and I don't know which would be the best solution. Maybe change the *if* to check only non-metadata files? I think I should use both files in order to create the bam sub-files, but this would be inside the Bam class, under *binary.py* file. Could you please guide me before I mess things up? Thanks so much -- Roberto Alonso Functional Genomics Unit Bioinformatics and Genomics Department Prince Felipe Research Center (CIPF) C./Eduardo Primo Yúfera (Científic), nº 3 (junto Oceanografico) 46012 Valencia, Spain Tel: +34 963289680 Ext. 1021 Fax: +34 963289574 E-Mail: ralonso(a)cipf.es

2 3

Shared Data Library issues with permissions
by Ganote, Carrie L 23 Apr '15

23 Apr '15

Hi Devs, Maybe I'm missing something, but I'm having trouble getting permissions right on the Data Libraries. I can reproduce this: As an admin, I go to Manage data libraries. On top right, I click Create new data library. Let's call it test. I Add a folder in test, called "more testing". I check the permissions of the library in Library Actions->Edit permissions and make sure that all roles associated are blank. I check permissions in more testing to make sure they are all blank. I click on Shared Data -> Data Libraries to see if I can view the files with the user perspective. I click on the Test library I just created: Data Library “Test” The data library 'Test' does not contain any datasets that you can access. I have a fairly recent (April 2015) update to Galaxy. Any advice? I first started having trouble getting access for a user to their own library; when I associated their role with their own folder, they still couldn't see anything they could access. I should at least get an empty folder. Sincerely, Carrie Ganote

2 1

Serving a galaxy instance over Apache
by Asma Riyaz 23 Apr '15

23 Apr '15

Hi, I have been through previous issues regarding this issue. So far, I have included the rewrite Engine logic written in /etc/apache2/sites-available/defaults - looks like so: <VirtualHost *:80> ..... ..... RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /seq/SOFTWARE/galaxy/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /seq/SOFTWARE/galaxy/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /seq/SOFTWARE/galaxy/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /seq/SOFTWARE/galaxy/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /seq/SOFTWARE/galaxy/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://[my IP address]/$1 [P] I restarted apache after this and accessed : http://[my IP address]/galaxy but no webpage is found. Any advice on how I can proceed to configure the apache correctly? Thank you, Asma

2 2

Re: [galaxy-dev] Galaxy on HPC and Bright Cluster Manager?
by David Trudgian 23 Apr '15

23 Apr '15

Hi Carlos, sorry for the slow reply. With a small 10 user setup (I assume just one group) your installation is probably going to be a lot less complex than ours. We are running services for users from multiple labs and departments, so a chief concern was making sure private datasets always stay private - which is relatively involved when using Galaxy to run jobs on a general-purpose shared cluster. I think if I had to recommend picking a job scheduler I would suggest SLURM since it's probably the most 'fashionable' choice at present. You also have the advantage that SLURM is used with Galaxy in the Galaxy docker image etc. which ensures people notice if the Galaxy->DRMAA->SLURM setup isn't working. I've also used Galaxy with GridEngine in the past, and that was fine - but is becoming a less common choice as a scheduler. Having said that, I don't think that the job scheduler needs to be your biggest concern. I would focus most on the file system and user account setup you are going to need. * How are you going to migrate from standalone Galaxy to a situation where your new cluster can see the Galaxy data files, tools etc. Are you purchasing storage with the cluster? If so do you move Galaxy onto that storage, or can you mount existing Galaxy data onto the cluster nodes? If you can do that, is your networking such that performance is sufficient for the type of analysis you are going to run? * Do you need to, or will you need to, keep track of per-user usage of the cluster for things that Galaxy will be running? If not then you can just have a galaxy user on your cluster and things are pretty easy for file permissions etc. If you need to track jobs per-user then it becomes more complex, and the solution depends on how much privacy you need for datasets, how your cluster will authenticate users etc. The filesystem and user accounts issues are, in my mind, the ones to focus on. You can always modify Galaxy's config to switch to a different job scheduler fairly easily. You cannot as easily move around large amounts of data, and reconcile local vs cluster user accounts, should that be necessary. Cheers, Dave Trudgian -----Original Message----- From: Carlos Lijeron [mailto:clijeron@hunter.cuny.edu] Sent: Wednesday, April 22, 2015 10:54 AM To: David Trudgian; John Chilton Cc: RODRIGO GONZALEZ SERRANO Subject: Re: [galaxy-dev] Galaxy on HPC and Bright Cluster Manager? Hello David, Thank you for the great feedback. We are at Hunter College in NYC, part of the City University of New York. We recently ordered the cluster which comes with Bright Cluster Management, and our PI wants to implement Galaxy for all the users (about 10) on the cluster and manage all job submissions through a job scheduler. So, to answer your question, we are not really using any scheduler at this point, but only a stand alone server with a local installation of Galaxy. Our Cluster should be assembled and installed by the end of May, so I¹m trying to gather as much information as possible in preparation for the deployment. Based on your experience, what do you think I should focus on to ensure we maximize outcome and reduce the possibility of mistakes? In other words, any lessons learned that you would like to share will be greatly appreciated. Thanks again, Carlos Lijeron. On 4/22/15, 10:34 AM, "David Trudgian" <David.Trudgian(a)UTSouthwestern.edu> wrote: >Carlo, > >We have Bright Cluster Manager in use on our cluster for node >provisioning etc. but the actual job scheduler in use in our case is >SLURM, which we use directly. > >Are you using one of the integrated workload managers such as SLURM / >SGE / TORQUE directly, or indirectly via cmsub? > >I guess the easiest way to come up with some kind of advice is if you >can provide an example of generic job script you are using on your system. >If you're using cmsub is it specifying a --wlmanager etc. > >DT > >-----Original Message----- >From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On >Behalf Of John Chilton >Sent: Wednesday, April 22, 2015 8:26 AM >To: Carlos Lijeron >Cc: galaxy-dev(a)lists.galaxyproject.org >Subject: Re: [galaxy-dev] Galaxy on HPC and Bright Cluster Manager? > >Hello Carlos, > > I have never heard of anyone running Galaxy with Bright Cluster >Manager (though hopefully someone will chime in if they have). If you >are interested in adding support it should be possible. One >complication is that Bright Cluster Manager doesn't appear to have a >DRMAA interface >(http://www.drmaa.org/) which is the most direct way to utilize new DRMs. >Without that my approach would be to build a new CLI runner: > >There are a few examples here that one can use as template: > >https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/jobs/runner >s/u >til/cli/job > >I guess you would have to write a new one targeting cmsub I guess - you >also need to be able to parse a job status somehow - I haven't figured >out how to do that from the documentation - but I assume there is a way. > >I looks like Bright supports running SGE, SLURM, and Torque on the >cluster - doing this and interfacing with one of those more common >options directly might be a better approach for Galaxy (and other users >if your cluster has them). > >-John > > > > > > > >On Wed, Apr 22, 2015 at 8:56 AM, Carlos Lijeron ><clijeron(a)hunter.cuny.edu> wrote: >> Good day everyone, >> >> >> >> Has anyone of you been able to implement Galaxy on a HPC using Bright >> Cluster Manager as the main DRM? I noticed that only a few have been >> known to work with Galaxy, but the list does not include Bright. Any >> advice/ideas will be greatly appreciated. >> >> >> >> TORQUE Resource Manager >> >> PBS Professional >> >> Open Grid Engine >> >> Univa Grid Engine (previously known as Sun Grid Engine and Oracle >> Grid >> Engine) >> >> Platform LSF >> >> HTCondor >> >> Slurm >> >> Galaxy Pulsar (formerly LWR) >> >> >> >> >> >> Thanks. >> >> >> >> >> >> Carlo >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this and other >> Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >___________________________________________________________ >Please keep all replies on the list by using "reply all" >in your mail client. To manage your subscriptions to this and other >Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > >To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > >________________________________ > >UT Southwestern > > >Medical Center > > > >The future of medicine, today. >

1 0