- galaxy-dev - lists.galaxyproject.org

Improving "New Galaxy tool shed repository alert" emails
by Peter Cock 12 Nov '12

12 Nov '12

Hi all, I would find the "New Galaxy tool shed repository alert" more useful if the important details were at the start of the message, so this is shown in the 'peek' text of my email client (in this case, GoogleMail). Perhaps going further the new repository's name could be in the title? Thanks, Peter

2 2

Can I change the tool runner for set_metadata.sh?
by Luobin Yang 12 Nov '12

12 Nov '12

Hi, guys, I noticed that set_metadata.sh is not in the tools folder as other tools. Can I set a tool runner for set_metadata.sh so that I can use specific resources (like specific compute node in the cluster) to run set_metadata.sh? Thanks, Luobin

2 2

Re: [galaxy-dev] Galaxy at Duke
by Dave Clements 12 Nov '12

12 Nov '12

Hi Alan, I'm CC'ing the Galaxy-Dev list on my response to your question. I talked with Jen Jackson on this. Her suggestion is that the symlink is most likely the problem somehow - this time. When in doubt, getting rid of these good place to start. So recommended starting solution to try - use full paths for everything. Also probably keep all on same partition for the best performance, but that would be hard to defend (there was some debate about this recently). Many thanks to Jen for her (as always) careful consideration. Dave C. On Mon, Nov 5, 2012 at 7:15 AM, Alan V. Cowles <alan.cowles(a)duke.edu> wrote: > Hey Dave, > > Thanks for getting back in touch with me. > > We are running our galaxy instance on a vm... with Centos (RHEL) 6.2, 2 > vcpus, and 2gb vram, with a remote postgresql prod server as our database > instance. > > Our first issue was trying to get it so we could load sequence and larger > files directly from an nfs mounted directory on the server, without > uploading via the web (php vs large files problem.) We accomplished this > via the data libraries interface but it's still not very user intuitive. > > Our first demo user attempted to upload 3 large files and actually filled > up the / partition on the server as galaxy was installed at /usr/local. > > After scrapping and rebuilding that, we now have a several hundred GB nfs > partition, mounted and /usr/local/galaxy is a symlink to a folder on this > volume. > > The issue we are currently seeing is during an upload, or even just > linking files it runs for a while before erroring out and saying there is > no information on the error. The file seems to be there and taking up the > correct amount of disk space but galaxy has a red bar. > > The galaxy server instance is running back grounded in a screen so we can > always switch over and look at the output it shows to standard out as > errors happen, and it seems that it has trouble changing permissions on the > files in the nfs directory, even though, root squash is disabled, and the > entire directory is 777 at the top level and everything underneath is owned > galaxy:galaxy. > > AC > > > > > -- <http://galaxyproject.org/wiki/GCC2012>http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://wiki.galaxyproject.org/

4 8

Re: [galaxy-dev] Error when running cleanup_datasets.py
by Liisa Koski 12 Nov '12

12 Nov '12

Hi Nate, I'm back to trying to figure this out again as I am running out of disk space. I added the bit of code you suggested below, but I don't think it helped, I'm not so familiar with python. I'm now running Galaxy Reports and it tells me that I have 7479 datasets that were deleted but have not yet been purged. I get the error below when I run cleanup_datasets.py with both the -5 and -4 flag Marking as deleted: LibraryDatasetDatasetAssociation id 6907 (for dataset id 51991) Deleting dataset id 51991 Deleting library dataset id 7225 Traceback (most recent call last): File "scripts/cleanup_datasets/cleanup_datasets.py", line 526, in <module> if __name__ == "__main__": main() File "scripts/cleanup_datasets/cleanup_datasets.py", line 124, in main purge_folders( app, cutoff_time, options.remove_from_disk, info_only = options.info_only, force_retry = options.force_retry ) File "scripts/cleanup_datasets/cleanup_datasets.py", line 247, in purge_folders _purge_folder( folder, app, remove_from_disk, info_only = info_only ) File "scripts/cleanup_datasets/cleanup_datasets.py", line 499, in _purge_folder _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) File "scripts/cleanup_datasets/cleanup_datasets.py", line 499, in _purge_folder _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) File "scripts/cleanup_datasets/cleanup_datasets.py", line 497, in _purge_folder _purge_dataset_instance( ldda, app, remove_from_disk, info_only = info_only ) #mark a DatasetInstance as deleted, clear associated files, and mark the Dataset as deleted if it is deletable File "scripts/cleanup_datasets/cleanup_datasets.py", line 373, in _purge_dataset_instance log.debug( '%s %s has None dataset' % ( type( dataset_instance ), dataset_instance.id ) ) AttributeError: 'NoneType' object has no attribute 'id' Thanks in advance for your help, Liisa From: Nate Coraor <nate(a)bx.psu.edu> To: Liisa Koski <liisa.koski(a)basf.com> Cc: galaxy-dev(a)lists.bx.psu.edu Date: 02/10/2012 10:50 AM Subject: Re: [galaxy-dev] Error when running cleanup_datasets.py On Oct 2, 2012, at 10:44 AM, Liisa Koski wrote: > Hi Nate, > That select statement does not return anything :( Could you add a bit of debugging to the script to see what the id is of the dataset_instance that has a None dataset? if dataset_instance is None: log.debug( '%s %s has None dataset' % ( type( dataset_instance ), dataset_instance.id ) ) Thanks, --nate > > Thanks, > Liisa > > > > > > From: Nate Coraor <nate(a)bx.psu.edu> > To: Liisa Koski <liisa.koski(a)basf.com> > Cc: galaxy-dev(a)lists.bx.psu.edu > Date: 01/10/2012 01:01 PM > Subject: Re: [galaxy-dev] Error when running cleanup_datasets.py > > > > On Sep 24, 2012, at 10:41 AM, Liisa Koski wrote: > > > Hello, > > I am trying to run the cleanup scripts on my local installation but get stuck when trying to run the following: > > > > ./scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 10 -5 -r > > > > Deleting library dataset id 7225 > > Traceback (most recent call last): > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 524, in <module> > > if __name__ == "__main__": main() > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 124, in main > > purge_folders( app, cutoff_time, options.remove_from_disk, info_only = options.info_only, force_retry = options.force_retry ) > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 247, in purge_folders > > _purge_folder( folder, app, remove_from_disk, info_only = info_only ) > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 497, in _purge_folder > > _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 497, in _purge_folder > > _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 495, in _purge_folder > > _purge_dataset_instance( ldda, app, remove_from_disk, info_only = info_only ) #mark a DatasetInstance as deleted, clear associated files, and mark the Dataset as deleted if it is deletable > > File "./scripts/cleanup_datasets/cleanup_datasets.py", line 376, in _purge_dataset_instance > > ( dataset_instance.__class__.__name__, dataset_instance.id, dataset_instance.dataset.id ) > > AttributeError: 'NoneType' object has no attribute 'id' > > Hi Liisa, > > It'd appear that you have a library_dataset_dataset_association in your dataset that lacks an associated dataset. Does 'select id from library_dataset_dataset_association where dataset_id is null' in your database return anything? > > --nate > > > > > > > Any help would be much appreciated. > > > > Thanks, > > Liisa > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > >

2 2

Tags API
by Kyle Ellrott 12 Nov '12

12 Nov '12

Are there any docs or examples about how to add/remove tags from items via the galaxy web-api? Thanks, Kyle Ellrott

4 6

workflow not working
by Rolando Mantilla 12 Nov '12

12 Nov '12

hello guys I'm kinda new using galaxy but I'm having an issue I tried to run my workflow and it just stopped, I was able to load both of the sequences but I could not run the whole workflow. I do have enable_job_execution=True, so I don't know what is wrong, please help!!

2 1

galaxy/cloudman failure handling
by Jorrit Boekel 12 Nov '12

12 Nov '12

Dear list, I would like to start using Amazon's spot pricing model for my Galaxy/Cloudman instances (e.g. an on demand master node with spot instance worker nodes). However, this means that Amazon at times of spot prices higher than my set price limit will shutdown my instances without notion. I was therefore looking for fault tolerance mechanisms in the galaxy project, which I seem to remember existed. Somehow I can't find anything about it right now though. I've tested a little bit, and it seems that as soon as one reboots instances or manually kills a job or task, the whole job is deleted and set to error state. I am not that knowledgeable in cluster computing, so I don't really know what handles what here, but this would be an ideal starting point to learn something about SGE and queue handling. Is there any mechanism in place that deals with node failure, network problems, etc? If not, would it be hard to implement? cheers, jorrit boekel

2 1

Phylotastic hackathon #2 -- last call for applications (due by Friday, 16 Nov, midnight EST)
by Arlin Stoltzfus 12 Nov '12

12 Nov '12

Dear galaxy developers-- I'd like to bring your attention to the call for participation included below this message. A "Phylotastic" system is a distributed tree-of-life delivery system composed of web services. Although it is not readily apparent from the presentation linked below, the project includes some work on using Galaxy as a framework for managing a phylotastic workflow. See the demos page (http://phylotastic.org/demos.html) and the "Making Galaxy Phylotastic!" screencast (http://youtu.be/kMME658xOu4 for more. I would be happy to respond to questions about the project. Please consider applying if you are excited by the project and think that you can contribute. Regards, Arlin ----- Phylogenies! Hacking! Tucson in January! An implicit promise of the Tree of Life project is that, ultimately, expert knowledge of species phylogeny will be accessible and usable by everybody. In other words, we will all be able to get the species trees we need, in a useable form, when we need them. The Phylotastic project aims to make that vision a reality, by developing a loosely coupled system of components that, in response to a user’s query (a list of taxa plus conditions), will rectify names, find suitable source trees, prune and graft to get the right species, estimate branch lengths, and return the results with metadata on sources and methods. In June 2012, the HIP (Hackathons, Interoperability, Phylogenies) working group of NESCent staged a hackathon (http://en.wikipedia.org/wiki/Hackathon) to prototype components of a Phylotastic system, and implement demos to show their potential. Thanks to 25 participants who responded to an open call (just like this one), that hackathon was an extraordinary success (see http://phylotastic.org/) Now we are gearing up for a second hackathon, to take another step toward providing computable, convenient, credible access to the Tree of Life. We aim to recruit a diverse group of participants, including students, faculty, postdocs, and staff of both genders and from different backgrounds. Only some of the seats at the hackathon will be filled by expert hackers who spend their time coding. We also need folks who can: * Provide knowledge of workflows and downstream uses of trees * Design user interfaces * Test software * Make screencasts, develop tutorials, and document user experiences Applications are now being accepted to Phylotastic 2, which will take place January 28th, 2013 to February 1st, 2013 at the BIO5 facility in warm and sunny Tucson, AZ (http://bio5.arizona.edu/about-bio5) Information on how to apply is provided below. Partial support (and, if funds allow, full support) for travel, food and lodging will be available to each successful applicant who indicates need. We particularly welcome applications from women and members of underrepresented groups. To understand how you might be able to contribute, review the slideshow (ppt format, http://bit.ly/RWRgIc, or PDF format, http://bit.ly/POaoci) consult the web site (phylotastic.org) or contact a member of the HIP leadership team (listed below). Sincerely, the HIP (Hackathons, Interoperability and Phylogenies) Leadership Team HOW TO APPLY Before 16 November, 2012 at midnight, EST, fill out the online form here http://tinyurl.com/PhyloTastic2. Note that: * The application form asks you to describe how you can contribute to Phylotastic. Remember that we are not just looking for expert coders. There are many other ways to contribute. * Support for travel, housing and meals will be available to each successful applicant who indicates need. * Please understand that your application is not a guarantee of participation in the event. Funds and space are limited: we may not be able to sponsor every individual who is qualified to participate. LEADERSHIP TEAM Karen Cranston, karen.cranston(a)nescent.org Brian Sidlauskas, brian.sidlauskas(a)oregonstate.edu Arlin Stoltzfus, arlin(a)umd.edu Mike Rosenberg mrosenb(a)asu.edu Brian O’Meara, bomeara(a)utk.edu Mark Westneat mwestneat(a)fieldmuseum.org Enrico Pontelli epontell(a)cs.nmsu.edu Rutger Vos rutgeraldo(a)gmail.com Naim Matasci nmatasci(a)iplantcollaborative.org ------- Arlin Stoltzfus (arlin(a)umd.edu) Fellow, IBBR; Adj. Assoc. Prof., UMCP; Research Biologist, NIST IBBR, 9600 Gudelsky Drive, Rockville, MD, 20850 tel: 240 314 6208; web: www.molevol.org

1 0

Bug report : JSONDecodeError
by Mathieu Bahin 12 Nov '12

12 Nov '12

Hi, I am using my own Galaxy instance and I had a problem removing a tool from the tool shed : mothur. The installation was ok and I the mothur tool running without any problem. But when I tried to uninstall the tool, I get an error (screenshot enclosed). And I get the same every time I try to go on the 'Manage tool shed repositories' in the Admin panel. I have tried to rebuilt the eggs and to restart Galaxy basically. Actually, I don't really understand the error message. Cheers, Mathieu Bahin

2 5

Error: change attributes generates error on fresh dist-release
by Bossers, Alex 12 Nov '12

12 Nov '12

Hi All, Due to previous DB errors we decided to let galaxy-dist repopulate the database (MySQL). We purged all tables from the DB and when restarting galaxy it repopulates the DB on the fly while doing all migration steps. A nicely running galaxy as the result. I encountered a strange error which might be linked to this... or not... When changing the attribute of a dataset I get the error below; We never had this before so my guess is that the DB is not constructed properly. We are at tip 7986:12fcd068b12e of the dist release. Any pointer to fix this? There is currently no user-data or histories anymore so I can experiment... before I open it for running publicly again... Thanks; Alex URL: http://galaxy.wur.nl:8080/galaxy/datasets/b4bd1d7d8ca50a82/edit Module paste.exceptions.errormiddleware:143 in __call__ >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> environ, self.app) Module paste.wsgilib:539 in intercept_output >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> body = method( trans, **kwargs ) Module galaxy.webapps.galaxy.controllers.dataset:496 in edit >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> all_roles = trans.app.security_agent.get_legitimate_roles( trans, data.dataset, 'root' ) Module galaxy.security:192 in get_legitimate_roles >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> return self.get_all_roles( trans, cntrller ) Module galaxy.security:168 in get_all_roles >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> return self.sort_by_attr( [ role for role in roles ], 'name' ) Module galaxy.security:137 in sort_by_attr >> <http://galaxy.wur.nl:8086/galaxy/datasets/b4bd1d7d8ca50a9f/edit> intermed = map( None, map( getattr, seq, ( attr, ) * len( seq ) ), xrange( len( seq ) ), seq ) AttributeError: 'NoneType' object has no attribute 'name'

1 1