October 2012 - galaxy-dev - lists.galaxyproject.org

jobName lost when drmaa used with real user use
by David Hoover 03 Oct '12

03 Oct '12

When I use the real user method for running drmaa jobs as the real user, the jobName value gets lost. It's kind of annoying, but not a show stopper. The other values (outputPath, errorPath, remoteCommand, nativeSpecification, etc.) are all there, but not jobName. This doesn't happen when the drmaa jobs are run using the default Galaxy user. Weird. David Hoover Helix Systems Staff

3 3

Cannot upload files onto Galaxy main
by So, Kathy GZ/US 03 Oct '12

03 Oct '12

Hi, I'm having problems uploading files. They are 3 - 30 MB files that are stuck in the "Job is waiting to run" stage. I've unsuccessfully tried uploading them from my computer and via ftp. I've tried closing out of Galaxy and logging in again. I've read that another user was having success with Internet Explorer 8.0, but that hasn't worked for me. Any help would be greatly appreciated. Thanks! Kathy

1 0

Re: [galaxy-dev] submitting jobs as the real user
by David Hoover 03 Oct '12

03 Oct '12

No, that's not it. The galaxy user, as well as all other users, have umask 022 by default. I believe the problem is with Apache uploading. I can't figure out how to reconfigure Apache to set the permissions they way I want them. David On Oct 2, 2012, at 1:17 PM, Oleksandr Moskalenko wrote: > Hi David, > > I wonder if the user's umask is the problem. > > I set the galaxy user's umask to 022 and our default umask on the cluster is 0022 for all users. > > On Oct 2, 2012, at 12:55 PM, David Hoover <hooverdm(a)helix.nih.gov> wrote: > >> I'm playing with the "submitting jobs as the real user" method, and immediately ran into a problem. Uploaded files are created in mode 600, owned by the user. Galaxy then gives an error that the tmp file is not readable: >> >> IOError: [Errno 13] Permission denied: '/spin1/users/galaxy/galaxy2/database/tmp/tmp3KHIDt' >> >> Is there any way of changing the permissions on the temporary uploaded file to 666? >> >> David Hoover >> Helix Systems Staff >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/

2 5

Re: [galaxy-dev] Trouble setting admin status and uploading files
by Nate Coraor 03 Oct '12

03 Oct '12

On Oct 3, 2012, at 10:03 AM, Jackie Lighten wrote: > Thanks for your reply, Nate. > > I guessed as much, and did attempt to log in via my registered account. I > get the same error. I am guessing this is because I am on > "localhost:8080". Do I need to edit to universe_wsgi.ini file to tell my > local instance to search the main galaxy database of registered users? > > I already edited this file with my mail settings. Hi Jack, You'll need to register a new account in your local instance. Local instances don't use the same user database as the public site. See the 'Register' item under the 'User' menu at the top of your Galaxy instance. --nate > > Thank for the help, > > Jack > > On 2012-10-03 9:42 AM, "Nate Coraor" <nate(a)bx.psu.edu> wrote: > >> On Oct 3, 2012, at 8:23 AM, Jackie Lighten wrote: >> >>> >>> >>> Hi All, >>> >>> I installed a local Galay instance but found uploads to be incredible >>> slow. I have seen this can be remedied through Admin controls. However >>> my modifications to "universe_wsgi.ini" still do not enable me to log in >>> as Admin showing the following error message: "No such user Š" >>> >>> I edited universe_wsgi.ini as follows: >>> >>> 1. ### Admin Users >>> admin_users = my email address >>> >>> I closed Galaxy and re-launched with the modified file, however still >>> no login ability. >>> I am running this on a desktop, no network. >>> Any help would be appreciated. >>> >>> Cheers, >>> >>> Jack >> >> Hi Jack, >> >> Setting an account as an admin does not create the account. You'll need >> to register it before you can log in to it. >> >> --nate >> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> > > >

1 0

Improved error logging in TaskWrapper
by Peter Cock 03 Oct '12

03 Oct '12

Hi all, I've been running into some sporadic errors on our Cluster while using the latest development Galaxy, and the error handling has made this quite difficult to diagnose. For a user perspective, the jobs seem to run, get submitted to the cluster, and finish, and the data looks OK via the 'eye' view icon, but is red in the history with: 0 bytes An error occurred running this job: info unavailable Furthermore, the stdout and stderr via the 'info' icon are blank. For watching the log (and adding more diagnosis lines), what is happening is the job is being split and sent out to the cluster fine, and starts running. If one of the tasks fails (and this seems to be happening due to some sort of file system error on our cluster), Galaxy spots this, and kills the rest of the jobs. That's good. The problem is it fails to record any record of why the job died. This is my suggestion for now - it would be nice to go further and fill the info text show in the history peep as well?: $ hg diff diff -r 4de1d566e9f8 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py Fri Sep 21 11:02:50 2012 +0100 +++ b/lib/galaxy/jobs/__init__.py Fri Sep 21 11:59:27 2012 +0100 @@ -1061,6 +1061,14 @@ log.error( "stderr for job %d is greater than 32K, only first part will be logged to database" % task.id ) task.stderr = stderr[:32768] task.command_line = self.command_line + + if task.state == task.states.ERROR: + # If failed, will kill the other tasks in this job. Record this + # task's stdout/stderr as should be useful to explain the failure: + job = self.get_job() + job.stdout = ("(From one sub-task:)\n" +task.stdout)[:32768] + job.stderr = ("(From one sub-task:)\n" +task.stderr)[:32768] + self.sa_session.flush() log.debug( 'task %d ended' % self.task_id ) Regards, Peter

2 2

Trouble setting admin status and uploading files
by Jackie Lighten 03 Oct '12

03 Oct '12

Hi All, I installed a local Galay instance but found uploads to be incredible slow. I have seen this can be remedied through Admin controls. However my modifications to "universe_wsgi.ini" still do not enable me to log in as Admin showing the following error message: "No such user " I edited universe_wsgi.ini as follows: 1. ### Admin Users admin_users = my email address I closed Galaxy and re-launched with the modified file, however still no login ability. I am running this on a desktop, no network. Any help would be appreciated. Cheers, Jack

2 1

Re: [galaxy-dev] Galaxy in cloud (not Amazon)
by Enis Afgan 02 Oct '12

02 Oct '12

Yes there is, the setup is just not documented very well at this point... Also, are you familiar with the cloud concepts or just looking to use Galaxy? The basic approach is to setup a CloudMan (usecloudman.org) machine image on the local cloud (using the image building process from https://github.com/chapmanb/cloudbiolinux is the recommended method). Install Galaxy as well as all of its tools, dependencies, and reference data on the appropriate block storage volumes and turn those into snapshots (see this paper for the architecture overview http://onlinelibrary.wiley.com/doi/10.1002/cpe.1836/full and then Galaxy's wiki (http://wiki.g2.bx.psu.edu/Admin) for the details on how to set everything up). Beyond that, it's a matter of making sure it all works as desired on your setup. You'll probably also want to use a version of the code similar to https://github.com/chapmanb/biocloudcentral to launch instances because for the non-amazon case, the user data ( http://wiki.g2.bx.psu.edu/CloudMan/UserData) required by an instance is a bit tedious to compose by hand (the user data link just explains what the user data is but the full set of user data fields required for non-amazon clouds is not yet documented - I'll do that soon). Hope this helps. I'm also CCing the galaxy-dev mailing list because others may be interested in this topic as well. Enis On Tue, Oct 2, 2012 at 9:37 PM, <pandorinpan(a)gmail.com> wrote: > Hi Enis! > In our institute we process biological data. And we have cloud, based on > OpenStack. > We want to use the Galaxy in the cloud (OpenStack). Are there any > solutions for this? (or just Amazon) > > Thank you so much for the Galaxy! :) And sorry for my English. >

1 0

Gestion d'un nombre indéfini de fichiers output / Managment of undefined outputs
by Sarah Maman 02 Oct '12

02 Oct '12

Bonjour, Je souhaiterai intégrer un script perl (script quantifier de mirdeep2) dans un nouveau tool de mon instance locale de Galaxy. Le problème est que ce script génère plusieurs outputs (fichiers images) mais le nombre d'outputs est variable d'une fois sur l'autre. Il me semble que le nombre d'outputs doit être défini et fixe dans le fichier xml du tool ? Ou existe t'il un moyen pour que le nombre d'outputs ne soit pas précisé à l'avance ? Merci d'avance pour votre aide, Sarah Maman ************** Hello, I would like to integrate a perl script (script quantifier of mirdeep2) in a new tool in my local instance of Galaxy. The problem is that this script generates several outputs (image files), but the number of outputs varies from one time to another. It seems that the number of outputs must be defined and fixed in the tool xml file ? Or is there a way to not specify the number of outputs ? Thank you in advance for your help, Sarah Maman

2 1

submitting jobs as the real user
by David Hoover 02 Oct '12

02 Oct '12

I'm playing with the "submitting jobs as the real user" method, and immediately ran into a problem. Uploaded files are created in mode 600, owned by the user. Galaxy then gives an error that the tmp file is not readable: IOError: [Errno 13] Permission denied: '/spin1/users/galaxy/galaxy2/database/tmp/tmp3KHIDt' Is there any way of changing the permissions on the temporary uploaded file to 666? David Hoover Helix Systems Staff

1 0

Error when running cleanup_datasets.py
by Liisa Koski 02 Oct '12

02 Oct '12

Hello, I am trying to run the cleanup scripts on my local installation but get stuck when trying to run the following: ./scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d 10 -5 -r Deleting library dataset id 7225 Traceback (most recent call last): File "./scripts/cleanup_datasets/cleanup_datasets.py", line 524, in <module> if __name__ == "__main__": main() File "./scripts/cleanup_datasets/cleanup_datasets.py", line 124, in main purge_folders( app, cutoff_time, options.remove_from_disk, info_only = options.info_only, force_retry = options.force_retry ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 247, in purge_folders _purge_folder( folder, app, remove_from_disk, info_only = info_only ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 497, in _purge_folder _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 497, in _purge_folder _purge_folder( sub_folder, app, remove_from_disk, info_only = info_only ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 495, in _purge_folder _purge_dataset_instance( ldda, app, remove_from_disk, info_only = info_only ) #mark a DatasetInstance as deleted, clear associated files, and mark the Dataset as deleted if it is deletable File "./scripts/cleanup_datasets/cleanup_datasets.py", line 376, in _purge_dataset_instance ( dataset_instance.__class__.__name__, dataset_instance.id, dataset_instance.dataset.id ) AttributeError: 'NoneType' object has no attribute 'id' Any help would be much appreciated. Thanks, Liisa

2 2