April 2012 - galaxy-dev - lists.galaxyproject.org

Error: Job output not returned from cluster
by Louise-Amélie Schmitt 30 Apr '12

30 Apr '12

Hello everyone, I'm still trying to set up the job submission as the real user, and I get a mysterious error. The job obviously runs somewhere and when it ends it is in error state and displays the following message: "Job output not returned from cluster" In the Galaxy log I have the following lines when the job finishes running: galaxy.jobs.runners.drmaa DEBUG 2012-04-23 10:36:41,509 (1455/9161620.pbs-master2.embl.de) state change: job finished, but failed galaxy.jobs.runners.drmaa DEBUG 2012-04-23 10:36:41,511 Job output not returned from cluster galaxy.jobs DEBUG 2012-04-23 10:36:41,547 finish(): Moved /g/funcgen/galaxy-dev/database/job_working_directory/001/1455/galaxy_dataset_2441.dat to /g/funcgen/galaxy-dev/database/files/002/dataset_2441.dat galaxy.jobs DEBUG 2012-04-23 10:36:41,755 job 1455 ended galaxy.datatypes.metadata DEBUG 2012-04-23 10:36:41,755 Cleaning up external metadata files galaxy.datatypes.metadata DEBUG 2012-04-23 10:36:41,768 Failed to cleanup MetadataTempFile temp files from /g/funcgen/galaxy-dev/database/job_working_directory/001/1455/metadata_out_HistoryDatasetAssociation_1606_npFIJM: No JSON object could be decoded: line 1 column 0 (char 0) The /g/funcgen/galaxy-dev/database/job_working_directory/001/1455/ directory is empty and /g/funcgen/galaxy-dev/database/files/002/dataset_2441.dat exists but is empty. Any ideas about what can go wrong there? Any lead would be immensely appreciated! Thanks, L-A

4 5

Error running job as real user: looking et the wrong dir?
by Louise-Amélie Schmitt 30 Apr '12

30 Apr '12

Hi everyone, I'm (still) having issues with running jobs as the real user on our PBS Pro cluster. When I try running a job, it ends up in error state and displays the following error message: touch: cannot touch `/home/galaxy/.drmaa/9167860.pbs-master2.embl.de.started': Permission denied /g/funcgen/galaxy-dev/database/pbs/galaxy_1516.sh: line 15: syntax error: unexpected end of file /var/spool/PBS/mom_priv/jobs/9167860.pbs-master2.embl.de.SC: line 1: /home/galaxy/.drmaa/9167860.pbs-master2.embl.de.exitcode: Permission denied I suspect this is due to some code executed as the real user (since the permission is denied) but then why looking at /home/galaxy instead of /home/realuser ? If you have any idea about what's going on, please share, I'll be forever grateful! Thanks, L-A

2 1

Re: [galaxy-dev] Interested in speaking with other
by Dorset, Daniel C 30 Apr '12

30 Apr '12

Hi Ann, Count me in. Great idea! -Dan

1 0

Stalled upload jobs under "Admin", "Manage jobs"
by Peter Cock 30 Apr '12

30 Apr '12

Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users. e.g. Job ID User Last Update Tool State Command Line Job Runner PID/Cluster ID 2352 xxxx 21 hours ago upload1 upload None None None ... 2339 yyyy 19 hours ago upload1 upload None None None The job numbers are consecutive (2339 to 2352) and reflect a problem for a couple of hours yesterday morning. I believe this was due to the underlying file system being unmounted (without restarting Galaxy), and at the time restarting Galaxy fixed uploading files. Test jobs since then have completed normally - but these zombie jobs remain. Using the "Stop jobs" option does not clear these dead upload jobs. Restarting the Galaxy server does not clear them either. This is our production server and was running galaxy-dist, changeset 5743:720455407d1c - which I have now updated to the current release, 6621:26920e20157f - which makes no difference to these stalled jobs. Does anyone have any insight into what might be wrong, and how to get rid of these zombie tasks? Thanks, Peter

3 5

Using REMOTE_USER with nginx
by Peter van Heusden 30 Apr '12

30 Apr '12

Hey there The instructions on using REMOTE_USER with nginx are still a bit vague in the wiki, so let me share how I got this working with nginx's http_auth_pam module and our local Kerberos setup. Really simple actually: First, I created a pam.d entry for nginx, as follows: auth [success=1 default=ignore] pam_krb5.so minimum_uid=1000 ignore_k5login auth requisite pam_deny.so auth required pam_permit.so That can of course be adapted for your authentication scheme of choice. The, after recompiling nginx to add the module (I actually used the source from the Ubuntu .deb and installed from this customised .deb), I added: auth_pam "SANBI Galaxy (dev)"; auth_pam_service_name "nginx"; proxy_set_header REMOTE_USER $remote_user; That auth_pam_service_name must be the name of the file you add in /etc/pam.d. So the complete location clause is now: location / { auth_pam "SANBI Galaxy (dev)"; auth_pam_service_name "nginx"; proxy_set_header REMOTE_USER $remote_user; proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-URL-SCHEME https; } Finally, set: use_remote_user = True remote_user_maildomain = <YOUR DOMAIN NAME> And restart nginx and galaxy, and you're done. Of course, since you're using Basic authentication, you should make sure that you are using ssl too. If this all looks ok, maybe someone can update the wiki? Peter

2 1

What does this line of code mean
by Ciara Ledero 30 Apr '12

30 Apr '12

Hi all, Can I ask what this line of code does? job_wrapper.finish( stdout, stderr ) It's the one from the local.py. Cheers, CL

2 1

Error running MACS in Galaxy
by Waldron, Michael H 30 Apr '12

30 Apr '12

We get the following error when trying to run MACS in our local installation of Galaxy: INFO @ Mon, 30 Apr 2012 09:02:14: # ARGUMENTS LIST: # name = MACS_in_Galaxy # ChIP-seq file = /proj/galaxy/000/dataset_552.dat # control file = None # effective genome size = 2.70e+09 # tag size = 36 # band width = 300 # model fold = 32 # pvalue cutoff = 1.00e-05 # Ranges for calculating regional lambda are : 1000,5000,10000 INFO @ Mon, 30 Apr 2012 09:02:14: #1 read tag files... INFO @ Mon, 30 Apr 2012 09:02:14: #1 read treatment tags... Traceback (most recent call last): File "/nas02/apps/python-2.6.5/bin/macs", line 238, in <module> main() File "/nas02/apps/python-2.6.5/bin/macs", line 53, in main (treat, control) = _load_tag_files_options (options) File "/nas02/apps/python-2.6.5/bin/macs", line 211, in _load_tag_files_options treat = load_tag_file(options.tfile,options.format) File "/nas02/apps/python-2.6.5/bin/macs", line 231, in load_tag_file raise Exception("Format \"%s\" cannot be recognized!" % (format)) Exception: Format "SAM" cannot be recognized! Does anyone know what causes this? We are using MACS-1.3.2. Mike Waldron

1 0

Merging BLAST database support into Galaxy?
by Peter Cock 30 Apr '12

30 Apr '12

Hi Edward, We're now running BLAST+ searches on our local Galaxy via our cluster, and some of the cluster nodes have relatively small amounts of RAM. This means I've become more aware of limitations in the NCBI BLAST+ tools' support for using a subject FASTA file (instead of making a local BLAST database), which turns out to be surprisingly RAM hungry. The logical step is to allow users to build a BLAST database as a new datatype in Galaxy - which is what you (Edward) did some time ago as a fork, later posted to the Galaxy Tool Shed. Edward - are you happy for me to merge your work into the main wrappers? I mentioned idea this a couple of months ago: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-February/008544.html Thanks, Peter

2 6

Re: [galaxy-dev] Interested in speaking with other institutions deploying
by Matloob Khushi 30 Apr '12

30 Apr '12

Hi Ann I am running an instance at The University of Sydney GMT+10 and would like to be part of any such online talk. Regards, Matloob

1 0

Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?
by Edward Hills 29 Apr '12

29 Apr '12

Hi Ann, We are in the process of doing the same thing here at the University of Otago (New Zealand). If we are able to sort something via the timezone difference (we are GMT+12, first to see the sun!) we would very much be interested. Cheers, Ed Hi everyone - > > Here at the University of Iowa we are working on deploying Galaxy > locally for campus wide access. I am interested in forming a community > of other institutions trying to deploy Galaxy locally and mange/operate > it on a broad level. Is anyone else? If there is enough interest, > possibly we could have a community conference call every other month to > have an open discussion on how we are all deploying galaxy, > customizations we are making, problems we are encountering, bugs, and > any add-on operations management for galaxy being developed, etc. > > Would love to hear from others operating Galaxy or in process of > standing up a local deployment. > > Thanks! > > Ann Black-Ziegelbein

2 1