December 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy users on virtual machine
by Ganote, Carrie L 18 Dec '12

18 Dec '12

Hi all, Right now I have drmaa running with Torque. I have a virtual machine with Galaxy installed on it, a filesystem mounted on the virtual machine that holds the dataset info, and a cluster that runs big jobs and also has access to the mounted filesystem. When running Galaxy as a real user, I get an error because the real user has no pwd listing in the virtual machine. In this entry <http://dev.list.galaxyproject.org/Different-User-with-Torque-td4655833.html…> there is mention of galaxy_user.username; where is that and how is it populated? I use kerberos to authenticate through apache. Any clues appreciated! Here's the dump: galaxy.jobs ERROR 2012-12-18 11:29:01,871 (989) Failed to change ownership of /Filesystem/job_working_directory/000/989, making world-writable instead Traceback (most recent call last): File "/Prefix/galaxy-dist/lib/galaxy/jobs/__init__.py", line 893, in change_ownership_for_run self._change_ownership( self.user_system_pwent[0], str( self.user_system_pwent[3] ) ) TypeError: 'NoneType' object is unsubscriptable galaxy.jobs.runners.drmaa ERROR 2012-12-18 11:29:01,928 Uncaught exception queueing job Traceback (most recent call last): File "/Prefix/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 145, in run_next self.queue_job( obj ) File "/Prefix/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 237, in queue_job log.debug( '(%s) submitting with credentials: %s [uid: %s]' % ( galaxy_id_tag, job_wrapper.user_system_pwent[0], job_wrapper.user_system_pwent[2] ) ) TypeError: 'NoneType' object is unsubscriptable Carrie Ganote

2 1

Parallelism tags and Best practices.
by James 17 Dec '12

17 Dec '12

Hi All, I was wondering how to add parrelism tags based on best practices recommended by the tool providers. E.g with IMPUTE at the moment split the file into region size that is recommended by the tool providers and then run each section sequentially. Would it be possibly to add it into the job splitters so that it splits based on megabase distance and not just merely breaking the file into a equal amount of lines. Cheers James.

1 0

Toolshed install 404
by Kyle Ellrott 17 Dec '12

17 Dec '12

Is anybody else getting 404 errors during toolshed installs? My instance is trying to access " http://toolshed.g2.bx.psu.edu/repository/get_readme_files?name=synapse_inte…" and gets a 404. Kyle

2 1

bai file missing in history export?!
by Bossers, Alex 17 Dec '12

17 Dec '12

Dear all, Are we the only ones that are missing bam index files (the .bai) when exporting a history that contains for instance bowtie mapped reads. In the original history we can save the bam and bai files but after export to a file (tgz) and later import into another (or the same) galaxy instance, the bai files cannot be accessed/saved anymore.... Alex

2 2

tophat: option --transcriptome-mismatches not recognized
by greg 15 Dec '12

15 Dec '12

I'm running tophat2 like this on my newly installed galaxy: http://snag.gy/OfLND.jpg (parameters not shown are just the defaults) And I'm getting this error: Error in tophat: tophat: option --transcriptome-mismatches not recognized for detailed help see http://tophat.cbcb.umd.edu/manual.html I don't understand why the option is offered in the galaxy UI if it's not available, or is there some kind of bug, or wrong version of tophat2? I'm on TopHat v2.0.6. Thanks, Greg

2 4

"default_cluster_job_runner" setting changed - no longer getting job output from cluster
by greg 14 Dec '12

14 Dec '12

Hi guys, I'm really stuck on this one. In my galaxy config file I changed the setting it uses to launch jobs from: default_cluster_job_runner = drmaa://-V/ (which worked fine) To default_cluster_job_runner = drmaa://-V -pe batch 8/ Now whenever I run a job I see a message like this: "Job output not returned from cluster" (Is there any other information I can use to debug this?) I see the information below in the log file if any of that is useful: galaxy.jobs.manager DEBUG 2012-12-14 13:27:36,961 (104) Job assigned to handler 'main' galaxy.jobs DEBUG 2012-12-14 13:27:37,584 (104) Working directory for job is: /home/galaxy/job_working_directory/000/104 galaxy.jobs.handler DEBUG 2012-12-14 13:27:37,586 dispatching job 104 to drmaa runner galaxy.jobs.handler INFO 2012-12-14 13:27:37,772 (104) Job dispatched galaxy.jobs.runners DEBUG 2012-12-14 13:27:38,078 Copying /home/galaxy/job_working_directory/000/104/global_model.txt to /home/galaxy/000/dataset_155.dat as directed by from_work_dir galaxy.jobs.runners DEBUG 2012-12-14 13:27:38,111 Copying /home/galaxy/job_working_directory/000/104/genes.fpkm_tracking to /home/galaxy/000/dataset_152.dat as directed by from_work_dir galaxy.jobs.runners DEBUG 2012-12-14 13:27:38,135 Copying /home/galaxy/job_working_directory/000/104/isoforms.fpkm_tracking to /home/galaxy/000/dataset_153.dat as directed by from_work_dir galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:38,819 (104) submitting file /home/galaxy/pbs/galaxy_104.sh galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:38,820 (104) command is: python /misc/local/galaxy/galaxy-dist/tools/ngs_rna/cufflinks_wrapper.py --input=/home/galaxy/000/dataset_93.dat --assembled-isoforms-output=/home/galaxy/000/dataset_154.dat --num-threads="4" -I 300000 -F 0.1 -j 0.15; cp /home/galaxy/job_working_directory/000/104/global_model.txt /home/galaxy/000/dataset_155.dat; cp /home/galaxy/job_working_directory/000/104/genes.fpkm_tracking /home/galaxy/000/dataset_152.dat; cp /home/galaxy/job_working_directory/000/104/isoforms.fpkm_tracking /home/galaxy/000/dataset_153.dat; cd /misc/local/galaxy/galaxy-dist; /misc/local/galaxy/galaxy-dist/set_metadata.sh /home/galaxy /home/galaxy/job_working_directory/000/104 . /misc/local/galaxy/galaxy-dist/universe_wsgi.ini /home/galaxy/job_working_directory/tmppjMj2j /home/galaxy/job_working_directory/000/104/galaxy.json /home/galaxy/job_working_directory/000/104/metadata_in_HistoryDatasetAssociation_176_zK8nkf,/home/galaxy/job_working_directory/000/104/metadata_kwds_HistoryDatasetAssociation_176_Dkq7kA,/home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_176_jR0e_Z,/home/galaxy/job_working_directory/000/104/metadata_results_HistoryDatasetAssociation_176_jl8W2w,,/home/galaxy/job_working_directory/000/104/metadata_override_HistoryDatasetAssociation_176_M6ctS5 /home/galaxy/job_working_directory/000/104/metadata_in_HistoryDatasetAssociation_177_Q1vGSc,/home/galaxy/job_working_directory/000/104/metadata_kwds_HistoryDatasetAssociation_177_MG96_w,/home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_177_NWTSVQ,/home/galaxy/job_working_directory/000/104/metadata_results_HistoryDatasetAssociation_177_iTNC5T,,/home/galaxy/job_working_directory/000/104/metadata_override_HistoryDatasetAssociation_177_MO0SIc /home/galaxy/job_working_directory/000/104/metadata_in_HistoryDatasetAssociation_174_HcoiD4,/home/galaxy/job_working_directory/000/104/metadata_kwds_HistoryDatasetAssociation_174_4j5SRC,/home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_174_irlquD,/home/galaxy/job_working_directory/000/104/metadata_results_HistoryDatasetAssociation_174_jfK1zk,,/home/galaxy/job_working_directory/000/104/metadata_override_HistoryDatasetAssociation_174_YQzY4e /home/galaxy/job_working_directory/000/104/metadata_in_HistoryDatasetAssociation_175_rjQ2Gp,/home/galaxy/job_working_directory/000/104/metadata_kwds_HistoryDatasetAssociation_175_HMeDGS,/home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_175_7sjyvl,/home/galaxy/job_working_directory/000/104/metadata_results_HistoryDatasetAssociation_175_cJ_Sw6,,/home/galaxy/job_working_directory/000/104/metadata_override_HistoryDatasetAssociation_175_JY4ngx galaxy.jobs.runners.drmaa INFO 2012-12-14 13:27:38,884 (104) queued as 4076946 galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:39,822 (104/4076946) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:42,830 (104/4076946) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:44,065 (104/4076946) state change: job finished normally galaxy.jobs.runners.drmaa DEBUG 2012-12-14 13:27:44,111 Job output not returned from cluster galaxy.jobs DEBUG 2012-12-14 13:27:44,147 Tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs DEBUG 2012-12-14 13:27:44,187 setting dataset state to ERROR galaxy.jobs DEBUG 2012-12-14 13:27:44,206 setting dataset state to ERROR galaxy.jobs DEBUG 2012-12-14 13:27:44,226 setting dataset state to ERROR galaxy.jobs DEBUG 2012-12-14 13:27:44,250 setting dataset state to ERROR galaxy.jobs DEBUG 2012-12-14 13:27:44,432 job 104 ended galaxy.datatypes.metadata DEBUG 2012-12-14 13:27:44,432 Cleaning up external metadata files galaxy.datatypes.metadata DEBUG 2012-12-14 13:27:44,452 Failed to cleanup MetadataTempFile temp files from /home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_176_jR0e_Z: No JSON object could be decoded: line 1 column 0 (char 0) galaxy.datatypes.metadata DEBUG 2012-12-14 13:27:44,475 Failed to cleanup MetadataTempFile temp files from /home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_177_NWTSVQ: No JSON object could be decoded: line 1 column 0 (char 0) galaxy.datatypes.metadata DEBUG 2012-12-14 13:27:44,491 Failed to cleanup MetadataTempFile temp files from /home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_175_7sjyvl: No JSON object could be decoded: line 1 column 0 (char 0) galaxy.datatypes.metadata DEBUG 2012-12-14 13:27:44,510 Failed to cleanup MetadataTempFile temp files from /home/galaxy/job_working_directory/000/104/metadata_out_HistoryDatasetAssociation_174_irlquD: No JSON object could be decoded: line 1 column 0 (char 0) galaxy.objectstore CRITICAL 2012-12-14 13:27:44,544 /home/galaxy/job_working_directory/000/104 delete error [Errno 39] Directory not empty: '/home/galaxy/job_working_directory/000/104' Thanks, -Greg

1 1

cannot "Delete Permanently"
by Fenglou Mao 14 Dec '12

14 Dec '12

I installed a local galaxy server, did some calculation. Now I want to clean datasets for some space. I tried "Delete Permanently" on both datasets and histories, none of them worked. The datasets and histories will be marked as "deleted", but it will not be really deleted even I am using "Delete Permanently". They will not be really removed, and space are not released. Sometimes it will give me a message "this feature is not enabled", sometimes it won't. Any one know how to enable "Delete Permanently" feature? Thanks, Fenglou

2 2

How are jobs counted in multi-threaded tools?
by Carlos Borroto 14 Dec '12

14 Dec '12

Hi all, I'm using 'user_job_limit = 8' and I will like to know if jobs for multi-threaded tools are counted based on some kind of multiple. I see for example a user can only run 2 ncbi_blast_plus jobs with my current configuration. Is this because the default numbers of threads for these jobs is 4? What happen if I modify the xml file and change the numbers of threads? Thanks, Carlos

1 0

Data library and symbolic links
by MONJEAUD 14 Dec '12

14 Dec '12

Hi all, I created data libraries using options "Upload files from filesystem paths" and "_Link to files without copying into Galaxy_". It worked but I use symbolic links because my data are frequently updated The problem is that Galaxy takes the real link (inside "external_filename" column in "dataset" table) and doesn't find the files when it is deleted. For example, I uploaded a dataset for a data library without copying : /databank/uniprot/current/fasta/all.fasta -> /databank/uniprot/01_12_2012/fasta/all.fasta The real path is put in the database and it's a problem when it is deleted. Is there a way for galaxy to take the symbolic link directly in the database instead of the real path. Thanks in advance! Cheers, -- Cyril Monjeaud Equipe Symbiose / Plate-forme GenOuest Bureau D156 IRISA-INRIA, Campus de Beaulieu 35042 Rennes cedex, France Tél: +33 (0) 2 99 84 74 17

3 3

Apologies and some more questions
by Philipe Moncuquet 14 Dec '12

14 Dec '12

Hi, I have been sending twice the same message to the mailing list because it didn't appears on my gmail account and I thought it didn't went true. Clearly it appears on the List archive, I checked my spam and it's not there. Does it have something to do with the fact that I am the author of the message, I guess so. Once again sorry for spamming. I still have one more question, is there anyway to edit the annotation of a workflow once you have created it. You can edit name and contents but I don't see how to edit the annotation. When you publish and realized there is a typo in the annotation, there is no other way than deleting the workflow and start all over. Did I missed something ? Philippe

2 1