August 2011 - galaxy-dev - lists.galaxyproject.org

Galaxy Cluster Configuration
by Christoph Ruttkies 22 Aug '11

22 Aug '11

Dear Galaxy-Dev Team, I have a problem concerning the integration of galaxy into a cluster environment. Galaxy is installed on a server that has access to the local cluster network and is configured as a submit host for the Sun Grid Engine running on the cluster nodes. So it is configured to use drmaa to submit its jobs. Everytime submitting a job from the galaxy webfrontent the qmaster daemon on the sge-admin host dies with the message !!!!!!!!!! got NULL element for SME_message_list !!!!!!!!!! in the logs. The SGE version is 6.2. Submitting jobs from the command-line with qsub is working without any problems. Do you have any idea? Best regards, Christoph

2 1

When writing wrapper do not use shutil.copy()
by Chorny, Ilya 19 Aug '11

19 Aug '11

If possible when writing wrappers do not use shutil.copy() when copying a file to a dataset_(job_id).dat file. shutil.copy() tries to change the permission of the copied to file. Better to use shutil.copyfile(). Ilya Chorny Ph.D. Bioinformatics - Intern ichorny(a)illumina.com<mailto:ichorny@illumina.com> 858-202-4582

1 0

trackster problem with IE8 and IE7 when using "June 23, 2011" release
by Hans-Rudolf Hotz 19 Aug '11

19 Aug '11

Hi I am sorry to be a pain in the neck, but as part of our quarterly upgrade procedure, I am stumbling across the next problem: Can anybody confirm, that the visualization does not work with the "June 23, 2011" release (changeset: 5743:720455407d1c) when using IE8 or IE7 on windows XP? (it works with chrome and firefox6 on windows XP, and no problems with mac and linux) And like the previous problems, is there a quick fix available? Thank you very much for your help Regards, Hans

3 2

Problems with Galaxy on a mapped drive
by Peter Cock 19 Aug '11

19 Aug '11

Hi all, In my recent email I mentioned problems with our setup and mapped drives. I am running a test Galaxy on a server under a CIFS mapped drive. If I map the drive with noperms then things seem to work with submitting jobs to the cluster etc, but that doesn't seem secure at all. Mounting with strict permissions seems to cause various network latency related problems in Galaxy though. Specifically during loading the converters and history export tool, Galaxy creates a temporary XML file which it then tries to parse. I was able to resolve this by switching from tempfile.TemporaryFile to tempfile.mkstemp and adding a 1s sleep, but it isn't very elegant. Couldn't you use a StringIO handle instead? Later during start up there were two errors with a similar issue - Galaxy creates a temp folder then immediately tries to write a tar ball or zip file to it. Again, adding a 1 second sleep after creating the directory before using it seems to work. See lib/galaxy/web/controllers/dataset.py After that Galaxy started, but still gives problems - like the issue reported here which Galaxy handled badly (see patch): http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-July/006213.html Here again, inserting a one second sleep between writing the cluster script file and setting its permissions made it work. If those are the only issues, that can be dealt with. But are there likely to be lots more similar problems of this nature later on? That is my worry. How are most people setting up mapped drives for Galaxy with a cluster? Thanks, Peter

3 3

job runner issue
by shashi shekhar 19 Aug '11

19 Aug '11

Hi In my galaxy instance, whatever jobs i am submitting it goes into queued state. If I restart the server then the previous submitted jobs state changes to running. but the newly submitted jobs again goes to queued state. I am at a loss to understand this behaviour of galaxy and unable to debug it. The job submission uses a customized runner. How is it actually goes into the queued state automatically when all the workers thread are free? Does galaxy_session table is_valid attribute makes jobs state true? Or what all places in tables the queued states are getting stored. I can only see that the jobs table state attribute only stores the state. The server logs points error here: galaxy.jobs ERROR 2011-07-29 11:01:28,098 failure running job 2243 Traceback (most recent call last): File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 202, in __monitor_step self.dispatcher.put( JobWrapper( job, self ) ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 856, in put self.job_runners[runner_name].put( job_wrapper ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/runners/gw.py", line 375, in put job_wrapper.change_state( model.Job.states.QUEUED ) File "/home/gwadmin/galaxy-central/lib/galaxy/jobs/__init__.py", line 437, in change_state self.sa_session.flush() Regards Karuna

2 1

Could anyone tell me why the result file in directary /galaxy-dist /database/files/000 do not return to the web page
by liyanhui607 19 Aug '11

19 Aug '11

Dear Sir, We have a program writen in perl. It can run nomally in linux environment with a file saved as result. After added it to galaxy system, we found that it can produce the right result file in directary "/var/we/galaxy-dist/database/files/000" , but the result file did not return to the web page where the state is always "Job is currently running". Could anyone tell me what is wrong ? why it does not work? The bagging-SVM.xml and the bagging-SVM.pl are attached. Thank you very much! Best Wishes! Yan-Hui Li

2 1

Galaxy: self-cleaning on file_path
by Sanka, Ravi 19 Aug '11

19 Aug '11

Hello, We have recently put a local installation of Galaxy on our network and connected it to a MYSQL database. In order to have more space available for uploaded and created files, we also changed the values of file_path and new_file_path in the config file. The file_path field was changed to a larger, communal location. The new_file_path was changed to a directory called "tmp" in that same communal location. This location, however, undergoes self-cleaning; deleting any files that reach 7 days old (and any directories that have no files less than 7 days old). This would include any datasets uploaded or created by Galaxy. Can the Galaxy framework and schema handle this occurrence? Or will it cause errors to occur? ----------------------------------------------- Ravi Sanka ICS - Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 -----------------------------------------------

2 3

database problem
by shashi shekhar 19 Aug '11

19 Aug '11

Hi ALL, I want to delete some data from history_dataset_association. Will it effect any other tables.and I want to delete records for last 3 days by sqlite query manually.How may tables will be effected by this and provide me tables name .How can i delete the records. Regards shashi shekhar

2 1

Cluster setup - shared temporary directory
by Peter Cock 19 Aug '11

19 Aug '11

Hi all, I'm reading http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster Could someone expand a little on this section please: > Create a shared temporary directory > > Some tools make use of temporary files created on the server, > but accessed on the nodes. For this, you'll need to make a > directory (galaxy_dist/database/tmp by default) ... I presume this is talking about the universe_wsgi.ini setting new_file_path = database/tmp (if so, could that be explicit)? I would like to know more about this from the tool author point of view. Could you at least give one example of a tool that uses this temporary folder? As a tool author I am unclear what the purpose is (and it would be a shock if I accidentally use this mapped folder instead of the local temp drive of a node). Thanks, Peter

4 6

Extract Genomic DNA insisting on build for GFF3 file
by Peter Cock 19 Aug '11

19 Aug '11

Hi all, Is this a bug, or have I misunderstood something? 1. Goto http://usegalaxy.org (or a local Galaxy running galaxy-dist) 2. Import this genomic FASTA file, ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Nanoarchaeum_equitans_Kin4_M_uid580… 3. Import this GFF3 file, ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Nanoarchaeum_equitans_Kin4_M_uid580… 4. Goto "Fetch Sequences", "Extract Genomic DNA" 5. Choose fetch sequence for intervals in NC_005213.gff, interpret yes, source history, NC_005213.ffn, output FASTA 6. Click execute. Expected result: Tool runs Actual result: Red error against the gff file, Unspecified genome build, click the pencil icon in the history item to set the genome build The fact I'm using a FASTA file from my history should mean the genome build is irrelevant as that only applies to "locally cached genomes" (right?). Thanks, Peter

2 18