concurrent job run parameter problem
Scenario: Running two velvet jobs with the same parameters except for the "Conditional (contig_lgth) either 0 or 1 (with minimum contig length = 500) If the scheduler runs these jobs separately they produce expected results. However, if they happen to start running simultaneously then one of the jobs looses its parameters and both jobs produce identical results corresponding to the contig_lgth not set. Galaxy config is load-balanced with jobs tracked in the database and metadata set externally in postgresql. the .sh files that are generated appear to be correct and job directories are created appropriately. Whatever fouls up the resuts seems to happen on the rear end of the job I guess. I wonder if anyone ran into this issue? It should be a common scenario where people try out different config parameters for the same tool running on the same dataset, so I wonder if it's more likely to be my galaxy configuration problem then an actual galaxy bug. My job runner config: http://pastebin.com/vMWDHAQm Thanks, Alex
On Dec 14, 2011, at 1:51 PM, Oleksandr Moskalenko wrote:
Scenario: Running two velvet jobs with the same parameters except for the
"Conditional (contig_lgth) either 0 or
1 (with minimum contig length = 500)
If the scheduler runs these jobs separately they produce expected results. However, if they happen to start running simultaneously then one of the jobs looses its parameters and both jobs produce identical results corresponding to the contig_lgth not set.
Galaxy config is load-balanced with jobs tracked in the database and metadata set externally in postgresql. the .sh files that are generated appear to be correct and job directories are created appropriately. Whatever fouls up the resuts seems to happen on the rear end of the job I guess. I wonder if anyone ran into this issue? It should be a common scenario where people try out different config parameters for the same tool running on the same dataset, so I wonder if it's more likely to be my galaxy configuration problem then an actual galaxy bug.
My job runner config: http://pastebin.com/vMWDHAQm
Hi Alex, Are you only starting one job runner process? When tracking jobs in the database and using recovery, it is only safe to start one. If the answer is yes, can you check your logs and the job table in the database to determine whether the two jobs have the same ID or different IDs? Thanks, --nate
Thanks,
Alex
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Nate Coraor -
Oleksandr Moskalenko