July 2013 - galaxy-dev - lists.galaxyproject.org

job_conf.xml questions
by Ido Tamir 01 Aug '13

01 Aug '13

Hi, I work myself through the job_conf.xml and have a question: a) In your advanced example you have: <destination id="remote_cluster" runner="drmaa" tags="longjobs"/> <destination id="real_user_cluster" runner="drmaa">  <param id="galaxy_external_runjob_script">scripts/drmaa_external_runner.py</param> <param id="galaxy_external_killjob_script">scripts/drmaa_external_killer.py</param> <param id="galaxy_external_chown_script">scripts/external_chown_script.py</param> </destination> Does this mean that remote_cluster jobs can not be killed unless I add the 3 scripts to this destination? Or does this have to do with the "run jobs as real user" that I don't need currently. I am using the galaxy user for all jobs for the moment? To phrase this differently: Do I need these three scripts? b) Can I set a wall time limit for only for my local runners? like: <destination id="local" runner="local"/> <param id="Resource_List">walltime=72:00:00</param> </destination> When I set it in limits, will it work for local runners and then I have to override it in every destination? thank you very much, ido

2 1

Velvet optimizer problems : Urgent help.
by Perez, Ricardo 31 Jul '13

31 Jul '13

Dear all, When trying to run velvet optimizer in galaxy, all the files we get return the following: "No peek" When looking at the output of galaxy, we get the following: galaxy.jobs.runners.local DEBUG 2013-07-31 09:31:46,829 execution finished: perl /usr/local/galaxy/bioinfosoft/shed_tools/toolshed.g2.bx.psu.edu/repos/simon-gladman/velvet_optimiser/43c89d82a7d3/velvet_optimiser/velvet_optimiser_wrapper_vlsci.pl '19' '23' '2' '0' 'short' 'False' '<galaxy.datatypes.sequence.FastqSanger object at 0x6e664d0>' '/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2067.dat''not_shortMP''other:''no_amos''not_verbose' '/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2665.dat''/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2666.dat' '/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2667.dat''/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2663.dat' '/usr/local/galaxy/galaxy-dist/database/files/002/dataset_2668.dat' > /usr/local/galaxy/galaxy-dist/database/files/002/dataset_2664.dat galaxy.jobs.runners DEBUG 2013-07-31 09:31:47,027 executing external set_meta script for job 1983: /usr/local/galaxy/galaxy-dist/set_metadata.sh ./database/files /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983 . /usr/local/galaxy/galaxy-dist/universe_wsgi.ini /usr/local/galaxy/galaxy-dist/database/tmp/tmp3460Jz /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/galaxy.json /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3117_enOT5T,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3117_vy3tDq,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3117_IFmVK8,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3117_0j3kza,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3117_zwMKHf /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3116_YppZ1L,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3116_Sniis0,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3116_l9RPFx,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3116_It8rUk,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3116_AwvXlS /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3115_TOhpBs,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3115_wUomyo,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3115_nsUrJl,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3115_DAmGEt,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3115_hFGdc0 /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3114_59uEii,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3114_UcezIt,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3114_YmOlHD,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3114_owVBfP,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3114_O3UH82 /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3113_vMrQrs,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3113_YhwIGZ,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3113_pAG6wP,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3113_9ImAZ6,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3113_uI5cbM /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_in_HistoryDatasetAssociation_3112_NwB9j6,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_kwds_HistoryDatasetAssociation_3112_LK2cWo,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_out_HistoryDatasetAssociation_3112__FDnXW,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_results_HistoryDatasetAssociation_3112_mZQULN,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1983/metadata_override_HistoryDatasetAssociation_3112_k3SWCq However when running velvet optimizer from the command line (the first command from above) we get the correct output just fine. We have re-installed the tools and velvet. It seems like galaxy is passing the command just fine, but we are getting nothing back to galaxy. Anyone has ideas on how to fix this problem? Thank you very much for your time and support, --Ricardo Perez

2 1

Problems with sharing datasets
by Ricardo Perez 31 Jul '13

31 Jul '13

Dear all, When trying to share a dataset in galaxy among users, usually this sharing fails. Usually when we go to "Dataset Securities" and try to change the permissions of the datasets the sharing fails. Instead, it keeps the original configuration and then it will open another galaxy page inside the galaxy browser that we already have. Is this due to misconfiguration that I may have? Thank you for your time, --Ricardo Perez

1 0

Build the Galaxy under Cluster
by shenwiyn 31 Jul '13

31 Jul '13

Hi everyone, I want to install my local Galaxy under the Cluster with TORQUE.I have a question about how they work together successfully.In addition,after my local Galaxy and TORQUE install successfully,anything else should I do to the Cluster Galaxy?Thank you very much. shenwiyn

2 3

Galaxy Proteomics mailing list
by Ira Cooke 31 Jul '13

31 Jul '13

Dear All, The galaxy community now has a mailing list dedicated to proteomics. If you're using galaxy to do proteomics, or if you are thinking of doing so then this list if for you. In case you missed it, there is also a proteomics wiki page at http://wiki.galaxyproject.org/Proteomics To subscribe to the galaxy-proteomics list please add your email address to the list here: http://lists.bx.psu.edu/listinfo/galaxy-proteomics See http://wiki.g2.bx.psu.edu/Mailing%20Lists for more information on this and other Galaxy mailing lists, including links to searchable archives of all Galaxy lists. Many thanks to Dave C. and the Galaxy team for setting this up. Thanks Ira

1 0

Running RabbitMQ listener requires a path update
by Ganote, Carrie L 30 Jul '13

30 Jul '13

Hi list, I ran into the following problem: $ sh run_galaxy_listener.sh Traceback (most recent call last): File "scripts/galaxy_messaging/server/amqp_consumer.py", line 33, in <module> from galaxy.web.api.requests import RequestsAPIController ImportError: No module named api.requests Changed this line in scripts/galaxy_messaging/server/amqp_consumer.py: from galaxy.webapps.galaxy.api.requests import RequestsAPIController Just an artifact from the restructuring of Galaxy. Hope this helps! Sincerely, Carrie Ganote

1 0

Local: BWA 0.5.9. SA Coordinate/Segmentation fault
by Moritz Juchler 30 Jul '13

30 Jul '13

Hey, 1 I installed Galaxy locally. 2 I also have BWA installed and then used the 3 shed repository to get the bwa wrapper. 4 I have the reference genome now I'm stuck with the calculate SA coordinate. Segmenation fault. Google tells me that it could be a problem with -space (I have 4GB left) -the version I use. (I use 0.5.9 and did a succesful alignment with 0.7.5a, but the tool shed says 0.5.9, hence *could I just change to the current version* (without Galaxy knowing it), with which I did a succesful alignment?) Best Moritz

1 0

Re: [galaxy-dev] CloudMan Error
by Dannon Baker 30 Jul '13

30 Jul '13

The warning message printed by the first ssh attempt indicates that you have no cloudman_key_pair.pem in the directory you executed the ssh command from. Find this file (or create a new one) and you'll be able to ssh. Good luck, and please keep threads on the list instead of emailing directly. Thanks! -Dannon On Mon, Jul 22, 2013 at 1:36 PM, <denizere(a)gmail.com> wrote: > Hi- the share string appears to be loading, but I am unable to ssh into my > instance using the provided command, or anything like it. Instead I get the > following: > > denizerezyilmaz$ ssh -i cloudman_key_pair.pem > ubuntu(a)ec2-54-242-61-164.compute-1.amazonaws.com > Warning: Identity file cloudman_key_pair.pem not accessible: No such file > or directory. > > OR: > > denizerezyilmaz$ ssh ec2-54-242-61-164.compute-1.amazonaws.com > Permission denied (publickey). > > What am I missing? Thank you, Deniz >

2 9

Re: [galaxy-dev] Number of outputs = number of inputs
by Shafer, Christina 30 Jul '13

30 Jul '13

My question is related to this post <http://dev.list.galaxyproject.org/Number-of-outputs-number-of-inputs-td4656…> in the archive. As recommended, I referred to http://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_… for some guidance on how to achieve multiple output files, specifically one set of six files per number of times the <repeat> tag is called. The example provided isn't that informative, however, as it doesn't show what the example_tool.sh command script is doing so I can make my perl wrapper script do the same thing. If I have six files total, do I only specify one in the <outputs> tag and then have the remaining 5 named according to the scheme in the perl wrapper? Or is it Galaxy that does the naming of the remaining files automatically? Is there a perl equivalent to the "$__new_file_path__" variable that I can use, or is this the literal resolved file path (e.g. "/opt/galaxy/... etc)? Right now, upon execution of my tool, if a user has two entries (so number of times <repeat> has executed is twice), my perl wrapper is called twice (good), but on the second call, Galaxy uses the same output filenames as the first run such that the first set of output files are all overwritten by the second execution (bad). How do I make sure that each <repeat> iteration results in a unique set of output files? Thanks for your help! Christina Shafer, Ph.D Regenerative Biology Laboratory Morgridge Institute for Research

2 1

job_conf.xml about running Galaxy tools on a cluster.
by shenwiyn 30 Jul '13

30 Jul '13

Hi everyone， I am a newer in Galaxy. I installed my Galaxy with Torque2.5.0 . I used my job_conf.xml for a test ,my Galaxy seems to work with TORQUE by the interface of pbs_modoule ( _pbs.so) successfully. In my test job_conf.xml , it seems that we should use <tools>collection of <tool id="bwa_wrapper" destination="multicorejobs4"/>to specify bwa to run as multicorejobs4. But I do not sure wheather all the tools in Galaxy we should specify their destination (such as, regularjobs,longjobs), or all the tools would work with a default destination(<destinations default="regularjobs">).,unless we specify them. Could anyone give me some help please? This is my test job_conf.xml: <?xml version="1.0"?> <job_conf> <plugins workers="4">  <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="2"/> <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner" workers="8"/> </plugins> <handlers default="batch">  <handler id="node01" tags="batch"/> <handler id="node02" tags="batch"/>  </handlers> <destinations default="regularjobs">  <destination id="local" runner="local"/> <destination id="regularjobs" runner="pbs" tags="cluster">  </destination> <destination id="longjobs" runner="pbs" tags="cluster,long_jobs">  </destination> <destination id="shortjobs" runner="pbs" tags="cluster,short_jobs">  </destination> <destination id="multicorejobs" runner="pbs" tags="cluster,multicore_jobs">  </destination> <destination id="real_user_cluster" runner="pbs"> <param id="galaxy_external_runjob_script">scripts/drmaa_external_runner.py</param> <param id="galaxy_external_killjob_script">scripts/drmaa_external_killer.py</param> <param id="galaxy_external_chown_script">scripts/external_chown_script.py</param> </destination> <destination id="dynamic" runner="dynamic">  <param id="type">python</param> <param id="function">interactiveOrCluster</param> </destination> </destinations> <tools>  <tool id="bwa_wrapper" destination="multicorejobs"/> </tools> <limits>  <limit type="registered_user_concurrent_jobs">500</limit> <limit type="unregistered_user_concurrent_jobs">1</limit> <limit type="concurrent_jobs" id="local">1</limit> <limit type="concurrent_jobs" tag="cluster">200</limit> <limit type="concurrent_jobs" tag="long_jobs">200</limit> <limit type="concurrent_jobs" tag="short_jobs">200</limit> <limit type="concurrent_jobs" tag="multicore_jobs">100</limit> </limits> </job_conf> shenwiyn

1 0