October 2013 - galaxy-dev - lists.galaxyproject.org

Running jobs as real user and extra_file_path
by Louise-Amélie Schmitt 28 Apr '17

28 Apr '17

Hi everyone, I just wanted to ask how the extra_file_path is handled in case of job running as the real user since the file_path is only writable by the galaxy user. Any clue? Thanks, L-A

3 3

error loading files into galaxy
by Hakeem Almabrazi 04 Mar '16

04 Mar '16

Hi, I started getting the following error whenever I try to load a file into Galaxy local. Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/tools/data_source/upload.py", line 8, in <module> from galaxy import eggs ImportError: cannot import name eggs I will appreciate if someone can tell me what could happen to cause such issue and how to resolve it. Regards,

2 1

Upload issue in local installation
by Batsal Devkota 05 Feb '16

05 Feb '16

I installed galaxy locally in a linux server. However, I cannot upload the files (no matter how small, I have tried few kb size fasta files). When I try to upload, the link to the file shows up in the History and gets a new number (purple box). When I click on the link I get 'Dataset is uploading' forever. In the terminal window where I start galaxy, I get the following error report: 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET / HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /root/tool_menu HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:32 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:33 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:40 -0400] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://redhat:8080/root/tool_menu" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /tool_runner/upload_async_create HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /tool_runner/upload_async_message HTTP/1.1" 200 - "http://redhat:8080/" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "GET /history HTTP/1.1" 200 - "http://redhat:8080/tool_runner/upload_async_message" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:33:59 -0400] "POST /root/user_get_usage HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:04 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" 92.17.41.13 - - [02/Aug/2012:15:34:08 -0400] "POST /root/history_item_updates HTTP/1.1" 200 - "http://redhat:8080/history" "Mozilla/5.0 (Windows NT 5.2; rv:13.0) Gecko/20100101 Firefox/13.0.1" The last line keeps going for ever... writes new line every 4 secs. I am stuck and don't know where to look. Please help. Batsal.

6 7

filtering a <param> of type 'data' so only one type is available
by Dan Tenenbaum 24 Oct '14

24 Oct '14

Hi, I have a tool wrapper with a <param> of type="data". Currently, this renders as a text box with a drop down list that has 52 items in it (the number of things I have in my history, I guess). I'd like to filter this list so that only some items in the history (in my case any item whose name ends with '.rda') are shown in this dropdown. It seems like maybe the 'format' parameter to the 'param' tag is what I want, but: 1) I tried format="rda" and that didn't seem to change anything. 2) "rda" is not in the datatypes.conf file which the documentation suggests is required? (.rda is an extension used for serialized R objects.) The tool I'm working on only accepts rda files as input and the reason I'm asking for this is that it is all too easy to accidentally feed it a file of some other type. If the dropdown could be filtered so that only items which will work with the tool are shown, that would be great. Is there a way to do that? Thanks, Dan

4 5

Workflow param error leads to template error
by Paul-Michael Agapow 19 Sep '14

19 Sep '14

Looking for pointers on what might be causing this problem: A user had a moderately complicated workflow that when run under certain parameter values results in a traceback that ends: Module workflow_run_mako:232 in render_body >> for ic in oc.input_step.module.get_data_inputs(): AttributeError: 'WorkflowStep' object has no attribute 'module' After some puzzling, I realized one of the parameters was outside the allowed range (a max on an integer param) and that Galaxy was trying to render "run.mako" flagging up the error but erroring out. Why Galaxy tries to call this non-existent member is unclear to me. Any insight or places I should start exploring? Galaxy version: various (started with an 6-month old one, upgraded to latest production to see if it would fix error, it didn't) Hosting OS: CentOS 6 ------ Paul Agapow (paul-michael.agapow(a)hpa.org.uk) Bioinformatics, Health Protection Agency (UK) ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

3 3

Using Mesos to Enable distributed computing under Galaxy?
by Kyle Ellrott 17 Jun '14

17 Jun '14

I think one of the aspects where Galaxy is a bit soft is the ability to do distributed tasks. The current system of split/replicate/merge tasks based on file type is a bit limited and hard for tool developers to expand upon. Distributed computing is a non-trival thing to implement and I think it would be a better use of our time to use an already existing framework. And it would also mean one less API for tool writers to have to develop for. I was wondering if anybody has looked at Mesos ( http://mesos.apache.org/) You can see an overview of the Mesos architecture at https://github.com/apache/mesos/blob/master/docs/Mesos-Architecture.md The important thing about Mesos is that it provides an API for C/C++, Java/Scala and Python to write distributed frameworks. There are already implementations of frameworks for common parallel programming systems such as: - Hadoop (https://github.com/mesos/hadoop) - MPI ( https://github.com/apache/mesos/blob/master/docs/Running-torque-or-mpi-on-m… ) - Spark (http://spark-project.org) And you can find example Python framework at https://github.com/apache/mesos/tree/master/src/examples/python Integration with Galaxy would have three parts: 1) Add a system config variable to Galaxy called 'MESOS_URL' that is then passed to tool wrappers and allows them to contact the local mesos infrastructure (assuming the system has been configured) or pass a null if the system isn't available. 2) Write a tool runner that works as a mesos framework to executes single cpu jobs on the distributed system. 3) For instances where mesos is not available at a system wide level (say they only have access to an SGE based cluster), but the user wants to run distributed jobs, write a wrapper that can create a mesos cluster using the existing queueing system. For example, right now I run a Mesos system under the SGE queue system. I'm curious to see what other people think. Kyle

5 8

Old Tool Shed URL http://community.g2.bx.psu.edu/ dead
by Peter Cock 28 May '14

28 May '14

Hello all, Either I made a typo in some old README files, or the old URL for the Tool Shed http://community.g2.bx.psu.edu/ has stopped working. Could this redirect to the current Tool Shed address http://toolshed.g2.bx.psu.edu/ please? Thanks, Peter

3 6

user specific access/options
by Jennifer Jackson 19 May '14

19 May '14

Hi Petr, I am going to forward your email to the galaxy-dev list so that the development community can offer comments/suggestions. Best, Jen Galaxy team On 8/30/11 2:27 AM, Petr Novak wrote: > Hi everybody, > I am developing the application on Galaxy server. One of the requirement > is to create user specific list of options. Is it possible to access > somehow $__user_email__ in <options> tag or in <conditional> ?. I did > not found documentation how to used cheetah in galaxy tool xml files but > from files provided with galaxy, cheetah is used only in <command> and > <config> tag. Is that rigth? If it can be used in any part of xml > definition file it would make much easier to generate xml dynamicaly > based on the $__user_email__ > Does anybody have an idea how to manage this problem? > Petr Novak > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/Support

3 2

Default value for data_column not working?
by Peter Cock 14 May '14

14 May '14

Hi all, I'm trying to write a new tool working with tabular data (specifically a Reciprocal Best Hits (RBH) tool using BLAST style tabular output). I want the user to be able to choose a column number (from one of the input files), but I have a default column in mind. However, Galaxy doesn't seem to obey the default column number: <inputs> <param name="a_vs_b" type="data" format="tabular" label="Hits from querying A against B" description="Tabular file, e.g. BLAST output" /> <param name="b_vs_a" type="data" format="tabular" label="Hits from querying B against A" description="Tabular file, e.g. BLAST output" /> <param name="id1" type="data_column" data_ref="a_vs_b" multiple="False" numerical="False" value="1" label="Column containing query ID" help="This is column 1 in standard BLAST tabular output" /> <param name="id2" type="data_column" data_ref="a_vs_b" multiple="False" numerical="False" value="2" label="Column containing match ID" help="This is column 2 in standard BLAST tabular output"/> <param name="score" type="data_column" data_ref="a_vs_b" multiple="False" numerical="False" value="12" label="Column containing containing score to rank on" help="The bitscore is column 12 in standard BLAST tabular output"/> </inputs> I've tried giving the default column value numerically (as shown), and also using value="c2" etc. Regardless, Galaxy just defaults to the first column. Is this a bug, or am I doing something wrong? Thanks, Peter

3 9

history item status not updating
by Robert Baertsch 12 May '14

12 May '14

I submitted a workflow that in turn submits a drm job to sun grid engine. The queue had an error (probably due to a problem with automount). Traceback (most recent call last): File "/inside/depot4/galaxy/lib/galaxy/jobs/runners/__init__.py", line 60, in run_next method(arg) File "/inside/depot4/galaxy/lib/galaxy/jobs/runners/drmaa.py", line 169, in queue_job external_job_id = self.ds.runJob(jt) File "/inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/__init__.py", line 331, in runJob _h.c(_w.drmaa_run_job, jid, _ct.sizeof(jid), jobTemplate) File "/inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/helpers.py", line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer)))) File "/inside/depot4/galaxy/eggs/drmaa-0.4b3-py2.7.egg/drmaa/errors.py", line 90, in error_check raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value)) DeniedByDrmException: code 17: error: no suitable queues When the sysadmin cleared the error, the job started running normally after being in an error state for 10 minutes. The cool thing is that galaxy kept running without a problem. galaxy.jobs.runners.drmaa DEBUG 2013-06-27 14:51:22,968 (10481/4767487) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:23,628 (10481/4767487) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-06-27 15:00:27,751 (10481/4767487) state change: job is running However in the history panel, the job shows as queued but not running, even if I refresh the history panel. Is this normal or should the status change to running? I'm using this version of straight galaxy: changeset: 10162:f295092476c7 branch: stable parent: 10160:2efb1083676b user: Dannon Baker <dannonbaker(a)me.com> date: Sat Jun 15 09:08:09 2013 -0400 summary: Fix reports import issue reported by Lance, https://trello.com/card/bug-in-reports-webapp-imports/506338ce32ae458f6d15e… -Robert Baertsch UC Santa Cruz

3 3