November 2012 - galaxy-dev - lists.galaxyproject.org

changing datasets file_path storage location
by Shantanu Pavgi (Campus) 19 Feb '13

19 Feb '13

We are planning to change datasets directory location (file_path setting in universe_wsgi.ini) in our Galaxy installation. We will be moving existing datasets to the new directory location and updating XSendfile setting in Apache as well. I was wondering if there are any side effects that we should be considering before making this change. There are certain fields in the database (e.g. command_line in jobs table) which contain absolute path of datasets. Do we need to update any of these fields to make sure Galaxy is functional after file_path is changed? -- Thanks, Shantanu

3 4

run_functional_tests.sh -sid option
by Peter 11 Feb '13

11 Feb '13

Hi, I'm trying to use run_functional_tests.sh to run all the tests for a section (group of tools). I've read the output from: ./run_functional_tests.sh help For example, from the tools_conf.xml.sample we have <section name="ENCODE Tools" id="EncodeTools"> <tool file="encode/gencode_partition.xml" /> <tool file="encode/random_intervals.xml" /> </section> And looking at these tools' XML files, <tool id="gencode_partition1" name="Gencode Partition"> and: <tool id="random_intervals1" name="Random Intervals"> I'd like to run the functional tests for the ENCODE tools, Using the switch for an individual tool id (-id) works, ./run_functional_tests.sh -id gencode_partition1 ... Ran 1 test in 22.302s ... and so does this (well, it say the tool doesn't have any tests which is true in this example): ./run_functional_tests.sh -id random_intervals1 ... Ran 1 test in 0.027s ... However, I also tried using the section id switch (-sid), ./run_functional_tests.sh -sid EncodeTools ... Ran 0 tests in 0.000s ... I also tried using the section name, /run_functional_tests.sh -sid "ENCODE Tools" Is this (-sid not working) a known issue, or am I using it wrong? Thanks, Peter

4 5

Visualise Galaxy workflows using graphviz
by Peter van Heusden 21 Jan '13

21 Jan '13

Hi there Because my Galaxy workflows tend to sprawl outside the viewable space in my browser, I've written a script that, when given a workflow JSON file as input, writes out a graphviz dot format graph of the workflow. By default the graph treats datasets as nodes and analyses as edges (unlike the Galaxy workflow editor, but quite like the way the history is presented), but you can choose to have analyses as nodes (and thus datasets as edges) too. In case anyone else finds this useful, the code can be downloaded from my bitbucket: https://bitbucket.org/pvanheus/galaxy/src/f29453f3d9d8/contrib/workflow_to_… Usage is something like: ./workflow_to_dot.py Galaxy-Workflow.ga |dot -Tsvg >workflow.svg and then you can view your workflow.svg with eog or another viewer. Peter

3 3

Job output not returned from Cluster
by Sascha Kastens 17 Jan '13

17 Jan '13

Hi all, I have exactly the same problem as described here: http://gmod.827538.n3.nabble.com/Job-output-not-returned-from-cluster-td879… All the SGE stuff is administrated in our company and unfortunately they know nothing about Galaxy and so it?s my task to get it running. Everything works fine except the problem mentioned above. I?ve contacted Erick but haven?t received a response yet. If anybody can give me some hints where I can look at to solve the problem or at least what I can tell the admins, I would be really grateful. Thanks in advance. If you need further information let me know! Cheers, Sascha

4 8

Uploading large files to history through API (some random thoughts)
by John Chilton 07 Jan '13

07 Jan '13

I went down something of a rabbit hole last night, I thought that uploading files through the API using multipart/form-data worked. It is only at the very end of my adventure that I realized it only works for libraries uploads. I think it would be great to get that working to histories as well. I cannot spend much more time on this right now (I thought I only needed to spend an hour to implement the client side stuff), but I thought I would post the progress I made here in case someone wants to take up the fight someday: The first thing is that when issuing multipart/form-data requests, inputs are not currently being deserialized from JSON (so for instance in the tools api code the "inputs" variable would be the string containing the json, not the python dictionary). To bring multipart/form-data requests in line with other content requests I modified this in ilb/galaxy/web/framework/__init__.py payload = kwargs.copy() named_args, _, _, _ = inspect.getargspec(func) for arg in named_args: payload.pop(arg, None) with this: payload = kwargs.copy() named_args, _, _, _ = inspect.getargspec(func) for arg in named_args: payload.pop(arg, None) for k, v in payload.iteritems(): if isinstance(v, (str, unicode)): try: payload[k] = simplejson.loads(v) except: # may not actually be json, just continue pass payload = util.recursively_stringify_dictionary_keys( payload ) One could also imagine doing this replacement in the tools api controller directly on payload['input'] instead. After that change, files still weren't being matched up with inputs properly: This debug statement I added to upload_common.py, demonstrates that file_data is None. galaxy.tools.actions.upload_common INFO 2012-11-27 00:22:17,585 Uploaded datasets is {'NAME': u'galxtest694734465387762969.txt', 'file_data': None, 'space_to_tab': None, 'url_paste': None, '__index__': 0, 'ftp_files': None} To address this, the files_*|file_data parameters need to be moved inside of the inputs dict. That means in lib/galaxy/webapps/galaxy/api/tools.py, changing this: inputs = payload[ 'inputs' ] To this: inputs = payload[ 'inputs' ] for k, v in payload.iteritems(): if k.startswith("files_"): inputs[k] = v Then the debug line becomes this: galaxy.tools.actions.upload_common INFO 2012-11-27 00:33:15,168 Uploaded datasets is {'NAME': u'galxtest2484272839208214846.txt', 'file_data': FieldStorage('files_0|file_data', u'galxtest2484272839208214846.txt'), 'space_to_tab': None, 'url_paste': None, '__index__': 0, 'ftp_files': None} Which matches pretty well with the same line coming from a web browser request: galaxy.tools.actions.upload_common INFO 2012-11-27 00:23:57,590 Uploaded datasets is {'NAME': u'', 'file_data': FieldStorage('files_0|file_data', u'second_step.png'), 'space_to_tab': None, 'url_paste': u'', '__index__': 0, 'ftp_files': None} These changes aside I still wasn't it sill didn't work, that is why this is a rambling e-mail and not a pull request. I got this exception http://pastebin.com/hhs1pjtP. I guess something to do with the session handling stuff being different between API calls and normal web calls. Anyway, I've inspected the actual requests I was generating and I think they are reasonable, Galaxy just needs to be augmented to accept them :). If someone does land up taking a look at this, I have committed my test case to blend4j so it can be used to really quickly test such a client request (requires git, Java, and maven): % git checkout git@github.com:jmchilton/blend4j.git % cd blend4j % mvn test -Dtest=ToolsTest -Dtest.galaxy.instance=http://localhost:8080/ -Dtest.galaxy.key=<testapikey> Thanks, -John

2 1

Problems with Internet Explorer 9?
by Peter Cock 19 Dec '12

19 Dec '12

Hello all, Are there (or were there) any known problems with Galaxy under Microsoft Internet Explorer 9? One of our biologists is now having problems with our local Galaxy server, probably after updating his browser. Specifically new history entries do not seem to automatically update from queued to running to done, and also the "peep" functionality to expand/collapse a history entry had stopped working. This was on a large history of about 100 entries. We installed Firefox 10, logged into Galaxy, and it was fine with this same history. A quick test on the public Galaxy with a single history entry seemed to work in both browsers, so perhaps this is something you've already fixed in the latest Galaxy - or perhaps it depends on the size of the history. Any ideas? Has anyone else seen something similar? Thanks, Peter

2 1

producing HTML output with images
by Nikhil Joshi 19 Dec '12

19 Dec '12

Hi all, I am having trouble producing HTML output with images. In the past I have been able to produce HTML files with no images and it seemed to work fine. However, now I am writing a script that produces diagnostic images and I want to display all of the images on one page using HTML. I am using the "files_path" variable to create the plots in the working directory, and then I am using the "extra_files_path" variable to access the final plot from the html file. I look at the resulting HTML files and it points to the proper plot full path, and the plot DOES exist... but when I click to view the output, the plot doesn't render. It just shows an empty box with a "broken image" icon, however, the text does render. I copy the plot file, and I can view it fine by itself. I copy the plot and the html and I can view the page just fine offline, but galaxy doesn't want to render the image for some reason. What am I doing wrong? - Nik. -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center Davis, CA

3 4

add new slurm-drmaa runner to galaxy-central
by Di Pe 10 Dec '12

10 Dec '12

may be some other people would find this a useful enhancement: https://bitbucket.org/galaxy/galaxy-central/issue/778/add-new-slurm-drmaa-r…

2 1

Permission denied error for velveth
by Andreas Kuntzagk 06 Dec '12

06 Dec '12

Hi, I enabled velvet on our local install. But when I try to use velveth, I get an "Permission denied: '/data/galaxy/galaxy-dist/database/files/027/dataset_27903_files'" This file does not exist. Only file there is /data/galaxy/galaxy-dist/database/files/027/dataset_27903_dat regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

4 7

Auto delete dataset after workflow run?
by Praveen Raj Somarajan 03 Dec '12

03 Dec '12

Hello All, Is there a way to set "auto delete" some datafiles at the end of workflow run? I'm asking this because it is sometimes useful to delete some outputs (say intermediate files, .logs, etc), for example "auto delete" .sam file after executing sam-to-bam tool, which would free-up lot of space. This is especially useful when work with large number of samples. Look forward to any comments/suggestions. Best, Raj. ________________________________ This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original message. Further, you are not to copy, disclose, or distribute this e-mail or its contents to any other person and any such actions that are unlawful. This e-mail may contain viruses. Ocimum Biosolutions has taken every reasonable precaution to minimize this risk, but is not liable for any damage you may sustain as a result of any virus in this e-mail. You should carry out your own virus checks before opening the e-mail or attachment. The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system. OCIMUMBIO SOLUTIONS (P) LTD

3 4