- galaxy-dev - lists.galaxyproject.org

October 2015 News of the Galaxy
by Dave Clements 01 Oct '15

01 Oct '15

1 0

Picard sortSam python error inside its Java app?
by Dooley, Damion 30 Sep '15

30 Sep '15

If any Picard / Samtools experts can help ... Ran into a picard sortSam libc / python error that is only showing up when launched within Galaxy - AND ONLY WITH SORT_ORDER="queryname" option. I run the same command line as the tool does, but as galaxy user on command line, it runs ok?! Is there some resource limit I need to raise or lower in Galaxy context? The server has tonnes of memory and cores. After the crash, I get this at bottom of STDERR along with error message. But I presume this may only be a side-effect: [bam_index_core] the alignment is not sorted (HWI-ST521:273:C4EH9ACXX:4:1101:10000:53458/1): 2632 > 1357 in 75-th chr [bam_index_build2] fail to index the BAM file. since no such message occurs via command line invocation. Advice greatly appreciated, Damion ********************************************* Details... Galaxy tool run: after a few minutes finally yields this on stderr: Picked up _JAVA_OPTIONS: -Xmx2048m -Xms256m *** glibc detected *** python: double free or corruption (top): 0x0000000005fb5200 *** ======= Backtrace: ========= /lib64/libc.so.6[0x3f4d675e66] ... /usr/lib64/libpython2.6.so.1.0(Py_Main+0xb62)[0x3f4f2ff8a2] /lib64/libc.so.6(__libc_start_main+0xfd)[0x3f4d61ed5d] python[0x400649] ... /galaxy_data/production1/job_working_directory/003/3583/galaxy_3583.sh: line 21: 46594 Aborted (core dumped) python ... Command line version:(Taken directly from Galaxy tool execution info; extract below shows with QUIET=false though I get same success with QUIET=true, and I loaded the right picard.jar path into JAVA_JAR_PATH) -salk tmp]$ _JAVA_OPTIONS=${_JAVA_OPTIONS:-'-Xmx2048m -Xms256m'} && export _JAVA_OPTIONS && java -jar $JAVA_JAR_PATH/picard.jar SortSam INPUT="/galaxy_data/production1/files/004/dataset_4987.dat" OUTPUT="/galaxy_data/production1/files/005/dataset_5408.dat" SORT_ORDER="queryname" VERBOSITY=ERROR VALIDATION_STRINGENCY=LENIENT Picked up _JAVA_OPTIONS: -Xmx2048m -Xms256m [Wed Sep 30 12:29:39 PDT 2015] picard.sam.SortSam INPUT=/galaxy_data/production1/files/004/dataset_4987.dat OUTPUT=/galaxy_data/production1/files/005/dataset_5408.dat SORT_ORDER=queryname VERBOSITY=ERROR VALIDATION_STRINGENCY=LENIENT QUIET=false COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json [Wed Sep 30 12:29:39 PDT 2015] Executing as galaxy@grl-salk on Linux 2.6.32-220.17.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.6.0_30-b30; Picard version: 1.136(f187319bf8bbde56892d5b5a1ce3fc0529b71a49_1436805856) IntelDeflater [Wed Sep 30 12:42:51 PDT 2015] picard.sam.SortSam done. Elapsed time: 13.21 minutes. Runtime.totalMemory()=1677066240 Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

1 0

Query in publishing the tool
by Gaurav Kumar 30 Sep '15

30 Sep '15

Hi there, Greetings. I have created a tool and would like to publish it in the galaxy, however I am having problems in doing that. My tool is a kind of pipeline and hence has a combination of several .xml and .py files. Now, ideally I would like to publish the tool in such a way that when somebody installs it should get the all the .xml and .py files right way. However, right now even I have uploaded the tool as a zipped repo, all the files are been uploaded separately and the user will have to download them one by one. I would highly appreciate if you please help me out in guiding the way that I can upload the tool as a single repo file with several files in it instead of uploading several individual files. Many thanks Gaurav

1 0

Question about using dynamic options and refresh_on_change for rendering 2 associated select lists
by Greg Von Kuster 30 Sep '15

30 Sep '15

Hello all, I’m working on adding support to the Galaxy framework for datatypes in the constructive solid geometry (CSG) space and I have several working Galaxy tools and a visualization plugin that deal with 3d shapes in this environment. All of this work will soon be contributed to open source. For one of the tools, I need to provide 2 select lists on the tool form that are associated in such a way that when an option in select list 1 is chosen, the options in select list 2 are altered and re-rendered. This behavior would probably use a combination of dynamic options and refresh_on_change between the 2 associated select lists. I’ve not seen a tool that does this precisely, so I’m hoping that if it is currently possible to do this within Galaxy tools, someone will point me to an example. If it is not currently possible, I will contribute a PR for supporting it, but would like some input as to how it should be done. Dynamic options are currently rendered using 4 approaches; from_file, from_dataset, from_parameter and from_data_table. One or more of several filters can then be applied to the options to alter them if desired. Since these CSG tools involve new datatypes, I have flexibility into how the metadata elements are set up that are needed to render these 2 select lists. A dictionary may work where the metadata element contents look like this: {a: [‘1’, ‘2’], b: [‘3’, ‘4’] } The options in select list 1 would be ‘a’ and ‘b’. When ‘a’ is selected, select list 2 would have options ‘1’ and ‘2’, and when ‘b’ is selected, select list 2 would have options ‘3’ and ‘4’. To support this approach, perhaps dynamic options could be enhanced to include a from_dict option. There would be some complxity involved in associating the lists using refresh_on_change so that select list 2 is refreshed with the correct options when select list one is changed. I haven’t yet worked out these details. Also, using a dictionary will not keep order for the options, so perhaps a list of tuples would be better? [ [ ‘a’, [‘1’, ‘2’]], [‘b’, [‘3’, ‘4’]]] Can someone let me know if this is currently possible, and if so, an example tool that does it? If it’s not possible, does the above approach seem reasonable, or is there a better way? Thanks very much, Greg Von Kuster

2 7

set_user_disk_usage.py as a cron job?
by Jose Juan Almagro Armenteros 28 Sep '15

28 Sep '15

Hello all, I have just encounter a bug with the disk-quota for many users and I run the set_user_disk_usage.py script. It recalculated the disk usage for most of them and now seems that everything is okay. My question is: should I add this script as a cron job or this kind of bugs are just something inusual that happens after upgrading? Best regards, Jose

1 0

Re: [galaxy-dev] validator type="empty_field" is not working within repeat tags
by Anmol Hemrom 25 Sep '15

25 Sep '15

Hi, I was trying to enforce not to allow empty string as an input to text field. It is working outside the repeat tags but not working within repeat tags. 1) Working fine <param name="input" type="text" label="Enter Name"> <validator type="empty_field"/> </param> 2) Not Working within repeat tags. <repeat name="test" title="Test"> <param name="input" type="text" label="Enter Name"> <validator type="empty_field"/> </param> </repeat> Thanks Anmol On Fri, May 29, 2015 at 2:00 PM, Dave Clements <clements(a)galaxyproject.org> wrote: > Hi Anmol, > > The best place to post this question is the Galaxy Dev List < > galaxy-dev(a)lists.galaxyproject.org>. You can also search to see if > anyone has asked a similar question at > https://galaxyproject.org/search/web/ > > Thanks, > > Dave C > > On Fri, May 29, 2015 at 6:56 AM, <anmol.jh(a)gmail.com> wrote: > >> Hi, >> >> I was trying to enforce not to allow empty string as an input to text >> field. It is working outside the repeat tags but not working within repeat >> tags. >> 1) Working fine >> >> <param name="input" type="text" label="Enter Name"> >> <validator type="empty_field"/> >> </param> >> >> 2) Not Working within repeat tags. >> >> <repeat name="test" title="Test"> >> <param name="input" type="text" label="Enter Name"> >> <validator type="empty_field"/> >> </param> >> </repeat> >> >> Thanks >> >> Anmol >> >> _____________________________________ >> Sent from http://dev.list.galaxyproject.org >> >> > > > -- > http://galaxyproject.org/ > http://getgalaxy.org/ > http://usegalaxy.org/ > https://wiki.galaxyproject.org/ >

2 1

Adding data libraries from filesystem path creating duplicates
by Martin Vickers 25 Sep '15

25 Sep '15

Hi All, I've noticed an issue a couple of times now where I've added a directory of fastq's from an NFS mounted filesystem (reference only rather than copying into galaxy) and then galaxy times out. Load average begins to get really high and then consumes all the RAM and sometimes crashes. These are the same symptom as I had before with this issue that was never resolved; http://dev.list.galaxyproject.org/run-sh-segfault-td4667549.html#a4667553 What I've noticed is that in the dataset I'm uploading to galaxy, there are suddenly many duplicates. In this example that's just happened, there are 288 fastq.gz files in the physical folder, but galaxy has created 6 references to each file resulting in 1728 datasets in the folder (see attached images). When this happened before and crashed the galaxy application, whenever it restarted it'd try to resume what it was doing which created an endless loop of retrying and crashing until the job was removed. Does anyone know what may be causing this? Cheers, Martin -- -- Dr. Martin Vickers Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University w: http://www.martin-vickers.co.uk/ e: mjv08(a)aber.ac.uk t: 01970 62 2807

2 3

_JAVA_OPTIONS config parameter in job_conf.xml.sample_advanced
by Dooley, Damion 24 Sep '15

24 Sep '15

Wondering if anyone has reported this for the doc/sample file team: Isn't the java config example line in that file wrong (at least for my java 1.6): <env id="_JAVA_OPTIONS">-Xmx=6GB</env> With a setting of -Xmx=10g, Picard wouldn't run sortSam, instead this stderr text: ***************** Fatal error: Exit code 1 () Picked up _JAVA_OPTIONS: -Xmx=10g Invalid maximum heap size: -Xmx=10g Could not create the Java virtual machine. ***************** I changed the <env> tag to take out "=" so it reads: <env id="_JAVA_OPTIONS">-Xmx6G</env> and now its working fine! Had to lookup -Xmx param format on the net from: http://javahowto.blogspot.ca/2006/06/6-common-errors-in-setting-java-heap.h… Regards, Damion. Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

1 0

Advice on Galaxy administration
by Raymond Wan 24 Sep '15

24 Sep '15

Dear all, I'm still new to Galaxy but was wondering if someone could offer some advice. On a local Galaxy instance, if I were the Galaxy administrator but do not have sudo access on the machine itself, is this a problem? Or are the two roles intricately linked so that the Galaxy administrator will have to pester the system administrator frequently? The web interface alone is not sufficient for the Galaxy administrator. (i.e., does a Galaxy administrator need sudo access ... or even just group access to the Galaxy subdirectory to effectively do his/her job? ...I mean making Galaxy owned by a Unix group and putting the Galaxy administrator in that group.) As an aside, I was trying to do some alignment with bwa and realized I needed to prepare the genome by creating the index and an accompanying .loc file. The instructions [1] seem to imply some kind of command-line access.. Thank you for your help! Ray [1] https://wiki.galaxyproject.org/Admin/DataPreparation

4 10

Galaxy on a Cluster -- Active Directory LDAP configuration
by Carlos Lijeron 24 Sep '15

24 Sep '15

Everyone, We are setting up Galaxy to work with our cluster and SLURM as the work manager. The cluster itself authenticates to our local Active Directory, so I’m wondering if the best way to track resource utilization of Galaxy users on the cluster is to also have Galaxy authenticate to the same Active Directory LDAP. Is anyone on this list using the same configuration and tracking resource utilization from Galaxy users submitting jobs to the cluster nodes? Please advise. Thank you all ! Carlos.

3 5