April 2015 - galaxy-dev - lists.galaxyproject.org

Galaxy on HPC and Bright Cluster Manager?
by Carlos Lijeron 22 Apr '15

22 Apr '15

Good day everyone, Has anyone of you been able to implement Galaxy on a HPC using Bright Cluster Manager as the main DRM? I noticed that only a few have been known to work with Galaxy, but the list does not include Bright. Any advice/ideas will be greatly appreciated. TORQUE Resource Manager PBS Professional Open Grid Engine Univa Grid Engine (previously known as Sun Grid Engine and Oracle Grid Engine) Platform LSF HTCondor Slurm Galaxy Pulsar (formerly LWR) Thanks. Carlo

3 2

Issue writing functional tool test with repeat tag in config
by Christy Shafer 21 Apr '15

21 Apr '15

Hello, I'm trying to format a test tag in a tool config for the public Tool Shed that includes a repeated input parameter. I followed the syntax suggested in the following archived galaxy-dev post (was unable to find much other guidance): http://git.net/galaxy-development-source-control/msg15241.html, but the config file must be invalid because my repository on the Tool Shed does not register as an installable Galaxy utility. Any guidance is much appreciated.. Below are the relevant config code sections: <inputs> <repeat name="Files" title="Join file"> <param name="joinMe" type="data" checked="yes" format="tabular" label="Join" /> <param name="joinCol" label="using column" type="data_column" data_ref="joinMe" /> </repeat> <param name="headerYes" type="select" label="Treat first line as header?"> <option value="yes" selected="true">Yes</option> <option value="no">No</option> </param> </inputs> <outputs> <data format="tabular" name="Joined_all" label="Multi-Join result"/> </outputs> <tests> <test> <param name="Files_0|joinMe" value="multi_join_serial_in1.tab" ftype="tabular"/> <param name="Files_0|joinCol" value="1"/> <param name="Files_1|joinMe" value="multi_join_serial_in2.tab" ftype="tabular"/> <param name="Files_1|joinCol" value="1"/> <param name="Files_2|joinMe" value="multi_join_serial_in3.tab" ftype="tabular"/> <param name="Files_2joinCol" value="2"/> <param name="headerYes" value="yes"/> <output name="Joined_all" value="multi_join_serial_out.tab" ftype="tabular"/> <test/> <tests/> Thanks for your help, CS

3 3

Problem killing jobs (via Admin>Manage Jobs)
by Poole, Richard 21 Apr '15

21 Apr '15

Hi all, I am having an issue killing jobs via Admin>Manage Jobs. When I try to submit kills I see this in the Galaxy log: galaxy.jobs.runners.local WARNING 2015-04-19 15:55:03,089 stop_job(): 10374: no PID in database for job, unable to stop I have updated my local Galaxy server (via hg) and am currently running 17052:28113eee212. I am scared the issue is a database issue, however everything else and communication with the PostgreSQL database seems functional. Any help greatly appreciated :) Richard Richard J Poole PhD Wellcome Trust Fellow Department of Cell and Developmental Biology University College London 21 University Street, London WC1E 6DE Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133)

1 0

GCC2015 Registration and Lodging Scholarship for US-Based Post-Docs and Students
by Dave Clements 21 Apr '15

21 Apr '15

Hello all, *We are pleased to offer registration <http://gcc2015.tsl.ac.uk/registration> and lodging <http://gcc2015.tsl.ac.uk/logistics/#Lodging> scholarships for GCC2015 for students and post-docs who are based in the United States, and who are members of historically underrepresented groups in life science research.* These scholarships cover registration costs <http://gcc2015.tsl.ac.uk/registration> for any GCC2015 events <http://gcc2015.tsl.ac.uk/proigramme> the recipients sign up for, and also covers lodging <http://gcc2015.tsl.ac.uk/logistics/#Lodging> during those events in the official conference lodging <http://gcc2015.tsl.ac.uk/logistics/#Lodging> on the nearby UEA campus. *The scholarships do not cover travel or other expenses.* To apply for a scholarship qualifying applicants need to 1. Submit a poster abstract <http://bit.ly/gcc2015sub>. *The deadline is May 1* 2. Have a faculty member / PI send a letter of recommendation to the Galaxy Community Fund Board <community-fund(a)lists.galaxyproject.org> no later than May 15. 3. Submit an application form <http://bit.ly/gcc2015scholarship>, no later than May 1. 4. Have sufficient funds to cover your travel and other expenses. Applicants will be notified of their scholarship status no later than May 19. Scholarships are funded by the *Galaxy Community Fund*. The Galaxy Community Fund was established with the surplus funds from GCC2014, and is currently administered by the GCC2014 organizers. Please do let us know if you have any questions, and we hope to see you at GCC2015 <http://gcc2015.tsl.ac.uk/>! Galaxy Community Fund Board <community-fund(a)lists.galaxyproject.org> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

Adding a dataset to a dataset list breaks "Run this job again"
by Brian Claywell 21 Apr '15

21 Apr '15

While testing an upgrade from latest_2014.10.06 to GitHub release_15.03 branch (cloned today) I ran into an issue trying to use the "Run this job again" functionality on a dataset. I was playing with creating dataset lists using "Operations on multiple datasets" and noticed that after adding a dataset to a dataset list, trying to run the job again fails. This is the tool form that gets displayed: [image: Inline image 1] Additionally an error is logged: galaxy.tools ERROR 2015-04-15 21:58:53,258 'HistoryDatasetAssociation' object has no attribute 'collection' Deleting the dataset list makes no difference. However, if I copy this dataset to a new history and click "Run this job again", no error occurs and the proper tool form appears. If I then add the dataset in the new history to a dataset list, the error reappears. Cheers, Brian -- Brian Claywell | programmer/analyst Matsen Group | http://matsen.fredhutch.org <http://matsen.fhcrc.org/> Fred Hutchinson Cancer Research Center

2 1

metadata in parallelization
by Roberto Alonso 21 Apr '15

21 Apr '15

Hello, I am writing some code to enable parallelization for some tool wrappers. First, I did it for simple bwa wrapper, but now I am modifying toolshed.g2.bx.psu.edu/repos/devteam/bwa/c71dd035971e/bwa/bwa-mem.xml to check if the code would work with this wrapper. So, I wrote some code that I thing was necessary in order to merge some bam and I added the parallelism tag (in bold) to the config file: <tool id="bwa_mem" name="BWA-MEM" version="0.1"> <macros> <import>bwa_macros.xml</import> </macros> <requirements> <requirement type="package" version="0.7.10.039ea20639">bwa</requirement> <requirement type="package" version="1.1">samtools</requirement> </requirements> <description>- map medium and long reads (> 100 bp) against reference genome</description> *<parallelism method="multi" split_size="3" shared_inputs="ref_file" split_mode="number_of_parts" merge_outputs="bam_output" split_inputs="fastq_input1,fastq_input2" ></parallelism>* <command> ... So, everything works well, and the resulting bam from parallelization mode and without the parallelization mode is the same but the Galaxy log throws an error regarding metadata, it says something like this: galaxy.jobs.splitters.multi DEBUG 2015-04-17 09:54:58,335 merge finished: /home/ralonso/galaxy/database/files/000/dataset_198.dat galaxy.jobs.runners.tasks DEBUG 2015-04-17 09:54:58,473 executing external set_meta script for job 200: python /home/ralonso/galaxy/database/tmp/set_metadata_E5fGIE.py /home/ralonso/galaxy/database/tmp/tmpHS8Byo /home/ralonso/galaxy/database/job_working_directory/000/200/galaxy.json /home/ralonso/galaxy/database/tmp/metadata_in_HistoryDatasetAssociation_198_yOGiQG,/home/ralonso/galaxy/database/tmp/metadata_kwds_HistoryDatasetAssociation_198_nAsQoq,/home/ralonso/galaxy/database/tmp/metadata_out_HistoryDatasetAssociation_198_I_cLs4,/home/ralonso/galaxy/database/tmp/metadata_results_HistoryDatasetAssociation_198_qhjzoV,/home/ralonso/galaxy/database/files/000/dataset_198.dat,/home/ralonso/galaxy/database/tmp/metadata_override_HistoryDatasetAssociation_198_ScKLqH Traceback (most recent call last): File "/home/ralonso/galaxy/database/tmp/set_metadata_E5fGIE.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() ImportError: No module named galaxy_ext.metadata.set_metadata galaxy.jobs.runners.tasks DEBUG 2015-04-17 09:54:58,624 execution of external set_meta finished for job 200 *galaxy.datatypes.metadata DEBUG 2015-04-17 09:54:58,714 setting metadata externally failed for HistoryDatasetAssociation 198: External set_meta() not called* When using no parallelization mode, there is no problem, also because Galaxy doesn't go through this part of code, I mean it doesn't execute this. I see that Galaxy have to do something with metada attributes, but what is t trying to do? is there any way to solve this? Thank you very much Regards, Roberto

2 1

pbs/torque config
by Beginner TI 21 Apr '15

21 Apr '15

Hi, I'm trying to enable the capability to submit jobs to a pbs cluster, however I am quite confused by the documentation. I don't know where I could specify the pbs server. I saw the following on the page: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster 1 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <plugins> 2 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner"/> 3 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> </plugins> 4 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <destinations default="pbs_default"> 5 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <destination id="pbs_default" runner="pbs"/> 6 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <destination id="other_cluster" runner="pbs"> 7 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <param id="destination">@other.cluster</param> 8 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> </destination> 9 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <destination id="long_jobs" runner="pbs"> 10 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <param id="Resource_List">walltime=72:00:00,nodes=1:ppn=8</param> 11 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> <param id="-p">128</param> 12 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> </destination> 13 <https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#CA-e4c8fb8b…> </destinations> is this <param id="destination"> the place you specify where your pbs/torque server? and the syntax is to prepend a '@' to the server name? for example, should a most basic config look like: <plugins> <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner"> </plugins> <destinations default="pbs"> <destination id="pbs" runner="pbs" > <param id="destination">@pbs_server_full_name</param> <param id="Resource_List">walltime=72:00:00,nodes=1:ppn=8</param> <param id="-q">myqueue</param> </destination> <destinations> I searched online and to my surprise I didn't find any useful information, or any working sample...please give me a hand guys! Thanks! Rui

2 1

Re: [galaxy-dev] Tool param type "library_data" (Dooley, Damion)
by Dooley, Damion 20 Apr '15

20 Apr '15

I was successful getting library dataset selection and form submission to happen within tool form tool form input <param type="library_data" ...> - Yay! But it appears one last fix (besides the "url: "/visualization/list_libraries",' bug) is needed to enable history tool item to be RERUN. So if a galaxy dev can tackle this, great! Otherwise I'll have to drop this approach :<( ***************** For others who may need... tool form's param: <param name="library_datasets" type="library_data" label="Reference Bin file" help="Select one reference bin at a time from the popup window's Reference Bin library."/> <command> tag contents.... #if $library_datasets -B " #for $i, $dataset in enumerate($library_datasets) #$dataset.get_file_name() $dataset.id #end for " #end if Successfully submits either the full file path of selected dataset, or its id. ***************** However trying to rerun this history tool item gets us an error (not related to the above <command> contents): Error Traceback: View as: Interactive | Text | XML (full) ⇝ AttributeError: 'int' object has no attribute 'id' .... Module galaxy.tools.parameters.basic:146 in value_to_basic view >> return self.to_string( value, app ) Module galaxy.tools.parameters.basic:2406 in to_string app <galaxy.app.UniverseApplication object at 0x59a0710> ldda 331 self <galaxy.tools.parameters.basic.LibraryDatasetToolParameter object at 0x7fced01d2090> value [331] >> return [ldda.id for ldda in value] AttributeError: 'int' object has no attribute 'id' which is in /lib/galaxy/tools/parameters/basic.py: class LibraryDatasetToolParameter( ToolParameter ): """ Parameter that lets users select a LDDA from a modal window, then use it within the wrapper. """ ... def to_string( self, value, app ): if not value: return value return [ldda.id for ldda in value] "Value" is getting an array of dataset ids, whereas this method is expecting value to be a list of dataset objects. I tried changing ldda.id to just "id" but found other parts of LibraryDatasetTool code were broken as they too were expecting an object. Maybe this is related to the tool job's history detail link showing "Reference Bin file" as a string label of an object?: Input Parameter Value Note for rerun BLAST results as XML 240: megablast Pasted Entry vs 16SMicrobial.fasta Reference Bin file [<galaxy.model.LibraryDatasetDatasetAssociation object at 0x7fcee04d4350>] Tabular Report Column Labels Short name HTML Report template templates.html_report ******************************** The other bug: 2) Tool_form.mako has a hardcoded url that doesn't work for galaxy installs that have a prefix so that needs adjustment: $(".add-librarydataset").click(function() { var link = $(this); $.ajax({ url: "/visualization/list_libraries",

2 3

April GalaxyAdmins Meetup: Video and Slides are now available
by Dave Clements 20 Apr '15

20 Apr '15

Hi all, Slides from both talks and video of the whole meetup are now available at https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_04_16 Many thanks to Carrie and Sam for presenting. See you at the next meetup June 18! Dave C On Thu, Apr 16, 2015 at 7:34 AM, Dave Clements <clements(a)galaxyproject.org> wrote: > Hello all, > > GalaxyAdmins is starting in about 25 minutes. The room is here: > > https://connect.johnshopkins.edu/galaxyadmins201504/ > > It takes a few minutes to navigate through Flash, so connecting a few > minutes early is encouraged. > > Thanks, > > Dave C > > > > On Wed, Apr 15, 2015 at 9:14 AM, Dave Clements <clements(a)galaxyproject.org > > wrote: > >> Hello all, >> >> Just a reminder that the April GalaxyAdmins online meetup is tomorrow, >> Thursday, at 5pm Central European Summer Time. See >> https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_04_16 >> for details. A link to the meetup will be posted about an hour before the >> meetup. >> >> Thanks, >> >> Dave C >> >> On Thu, Apr 9, 2015 at 12:58 PM, Dave Clements < >> clements(a)galaxyproject.org> wrote: >> >>> Hello all, >>> >>> The April 2015 GalaxyAdmins meetup >>> <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2015_04_16> >>> will be held online on Thursday, 16 April (a week from today). We'll meet >>> at 5pm Central European Summer Time (see your local time >>> <http://bit.ly/1EFUf0M>). >>> >>> Carrie Ganote <http://www.researchgate.net/profile/Carrie_Ganote> from >>> the National Center for Genome Analysis Support (NCGAS) >>> <http://ncgas.org/> and Pervasive Technology Institute >>> <http://pti.iu.edu/> at Indiana University will talk about her Galaxy >>> work with Trinity, IU Galaxy, and the Open Science Grid. And Sam >>> Guerler <https://wiki.galaxyproject.org/guerler> of the Galaxy Team >>> will also discuss recent work to update Galaxy's tool forms. >>> >>> GalaxyAdmins <https://wiki.galaxyproject.org/Community/GalaxyAdmins> is >>> a discussion group for Galaxy community members who are responsible for >>> Galaxy installations. Please see the meetup page ( >>> http://bit.ly/gxyadm201504) for more details. >>> >>> Hope to see (hear?) you there, >>> >>> Dave C (channeling Hans-Rudolf Hotz) >>> >>> -- >>> http://galaxyproject.org/ >>> http://getgalaxy.org/ >>> http://usegalaxy.org/ >>> https://wiki.galaxyproject.org/ >>> >> >> >> >> -- >> http://galaxyproject.org/ >> http://getgalaxy.org/ >> http://usegalaxy.org/ >> https://wiki.galaxyproject.org/ >> > > > > -- > http://galaxyproject.org/ > http://getgalaxy.org/ > http://usegalaxy.org/ > https://wiki.galaxyproject.org/ > -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

GCC2015 Oral Presentation Abstract Submission closes 20 April
by Dave Clements 20 Apr '15

20 Apr '15

Hello all, Just a reminder that oral presentation abstract submission for GCC2015 closes on Monday, 20 April, 5 days from now. Poster submission closes 1 May. See http://gcc2015.tsl.ac.uk/Abstracts for more. The 2015 Galaxy Community Conference is an opportunity to share your work with 200+ other researchers working in data-intensive biology. GCC2015 features 2 days of (2 different) hackathons, 2 days of training, and 2 days of meeting, posters, presentations, birds of a feather, and networking. See http://gcc2015.tsl.ac.uk/ for more on GCC2015. Looking forward to seeing you in July in Norwich! Dave C. On Wed, Feb 25, 2015 at 12:04 AM, Dave Clements <clements(a)galaxyproject.org> wrote: > Hello all, > > *We are pleased to announce that abstract submission for Oral and Poster > Presentations <http://gcc2015.tsl.ac.uk/Abstracts> at the 2015 Galaxy > Community Conference (GCC2015) <http://gcc2015.tsl.ac.uk/> is now open.* > > Abstract submission <http://gcc2015.tsl.ac.uk/Abstracts> for oral > presentations closes 10 April, and poster submission closes 1 May. Poster > authors will be notified of acceptance status within two weeks of > submission, while oral presentation authors will be notified no later than > 24 April. *Please consider presenting your work. If you are dealing with > big biological data, then this meeting wants to hear about it.* > > *About GCC2015* > > GCC2015 <http://gcc2015.tsl.ac.uk/> is hosted by The Sainsbury Lab > <http://www.tsl.ac.uk/>, in Norwich, United Kingdom > <http://www.visitnorwich.co.uk/>, in July 2015. GCC has been expanded > this year and will now offer two hackathons, two Training Days, and a two > day conference <http://gcc2015.tsl.ac.uk/programme/>. The Training Days > <http://gcc2015.tsl.ac.uk/training-day/> cover 15 different topics > spanning the full Galactic spectrum of topics. *Take a look > <http://gcc2015.tsl.ac.uk/programme/>.* > > Thanks, > > GCC2015 Organising Committee <http://gcc2015.tsl.ac.uk/organisers/> > > -- > http://galaxyproject.org/ > http://getgalaxy.org/ > http://usegalaxy.org/ > https://wiki.galaxyproject.org/ > -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 1