- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Dynamic data library
by Hans-Rudolf Hotz 02 Oct '13

02 Oct '13

On 10/01/2013 03:53 PM, Cole, Nathan (NIH/NCI) [C] wrote: > Thank you both for your responses. I will be looking into both of these. > > With regard to the from_file option to add the sample selection into the tool: I assume this means that the metadata and everything is loaded into galaxy at the time the tool is run. This depends on how you write your tool. Do you just wanna read the ie fastq file or do you also wanna read the meta data. Also, how is the meta data accessible? eg. is it stored in a txt file at the same location as the fastq file? > Does this create a copy of the loaded file or simply read it in place? Also are there any efficiency issues created using this method, outside of the tool run time increase due to the load of the data taking place in-tool? It should just read it in place Hans-Rudolf > > Thanks, > Nathan > > -----Original Message----- > From: Hans-Rudolf Hotz [mailto:hrh@fmi.ch] > Sent: Tuesday, October 01, 2013 4:07 AM > To: Cole, Nathan (NIH/NCI) [C] > Cc: Martin Čech; galaxy-dev(a)lists.bx.psu.edu > Subject: Re: [galaxy-dev] Dynamic data library > > Hi Nathan > > > Do you have many tools working with those samples or just a few? If you only have a limited, predefined set of tools you might wanna consider adding the sample selection into the tool. > > You can use the from_file, or from_data_table options to dynamically create sample selection list. You can even drill down a hierarchical list. Have a look at ~/tools/annotation_profiler/annotation_profiler.xml > which uses the file > ~/tool-data/annotation_profiler_options.xml > > All you need to do is keeping the file in sync with the directory structure of your samples directory > > > Regards, Hans-Rudolf > > > > > > > On 09/30/2013 09:48 PM, Martin Čech wrote: >> Hi Nathan, >> >> Dannon answered similar question few days ago: >> >> There's an import mechanism in libraries that'll allow you to simply >> link to the file on disk without copy/upload. I believe the >> "example_watch_folder.py" sample script (in the distribution) does >> just this via the API, if you want an example. >> >> >> This might be what you are looking for. >> >> Martin >> >> >> On Mon, Sep 30, 2013 at 2:43 PM, Cole, Nathan (NIH/NCI) [C] >> <nathan.cole(a)nih.gov <mailto:nathan.cole@nih.gov>> wrote: >> >> Hello, we’ve set up a local Galaxy instance in our genotyping and >> next-gen sequencing lab with local Apache LDAP (AD) integration, NFS >> mounts to a large NAS, and cluster integration coming. Do to the >> high volume of samples and staff that will be using the system, I >> want to set up data libraries (without copying to Galaxy). This is >> obviously no problem the first time, however I was wondering if >> there was a way to make a library, added from a system path, be >> dynamic so that it would stay synchronized with the underlying file >> structure?____ >> >> __ __ >> >> If a try dynamic library is not possible, is there a method for >> adding files to an existing library via that same system path that >> would not duplicate all of the original files in the data library?____ >> >> __ __ >> >> I did some scouring of the list and found some old unanswered >> questions and some things tangentially related topics, but I was >> unable to find a true answer or solution to my problem. Any >> information on how to do the tasks above or other solutions to >> provide the same functionality would be greatly appreciated.____ >> >> __ __ >> >> Thanks,____ >> >> Nathan____ >> >> __ __ >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >>

2 2

how to generate .len file for custom genome? (trackster error)
by Ben Gift 02 Oct '13

02 Oct '13

Trackster complains to me when I try to load up my custom reference genome to compare some sample data to. "could not load chroms for this dbkey" I think it's because I'm using the newest g1k human ref genome v37, and I don't know if the hg19.len file works with it... I generated my own twobit file for it, but now I think I need the .len file... To be honest I'm kind of lost. Where does trackster even look for my genome, and how do I generate a .len file? (I read the visualizations page and the custom genome docs page)

2 1

Test post, please ignore
by Guest, Simon 01 Oct '13

01 Oct '13

(Just testing that my Exchange Server admin guy has managed to disable the stupid legal disclaimers on my postings to this list.)

1 0

configuring auth_ldap for galaxy
by Iry Witham 01 Oct '13

01 Oct '13

Hi Team, Has anyone had any experience with setting up auth_ldap for nginx? I am attempting to make this work, but nginx fails to start with the following message: Starting nginx nginx: [emerg] unknown directive "auth_ldap_url" in /usr/local/nginx/conf/nginx.conf:48 startproc: exit status of parent of /usr/local/nginx/sbin/nginx: 1 failed Any help would be appreciated. Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.

3 4

Local Galaxy Install - Debug Report FROM address as Admin and not User?
by Adam Brenner 01 Oct '13

01 Oct '13

Howdy, On our local Galaxy install, everytime a user submits a bug report, it does email all the admins -- as defined in universe_wsgi.ini -- but the FROM address is not of the user, but rather the first email address from the admin list. Is this normal? May we have it changed so the FROM address is that of the user? Thanks, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne(a)uci.edu

2 3

2 RNAseq files in queue to map w/Tophat in Main since last Thursday, please advise
by Theresa Stueve 01 Oct '13

01 Oct '13

Greetings, I have had two RNAseq files in queue to map with Top hat in Galaxy Main since last Thursday (History began Sept 19). Yesterday I got word that NGS was stalled, my jobs were mistakenly believed to be moving afterwards when they weren't- but are now under review again. This morning the 2 mapping jobs are still waiting to run. I have a data review meeting this thursday and hope you can advise me on contingency plans to get these 2 mapping jobs running. Can you tell me whether I should delete the jobs and re-run them as new? My concern is that they are still "believed to be moving" as was indicated by one of your galaxy stewards, but are actually still stalled in space. I have a data meeting Thursday (10/4) and hoped to be ready by starting this project Sept 19th. Can you tell me if you are aware of another public Galaxy server (other than Main) with 'Top Hat' for mapping RNAseq data? I joined Cistrome (Harvard) but they only have Cufflinks (for previously mapped RNAseq analysis) below is my history thank you very much for your help https://main.g2.bx.psu.edu/u/trstueve/h/rnaseqjegcsc3wkrun130924-1 <https://main.g2.bx.psu.edu/history/sharing#> Theresa Ryan Stueve 510-225-8894 (text better than voicemail for quick response)

1 0

Download URL not accepted in tool_dependencies.xml
by Joachim Jacob | VIB | 01 Oct '13

01 Oct '13

Hi all, The download URL seems not to be accepted in tool_dependencies.xml. It is an URL from sourceforge, the direct link to a package. The error upon uploading the tool to my toolshed: ************** Metadata may have been defined for some items in revision '5665a799775d'. Correct the following problems if necessary and reset metadata. tool_dependencies.xml - Exception attempting to parse /mnt/toolsheddb/database/000/repo_26/tool_dependencies.xml: not well-formed (invalid token): line 6, column 149 ************** The tool_dependencies.xml file: ************** 1 <?xml version="1.0"?> 2 <tool_dependency> 3 <package name="transpose" version="2.0.0"> 4 <install version="1.0"> 5 <actions> 6 <action type="download_by_url">http://downloads.sourceforge.net/project/transpose/transpose/transpose-2.0/…</action> 7 <action type="shell_command">mkdir bin</action> 8 <action type="shell_command">unzip transpose-2.0.zip</action> <action type="shell_command">cd transpose-2.0/src</action> <action type="shell_command">gcc transpose.c -o transpose</action> <action type="move_file"> <source>$INSTALL_DIR/transpose-2.0/src/transpose</source> <destination>$INSTALL_DIR/bin</destination> </action> <action type="shell_command">chmod +x $INSTALL_DIR/bin/transpose</action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> Compiling transpose and putting in the path. </readme> </package> </tool_dependency> ************** Cheers, Joachim -- Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team

5 10

Dynamic_options + DataToolParameter + refresh_on_change_values + multiple="true"
by Y. Hoogstrate 01 Oct '13

01 Oct '13

Dear Galaxy user, I'm working on a interface where the user can choose multiple data files using <param name="data_selection" type="data" multiple="true" />. Although I didn't find details about this in the manual, it does work. The next step in the interface are two select boxes (<param type="select" samples" multiple="true" dynamic_options="copy_selected_data_names(data_selection)" />) which allow further selection on the data chosen in "data_selection". I created a small python script that copies the entries selected in "data_selection" as follows: def copy_selected_data_names(DataToolParameter_multiple_object): items = [] if(DataToolParameter_multiple_object != NULL): if(type(DataToolParameter_multiple_object) == type([])): ## If it is a list for g in DataToolParameter_multiple_object: items.append([g.name,g.file_name,False]) else: ## If it isn't a list items.append([DataToolParameter_multiple_object.name,DataToolParameter_multiple_object.file_name,False]) return items I noticed that once only one single object is selected, the DataToolParameter returns directly one HistoryDatasetAssociation object. Once several are selected, it returns a list with HistoryDatasetAssociation objects. I think this is not correct; the DataToolParameter is a object where multple is set to "true" so it should always return a list even if none or 1 object is selected. Nevertheless, this part of the code works. However, the update of the dynamic_options="copy_selected_data_names(data_selection)" only takes place whenever new objects (in repeat blocks) are added or removed elsewhere in the interface. Thus, the copying of the content doesn't take place when the selection has changed. My question is: is there a way to make dynamic_options="" update whenever the selection in "data_selection" has changed? I checked the code and I found some variables in the DataToolParameter class that may have something to do with it: self.refresh_on_change = False self.refresh_on_change_values = [] But I'm not able to solve the puzzle on my own. Is there anyone who can help in this direction? Thanks in advance!

1 0

Dynamic data library
by Cole, Nathan (NIH/NCI) [C] 01 Oct '13

01 Oct '13

Hello, we've set up a local Galaxy instance in our genotyping and next-gen sequencing lab with local Apache LDAP (AD) integration, NFS mounts to a large NAS, and cluster integration coming. Do to the high volume of samples and staff that will be using the system, I want to set up data libraries (without copying to Galaxy). This is obviously no problem the first time, however I was wondering if there was a way to make a library, added from a system path, be dynamic so that it would stay synchronized with the underlying file structure? If a try dynamic library is not possible, is there a method for adding files to an existing library via that same system path that would not duplicate all of the original files in the data library? I did some scouring of the list and found some old unanswered questions and some things tangentially related topics, but I was unable to find a true answer or solution to my problem. Any information on how to do the tasks above or other solutions to provide the same functionality would be greatly appreciated. Thanks, Nathan

3 2

Problem with get data/uploading file.
by Luobin Yang 01 Oct '13

01 Oct '13

Hi, guys, Recently I found the Upload File tool under the Get Data category runs forever on my locally installed Galaxy system. It worked fine for a long time, but now it's not working anymore. Have you guys experienced similar things? Thanks, Luobin

2 1