- galaxy-dev - lists.galaxyproject.org

lims integration
by Craig Blackhart 02 Dec '11

02 Dec '11

I am newish to Galaxy and trying to learn how I might integrate it with our workflows and LIMS for automated data handling. I am aware of the API and have looked up all the documentation that I could find. However, there are many things I cannot make sense of, and have not been able to find information to help me out. I think a good place to start asking questions is with how to run workflow_execute.py and ask what each of the parameters are and where to get the information from them Arguments *API key - got this and understand *url - got this and understand *workflow_id - I have created workflows and have been able to find what looks to be a workflow_id by clicking on the workflow name and selecting "Download or Export". It seems this may be correct, is it? *history - a named history to use? Should this already exist? I have no idea here. *step=src=dataset_id - ??? I have no idea ??? I have seen how to create data libraries manually at the command line; does this factor in? If anyone has information they can help me out with, it would be much appreciated. Thanks Craig Blackhart Computer Scientist Applied Engineering Technologies Los Alamos National Laboratory 505-665-6588 This message contains no information that requires ADC review

3 2

Job output not returned from cluster
by Joseph Hargitai 02 Dec '11

02 Dec '11

Hi, i was browsing through the list and found many entries for this issue but not a definite answer. We are actually running into this error for simple file uploads from the internal filesystem. thanks, joe

5 9

synchronous data depositing
by James Ireland 01 Dec '11

01 Dec '11

Greetings, I've been attempting to return data to Galaxy via the synchronous data depositing protocol. Using the Biomart, UCSC Table Browser, etc as examples in the data_source tools directory, I've been able to get the initial GET request to my site just fine. However, when I POST back to galaxy I immediately get a redirect to the welcome page and Galaxy never resubmits back to my site. I was wondering if there is more to the protocol than is covered here: http://wiki.g2.bx.psu.edu/Admin/Internals/Data%20Sources or perhaps configuration I need to perform on my local Galaxy installation to correctly handle the POSTs back to tool_runner? Also, are there any code examples I should be looking at? Thanks for your help! -James -- J Ireland www.5amsolutions.com | Software for Life(TM) m: 415 484-DATA (3282)

2 8

Re: [galaxy-dev] Local installation of the tool shed
by Greg Von Kuster 01 Dec '11

01 Dec '11

Hello Louise-Amelie, On Dec 1, 2011, at 4:08 AM, Louise-Amélie Schmitt wrote: > Yes this fixes the problem, it should work fine now :) > > Thanks a lot! > > Hehe, next issues: > > 1) When I have my repo created and I click on the "Upload files to repository", I get a Not found error in the browser: > Not Found > The requested URL /toolshed/upload/upload was not found on this server. This is probably due to your apache rewrite rule: RewriteRule ^/toolshed/upload/(.*) /home/galaxy/galaxy-dev/static/automated_upload/$1 [L] Is this something proprietary you have set up on your local galaxy instance? Try removing it from your rewrite rules for your local tool shed. > > 2) When we try to access our local toolshed from our Galaxy instance, it appears in the "Accessible tool sheds" list along with your two public repos, but when we click on it, the "Valid repositories" list is empty. Is this a bug or does a repo have to actually contain files so it can appear in this list? The list of valid repositories will only include repositories that have content that is valid for a Galaxy instance. These repositories are defined as either "valid" or in some cases "downloadable" (I'm working to replace the latter with the former). See the following sections of the tool shed wiki for details: http://wiki.g2.bx.psu.edu/Tool%20Shed#Repository_revisions:_downloadable_to… http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_tool… > > 3) Where should the hgweb.config be? This file should be left in the Galaxy root install directory. > Actually, we have absolute paths since in the community_wsgi.ini we set the file_path option to an absolute path. I advise against this - I'm fairly certain it will pose problems at some point. Tool shed paths should be relative. Here are some example entries in only of my local tool sheds: [paths] repos/test/filter = database/community_files/000/repo_1 repos/test/workflow_with_tools = database/community_files/000/repo_2 repos/test/heteroplasmy_workflow = database/community_files/000/repo_3 > > Thanks for your patience! > L-A > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Re: [galaxy-dev] [galaxy-user] Using Galaxy Cloudman for a workshop
by Clare Sloggett 01 Dec '11

01 Dec '11

Hi Jeremy, Enis, That makes sense. I know I can configure how many threads BWA uses in its wrapper, with bwa -t. But, is there somewhere that I need to tell Galaxy the corresponding information, ie that this command-line task will make use of up to 4 cores? Or, does this imply that there is always exactly one job per node? So if I have (for instance) a cluster made of 4-core nodes, and a single-threaded task (e.g. samtools), are the other 3 cores just going to waste or will the scheduler allocate multiple single-threaded jobs to one node? I've cc'd galaxy-dev instead of galaxy-user as I think the conversation has gone that way! Thanks again, Clare On Fri, Nov 18, 2011 at 2:36 PM, Jeremy Goecks <jeremy.goecks(a)emory.edu> wrote: > >> On Fri, Nov 18, 2011 at 12:56 AM, Jeremy Goecks <jeremy.goecks(a)emory.edu> wrote: >> >>> Scalability issues are more likely to arise on the back end than the front end, so you'll want to ensure that you have enough compute nodes. BWA uses four nodes by default--Enis, does the cloud config change this parameter?--so you'll want 4x50 or 200 total nodes if you want everyone to be able to run a BWA job simultaneously. >>> >> >> Actually, one other question - this paragraph makes me realise that I >> don't really understand how Galaxy is distributing jobs. I had thought >> that each job would only use one node, and in some cases take >> advantage of multiple cores within that node. I'm taking a "node" to >> be a set of cores with their own shared memory, so in this case a VM >> instance, is this right? If some types of jobs can be distributed over >> multiple nodes, can I configure, in Galaxy, how many nodes they should >> use? > > You're right -- my word choices were poor. Replace 'node' with 'core' in my paragraph to get an accurate suggestion for resources. > > Galaxy uses a job scheduler--SGE on the cloud--to distribute jobs to different cluster nodes. Jobs that require multiple cores typically run on a single node. Enis can chime in on whether CloudMan supports job submission over multiple nodes; this would require setup of an appropriate parallel environment and a tool that can make use of this environment. > > Good luck, > J. > > > > -- E: sloc(a)unimelb.edu.au P: 03 903 53357 M: 0414 854 759

2 5

Lost my Data Libraries when I updated to the lastest galaxy-dist release?
by Liisa Koski 30 Nov '11

30 Nov '11

Hi, I lost my Data Libraries when I updated to the latest galaxy-dist release :( What would be the best way to go about getting them back? Thanks in advance, Liisa

2 2

Local installation of the tool shed
by Nicolas Delhomme 30 Nov '11

30 Nov '11

Hi Greg, I've been scanning the mailing list, but wasn't lucky enough to find the answer I was looking for. Basically, we would be interested to have our own Galaxy Tool Shed in-house, to develop and test tools within our local Galaxy instance. What I'm thinking of, is to use the power of the tool shed to develop new algorithm, new tools, etc. that could be published afterwards. Because these would be sensitive material, we would not want to put them right away on the public test Tool Shed. Having a local Tool Shed instance would still be very helpful in releasing these tools to the community afterwards, as they would have been integrated and developed within that framework from the start. Any pointers on how to achieve this are welcome as I'm not so familiar with the Tool Shed yet, e.g. would making a local clone of the tool shed repository be enough? Thanks, Nico --------------------------------------------------------------- Nicolas Delhomme Genome Biology Computational Support European Molecular Biology Laboratory Tel: +49 6221 387 8310 Email: nicolas.delhomme(a)embl.de Meyerhofstrasse 1 - Postfach 10.2209 69102 Heidelberg, Germany ---------------------------------------------------------------

3 21

Re: [galaxy-dev] Toolshed not accessible
by Greg Von Kuster 30 Nov '11

30 Nov '11

You should not have to change things in your ini file if using the main Galaxy tool shed. On Nov 30, 2011, at 10:49 AM, Sajoscha Sauer wrote: > Now I'm confused :) > > I'm trying to use the Galaxy main toolshed (http://toolshed.g2.bx.psu.edu/) not a local one. I guess I don't have to change something in my ini file, right? > > We also updated to the revision 834de698004e > > Thanks, > Sajoscha > > On Nov 30, 2011, at 3:36 PM, Greg Von Kuster wrote: > >> I've recently added the following section to the tool shed wiki. Since you are using apache, make sure you have all of the necessary settings in community_wsgi.ini. Let me know if this is not the issue, and I'll need to see a complete paste log showing the exceptions as well as a copy of your community_wsgi.ini file. >> >> thanks, >> >> Greg Von Kuster >> >> http://wiki.g2.bx.psu.edu/Tool%20Shed#Hosting_your_own_local_tool_shed >> >> >> On Nov 30, 2011, at 7:22 AM, Sajoscha Sauer wrote: >> >>> Hi Greg, >>> >>> Sorry for the late reply! >>> >>> Yes, we see the error when trying to install a tool from the Galaxy tool sheds ('Install to local Galaxy') >>> >>> We are now back on yesterdays revision (3ee9430186fb) >>> >>> >>> Thanks, >>> Sajoscha >>> >>> On Nov 23, 2011, at 5:34 PM, Greg Von Kuster wrote: >>> >>>> Hi Sajoscha, >>>> >>>> What were you attempting to do when you saw this error? Are you still seeing this when you attempt to install a tool from the tool shed? By the way,m I just sent you an email - there are a few more fixes in the latest change set. >>>> >>>> >>>> On Nov 23, 2011, at 11:26 AM, Sajoscha Sauer wrote: >>>> >>>>> Hi Greg, >>>>> >>>>> Thanks for the quick response! >>>>> >>>>> We updated to the newest version from the Galaxy central repository, but still get the same error. But now there is a different error (or better, a different referer) in the apache log, see below. >>>>> >>>>> <Screen Shot 2011-11-23 at 5.11.19 PM.png> >>>>> >>>>> >>>>> >>>>> Thanks again for your help! >>>>> >>>>> >>>>> Sajoscha >>>>> >>>>> >>>>> On Nov 22, 2011, at 4:22 PM, Sajoscha Sauer wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> We are just about to enable the Toolshed on our development installation of Galaxy. >>>>>> >>>>>> We run the latest build (rev. b258de1e6cea) of Galaxy. >>>>>> >>>>>> >>>>>> The toolshed that we try to access is 'http://toolshed.g2.bx.psu.edu/' >>>>>> >>>>>> Our workflow is the following: >>>>>> - On the admin panel, click 'Search and browse tool sheds' >>>>>> - 'Browse valid repositories' on 'Galaxy main tool shed' >>>>>> - Select any tool and 'Preview and install' >>>>>> - 'Install to local Galaxy' >>>>>> >>>>>> >>>>>> What we see on the website is: >>>>>> >>>>>> <Screen Shot 2011-11-22 at 4.13.56 PM.png> >>>>>> >>>>>> The apache error log says: >>>>>> >>>>>> <Screen Shot 2011-11-22 at 4.06.34 PM.png> >>>>>> >>>>>> >>>>>> Any ideas and/or hints? >>>>>> >>>>>> >>>>>> Thanks a lot for your help! >>>>>> >>>>>> Sajoscha >>>>>> ___________________________________________________________ >>>>>> Please keep all replies on the list by using "reply all" >>>>>> in your mail client. To manage your subscriptions to this >>>>>> and other Galaxy lists, please use the interface at: >>>>>> >>>>>> http://lists.bx.psu.edu/ >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> >>>> Greg Von Kuster >>>> Galaxy Development Team >>>> greg(a)bx.psu.edu >>>> >>>> >>>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Web links not leading to appropriate location anymore.
by Louise-Amélie Schmitt 30 Nov '11

30 Nov '11

Hello, We're experiencing a major issue in the web application and don't know where to look for a potential solution: Whichever link we click on it always brigs us back to the welcome page (galaxy/root). There are a few exceptions to that: - saved histories, saved datasets and api key in the User menu that lead to the appropriate pages - the tools on the left: even better, they open an new full set of Galaxy's frames in the middle frame. We tried reverting to the last pull, restarting Apache, checking the Apache config file, checking the rights in the static directory... and we still do not have the least idea of where to look. Does anyone have an idea, a lead to explore? Thanks, L-A

3 5

Toolshed not accessible
by Sajoscha Sauer 30 Nov '11

30 Nov '11

Hi all, We are just about to enable the Toolshed on our development installation of Galaxy. We run the latest build (rev. b258de1e6cea) of Galaxy. The toolshed that we try to access is 'http://toolshed.g2.bx.psu.edu/' Our workflow is the following: - On the admin panel, click 'Search and browse tool sheds' - 'Browse valid repositories' on 'Galaxy main tool shed' - Select any tool and 'Preview and install' - 'Install to local Galaxy' What we see on the website is: The apache error log says: Any ideas and/or hints? Thanks a lot for your help! Sajoscha

2 6