March 2012 - galaxy-dev - lists.galaxyproject.org

[Solved] Galaxy and Cluster options
by Tanguy LE CARROUR 07 Mar '12

07 Mar '12

Hello! Sorry but I'm not able to answer the e-mail in the thread as I wasn't subscribed to the mailing list! But I now am. Thanks to L-A, Carlos and Chris for their answers! I can live with the config based tool configuration... but a dynamic one would be nice to have! Cheers, Tanguy

1 0

Re: [galaxy-dev] [galaxy-user] Problem using Galaxy on the cloud
by Jennifer Jackson 07 Mar '12

07 Mar '12

Hi Jan, Since this question is about a Cloud installation, I am going to forward your question over to the galaxy-dev(a)bx.psu.edu mailing list so that the development community will have a better chance of seeing it and providing feedback. http://wiki.g2.bx.psu.edu/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/6/12 1:44 PM, Jan R McDowell wrote: > Hi all, > I have been trying to get an instance of Galaxy going on the EC2. I have no problem going through BioCloudCentral and getting an instance going. I can also successfully load my data from an S2 bucket into Galaxy. The problem occurs when I try to use velveth. I always says 'job waiting to run'. As a matter of curiosity, I then used SSH to get into CloudBioLinux, which worked. However, when I try to use NX to get the virtual desktop going I get the message usr/bin/nxserver: line 381: echo: write error: No space left on device. > > Using the df -h command, I get: > > Filesystem Size Used Avail Use% Mounted on > /dev/xvda1 20G 19G 0 100% / > udev 8.4G 4.0K 8.4G 1% /dev > tmpfs 3.4G 660K 3.4G 1% /run > none 5.0M 0 5.0M 0% /run/lock > none 8.4G 0 8.4G 0% /run/shm > /dev/xvdb 404G 202M 383G 1% /mnt > /dev/xvdg1 700G 654G 47G 94% /mnt/galaxyIndices > /dev/xvdg2 10G 1.7G 8.4G 17% /mnt/galaxyTools > /dev/xvdg3 200G 11G 190G 6% /mnt/galaxyData > > > So, I guess my question as a new user is: How do I point Galaxy and CloudBioLinux to all of this unused space? I assume the problem is with the /dev/xvda1 that is 100% full. I am obviously doing something silly and/or missing a really big step. Any help would be greatly appreciated. > > Many thanks in advance, > Jan > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

2 1

Re: [galaxy-dev] [galaxy-user] Advice on disabling SGE queue on galaxy-cloudman
by Jennifer Jackson 07 Mar '12

07 Mar '12

Hi John, Since this question is about a Cloud installation, I am going to forward your question over to the galaxy-dev(a)bx.psu.edu mailing list so that the development community will have a better chance of seeing it and providing feedback. http://wiki.g2.bx.psu.edu/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/6/12 11:23 AM, John Major wrote: > Hello All- > > I'd like to launch a galaxy-cloudman head node which does not accept SGE > jobs, but as jobs are submitted go to compute nodes (or cause compute > node to be added when auto-scale is on). > Primarily, this is b/c I'd like to have the head node be a cheaper > instance which can run long term, and only fire up more expensive > compute nodes as they are actually needed. > > How would I enable this? > > Thanks- > John > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

1 0

Possible Galaxy memory leak?
by Joshua Gross 06 Mar '12

06 Mar '12

Hi all-- Our group is running a local instance of Galaxy on a Linux box. We have been noticing a memory issue, where if we leave the server running for a few days, soon all available memory (~8gb) will be used up to the point where the tools fail because they are unable to allocate new memory. While we haven't done any rigorous testing, this issue seems to be independent of the specific tools we run, and actually seems to still appear if we do not run any tools at all. The process "python" is what is taking up all the memory, and the issue is not instantaneous but happens over time. Restarting the server frees the memory immediately. Is this a known issue? If so, is there a known fix for it? Any insight on this would be much appreciated, as it is complicating our efforts to automate long work-flows. Thank you so much for your time Josh Gross, Western Washington University.

4 3

Re: [galaxy-dev] [galaxy-user] Getting errors trying to enable samtools_mpileup
by Nate Coraor 06 Mar '12

06 Mar '12

On Mar 1, 2012, at 8:49 AM, Waldron, Michael H wrote: > I am running a local copy of Galaxy, last ran 'hg pull -u' on 2/20/12. > > I am trying to enable use of Mpileup for SAM Tools, and have added the entry for the samtools_mpileup.xml file in tool_conf.xml. However, when I startup Galaxy, the link for Mpileup does not appear in the Tools pane, and I see the following errors from the Galaxy startup: > > galaxy.tools ERROR 2012-02-29 16:28:51,919 error reading tool from path: samtools/samtools_mpileup.xml > Traceback (most recent call last): > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 212, in load_tool_tag_set > tool = self.load_tool( os.path.join( tool_path, path ), guid=guid ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 305, in load_tool > return ToolClass( config_file, root, self.app, guid=guid ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 531, in __init__ > self.parse( root, guid=guid ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 641, in parse > self.parse_inputs( root ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 723, in parse_inputs > display, inputs = self.parse_input_page( page, enctypes ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 922, in parse_input_page > inputs = self.parse_input_elem( input_elem, enctypes ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 989, in parse_input_elem > case.inputs = self.parse_input_elem( case_elem, enctypes, context ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 945, in parse_input_elem > group.inputs = self.parse_input_elem( elem, enctypes, context ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 1015, in parse_input_elem > param = self.parse_param_elem( elem, enctypes, context ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/__init__.py", line 1027, in parse_param_elem > param = ToolParameter.build( self, input_elem ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 176, in build > return parameter_types[param_type]( tool, param ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 1330, in __init__ > ToolParameter.__init__( self, tool, elem ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/parameters/basic.py", line 43, in __init__ > self.validators.append( validation.Validator.from_element( self, elem ) ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/parameters/validation.py", line 23, in from_element > return validator_types[type].from_element( param, elem ) > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/parameters/validation.py", line 279, in from_element > tool_data_table = param.tool.app.tool_data_tables[ table_name ] > File "/nas02/apps/galaxy-prod/galaxy-dist/lib/galaxy/tools/data/__init__.py", line 21, in __getitem__ > return self.data_tables.__getitem__( key ) > KeyError: 'sam_fa_indexes' > > Can someone tell me what is wrong here? Am I missing something in the database? Hi Mike, I moved this over to the galaxy-dev list since it pertains to a local installation. You'll need to update your tool_data_table_conf.xml from the sample. If you haven't made any modifications, the simplest solution is to just `cp tool_data_table_conf.xml.sample tool_data_table_conf.xml`. --nate > > Thanks, > > Mike Waldron > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0

Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
by Ann Black 06 Mar '12

06 Mar '12

Good Morning, We are also interested in this capability. To give a concrete example, we sometimes receive multiple sequence runs 1* for the same sample. We would like to be able to process each run of the sample through BWA and then merge them together, post process it a bit, and then send the merged bam file through the rest of our standard pipeline. Ideally this would be automated. Aurélien, I am interested in your workaround this might get us part of the way there, as we could concatenate the fastq files together and run BWA once. Would you be willing to share some of your custom tools for us to iterate on? But ideally, for performance, we would run the bwa steps in parallel. Is the galaxy team looking into these types of features or do other people have custom solutions they are using? Thanks so much, Ann

2 1

can the sample tracking system transfer (copy) data to a dataset library not in private Galaxy server data directory tree and then just link files?
by Leandro Hermida 06 Mar '12

06 Mar '12

Dear all, Is there a way with the Galaxy sample tracking system to have the dataset transfer feature copy data to a dataset library outside the private Galaxy server data directory tree and then the dataset library just link (like you can do with upload files feature when managing data libraries)? The reason for this is that it is common to have such huge data also available outside of Galaxy on the filesystem for bioinformatians/computational biologists to access directly without having to have two copies. regards, Leandro

2 1

Re: [galaxy-dev] [galaxy-user] Data not uploading in Local instance
by Nate Coraor 06 Mar '12

06 Mar '12

Hi Luciano, I've moved this over to galaxy-dev since it concerns a local installation. On Feb 26, 2012, at 6:19 PM, Luciano Cosme wrote: > I have python 2.7 and it seems to be the problem with it. Just formated the partition I had linux and fresh install everything and still have same problem. I did what is recommended here too. We use Python 2.7 in development and on the public site. If there is a specific problem with 2.7, could you provide details on what it is? > Check your Python version > > Galaxy's core functionality is compatible with Python versions 2.5, 2.6, and 2.7. (Version 2.4 support was last included in the November 18, 2011 Distribution). Before reporting bugs, please ensure that python --version reports one of these three versions. If this is not the case, a simple solution is to manipulate your shell's $PATH variable to place the correct version first. This can be done for only Python by creating a new directory at the front of $PATH and creating a symbolic link to python in that directory: > > > Toggle line numbers > 1 % mkdir ~/galaxy-python > 2 % ln -s /path/to/python2.5 ~/galaxy-python/python > 3 % export PATH=~/galaxy-python:$PATH > > > > > > On Sat, Feb 25, 2012 at 5:50 PM, Luciano Cosme <cosme.simple(a)gmail.com> wrote: > Hi Everyone, > I have been running a Local Galaxy instance for a long time, but now I am having some problems with uploading data. I am getting the following error: > Can't create peek [Errno 2] No such file or directory: '/home/koala2/galaxy-dist/database/files/000/dataset_13.dat' Can you make sure that you are starting with an empty /home/koala2/galaxy-dist/database/files/ directory? Is that directory writable by the user running Galaxy? The problem is that the tool runs and Galaxy does not detect failure (i.e. something output to STDERR) and yet the tool's output file does not exist. --nate > > > > > and from the Terminal: > 127.0.0.1 - - [25/Feb/2012:17:45:00 -0500] "POST /library_common/library_item_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/library_common/browse_library? > > > > sort=name&operation=browse&f-description=All&f-name=All&f-deleted=False&cntrller=library_admin&async=false&show_item_checkboxes=false&webapp= > > > > galaxy&id=f2db41e1fa331b3e&page=1" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2" > > > > > > I reinstalled Unbutu and still having the same issue. I also removed Postgresql 9.1 and reinstalled it. I that it might be an issue with postgresql and not Galaxy, but I am not an expert. > Any suggestions will be appreciated. > > > Thank you. > > > Luciano > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0

Re: [galaxy-dev] Using complete workflow in another workflow
by Ann Black 06 Mar '12

06 Mar '12

Hello - I would also like the +1 this feature. I currently use the "clone" feature of a workflow to make a copy and modify, but if I change the original workflow, I think have to remember which workflows I cloned into and hand modify them as well. If we had the ability to re-use workflows, then we could develop modules that could be re-used in larger workflows. Thanks much for the consideration, Ann > >------------------------------ > >Message: 3 >Date: Thu, 01 Mar 2012 20:14:25 +0000 (GMT) >From: thondeboer(a)me.com >To: Galaxy-dev <galaxy-dev(a)lists.bx.psu.edu> >Subject: [galaxy-dev] Using complete workflow in another workflow >Message-ID: <7d9b3791-1822-e517-b5b9-e1ba1579b99e(a)me.com> >Content-Type: text/plain; charset="utf-8"; Format="flowed" > > >Hi, > >How far along are we about thinking about being able to re-use a complete >workflow as a workflow step in another workflow? > >This would really allow us to modularize certain aspects of the analyses >and would allow us to re-use a workflow in another. > >Barring that, a simple copy/paste from one workflow into another would >also really help... > >Any plans in that direction? > >Thanks > >Thon? >

2 1

Variable number of inputs in Workflow (or optional inputs)
by Jennifer Jackson 06 Mar '12

06 Mar '12

Hello Daniel, Examining existing tools that have multiple/optional inputs can be a good way to see how this is done. One example is the tool: "NGS: QC and manipulation -> Manipulate FASTQ". The 'Manipulate FASTQ' tool source is here: http://bitbucket.org/galaxy/galaxy-central/src/e58a87c91bc4/tools/fastq/ fastq_manipulation.py fastq_manipulation.xml The tool tag set is defined in this wiki; http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax See <repeat>, <conditional>, and <when> for more details and examples. Questions about tools are best sent to the galaxy-dev(a)bx.psu.edu mailing list (the galaxy-user list is primarily for questions about data/tools usages on the main public instance). I am going to forward your question over there so that the development community can add to my reply in case they have a simpler way of doing this or other advice. Best, Jen Galaxy team On 3/5/12 7:32 AM, Daniel Sobral wrote: > Hi, > > I wanted to build a workflow where the first step would be a tool that > can have a variable number of inputs (e.g. a series). > It seems that I need to predefine apriori the number of inputs to give > to the tool. > > Is there a way to define the number of inputs at runtime? > E.g. a series-like input? > > Thanks, > Daniel > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support

3 3