- galaxy-dev - lists.galaxyproject.org

Converting .gff3 to 12-column .bed
by lrutter ＠iastate.edu 13 Nov '13

13 Nov '13

Hello Galaxy: I am trying overall to convert a .gff3 file to 12-column .bed file. I first tried GFF-to-BED converter, but it gave a 6-column .bed file. Then, I tried BED-to-bigBed converter by inputting the 6-column .bed file. I get an error "Unspecified genome build, click the pencil icon in the history item to set the genome build". So, I click the pencil icon, and see 4 tabs at the top. I set the "Attributes" tab as in the attached image (Attributes.png). But then, when I select "Convert Format", I am only seeing an option that outputs .bed12 file as "Convert Genomic Intervals to Strict BED12". I am a bit confused about this because I specified the input file as a .bed file (and not genomic intervals, unless I am misunderstanding something). In any case, when I select "Convert Genomic Intervals to Strict BED12", I do get a .bed file with 12 columns. But I would like to ask if I may have lost information going from the .gff3 to .bed(6) to .bed(12)? (I feel that scores were all set to "0" from .gff3 to .bed(6), and columns 10, 11, 12 (block counts, sizes, and starting positions) were all set to zero going from .bed(6) to .bed(12)). If I am correct that there is information loss, is there a system in Galaxy to prevent this, and transfer as much information as possible from .gff3 to .bed(12)? Thank you. L. Rutter ** Below is a head of my three files (the species is P. dominula): .gff3 file ##gff-version 3 ##date Mon Nov 4 14:54:42 2013 ##source gbrowse gbgff gff3 dumper PdomScaf0001 maker gene 15 1963 . - . Name=PdomGene00025;ID=1;Dbxref=MAKER:maker-PdomScaf0001-snap-gene-0.274 PdomScaf0001 maker mRNA 15 1963 . - . Name=PdomMRNA00025.1;Parent=1;ID=2;_QI=216%7C0%7C0.2%7C0.6%7C0.5%7C0.6%7C5%7C0%7C98;_eAED=0.43;_AED=0.43;Dbxref=MAKER:maker-PdomScaf0001-snap-gene-0.274-mRNA-1 PdomScaf0001 maker exon 15 100 -0.094 - . Parent=2;ID=3 PdomScaf0001 maker CDS 15 100 . - 2 Parent=2;ID=4 PdomScaf0001 maker exon 223 300 21.8 - . Parent=2;ID=5 PdomScaf0001 maker CDS 223 300 . - 2 Parent=2;ID=6 PdomScaf0001 maker exon 717 765 22.4 - . Parent=2;ID=7 .bed(6) file PdomScaf0001 14 1963 gene 0 - PdomScaf0001 14 1963 mRNA 0 - PdomScaf0001 14 100 exon 0 - PdomScaf0001 14 100 CDS 0 - PdomScaf0001 222 300 exon 0 - PdomScaf0001 222 300 CDS 0 - PdomScaf0001 716 765 exon 0 - PdomScaf0001 716 765 CDS 0 - PdomScaf0001 906 947 exon 0 - PdomScaf0001 906 947 CDS 0 - .bed(12) file PdomScaf0001 14 1963 gene 0 - 14 1963 0 0 , , PdomScaf0001 14 1963 mRNA 0 - 14 1963 0 0 , , PdomScaf0001 14 100 exon 0 - 14 100 0 0 , , PdomScaf0001 14 100 CDS 0 - 14 100 0 0 , , PdomScaf0001 222 300 exon 0 - 222 300 0 0 , , PdomScaf0001 222 300 CDS 0 - 222 300 0 0 , , PdomScaf0001 716 765 exon 0 - 716 765 0 0 , , PdomScaf0001 716 765 CDS 0 - 716 765 0 0 , , PdomScaf0001 906 947 exon 0 - 906 947 0 0 , , PdomScaf0001 906 947 CDS 0 - 906 947 0 0 , ,

4 3

Re: [galaxy-dev] dynamic_options selected value as input for second dynamic_options function call
by Hans van Leeuwen 12 Nov '13

12 Nov '13

Hi, I found a working workaround using the deprecated <page> tags: <inputs> <page> <param name = "State_id" type="select" dynamic_options="getStates()" label="Select State" display="radio" multiple="false"/> </page> <page> <param name = "Town_id" type="select" dynamic_options="getTowns(State_id)" label="Select Town" display="radio" multiple="false"/> </page> </inputs> This works; both select lists (on separate GUI pages) are dynamically generated when the called function connects to a database to retrieve the options. However, as it is recommended not to use <page> tags any longer, can anyone suggest how to write above example using <condition> and <when> tags instead? I have searched a lot but can't find out how to do it. Thank you, Hans

1 0

Error editing users associated with a quota
by Peter Cock 12 Nov '13

12 Nov '13

Hi all, I'm trying to remove a user from our default quota (while they run a large analysis), but am getting an error: Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. Has anyone else seen this? Thanks, Peter $ hg branch stable [galaxy@ppserver galaxy-dist]$ hg head | more changeset: 11219:5c789ab4144a branch: stable tag: tip user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Nov 04 15:04:42 2013 -0500 summary: Added tag release_2013.11.04 for changeset 26f58e05aa10 changeset: 11216:c458a0fe1ba8 parent: 11213:6d633418ecfa parent: 11215:f79149dd3d35 user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Nov 04 14:56:57 2013 -0500 summary: Merge security fix for filtering tools from stable/next-stable. >From the pasteur.log file, xxx.xxx.xxx.xxx - - [11/Nov/2013:12:04:45 +0100] "POST /galaxy/admin/manage_users_and_groups_for_quota?id=7a72206f0ccffc2e HTTP/1.1" 500 - "http://xxx/galaxy/admin/manage_users_and_groups_for_quota?webapp=galaxy&id=…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0" Error - <type 'exceptions.AttributeError'>: 'NoQuotaAgent' object has no attribute 'set_entity_quota_associations' URL: http://ppserver/galaxy/admin/manage_users_and_groups_for_quota?id=7a72206f0… File '/mnt/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 229 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin.py', line 531 in manage_users_and_groups_for_quota quota, params = self._quota_op( trans, 'quota_members_edit_button', self._manage_users_and_groups_for_quota, kwd ) File '/mnt/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin.py', line 664 in _quota_op message = op_method( quota, params ) File '/mnt/galaxy/galaxy-dist/lib/galaxy/actions/admin.py', line 80 in _manage_users_and_groups_for_quota self.app.quota_agent.set_entity_quota_associations( quotas=[ quota ], users=in_users, groups=in_groups ) AttributeError: 'NoQuotaAgent' object has no attribute 'set_entity_quota_associations' CGI Variables ------------- CONTENT_LENGTH: '-1' CONTENT_TYPE: 'application/x-www-form-urlencoded' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-gb,en;q=0.7,en-us;q=0.3' HTTP_CONNECTION: 'Keep-Alive' HTTP_COOKIE: 'galaxysession=fed9265fae01dfd85c30fe1bbf5a988ae27edcb17a4961e7cef7bef77b0eeb9be91a902d5f0aacdf' HTTP_DNT: '1' HTTP_HOST: 'ppserver' HTTP_REFERER: 'http://ppserver/galaxy/admin/manage_users_and_groups_for_quota?webapp=galax…' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0' PATH_INFO: '/admin/manage_users_and_groups_for_quota' QUERY_STRING: 'id=7a72206f0ccffc2e' REMOTE_ADDR: '143.234.97.120' REQUEST_METHOD: 'POST' SCRIPT_NAME: '/galaxy' SERVER_NAME: 'xxx' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x910c890> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='fed9265fae01dfd85c30fe1bbf5a988ae27edcb17a4961e7cef7bef77b0eeb9be91a902d5f0aacdf'>, 'galaxysession=fed9265fae01dfd85c30fe1bbf5a988ae27edcb17a4961e7cef7bef77b0eeb9be91a902d5f0aacdf') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x783fd50> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x30eea90> paste.parsed_querystring: ([('id', '7a72206f0ccffc2e')], 'id=7a72206f0ccffc2e') paste.recursive.forward: <paste.recursive.Forwarder from /galaxy> paste.recursive.include: <paste.recursive.Includer from /galaxy> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /galaxy> paste.recursive.script_name: '/galaxy' paste.throw_errors: True request_id: '744ee0cc4ac911e3bbcc0026b95a4d69' webob._parsed_post_vars: (MultiDict([('in_users', '22'), ('in_users', '6'), ('in_users', '12'), ('in_users', '14'), ('in_users', '15'), ('in_users', '4'), ('in_users', '5'), ('in_users', '20'), ('in_users', '9'), ('in_users', '16'), ('in_users', '11'), ('in_users', '7'), ('in_users', '17'), ('in_users', '23'), ('in_users', '19'), ('in_users', '10'), ('in_users', '32'), ('out_users', '21'), ('quota_members_edit_button', 'Save')]), <FakeCGIBody at 7f3708159910 viewing MultiDict([('in...e')])>) webob._parsed_query_vars: (MultiDict([('id', '7a72206f0ccffc2e')]), 'id=7a72206f0ccffc2e') wsgi process: 'Multithreaded' ------------------------------------------------------------

2 4

dynamic_options selected value as input for second dynamic_options function call
by Hans van Leeuwen 12 Nov '13

12 Nov '13

Hello, I would like to have a tool form with a two parameters: 1. Select State 2. Select Town The options for 'Select State' are dynamically generated with dynamic_options which calls a python function, which connects to a database and returns the list of States. This works fine in my form xml code: <param name = "State_id" type="select" dynamic_options="getStates()" > label="Select State" display="radio" multiple="false"> Now I want to dynamically create the options for '2. Select Town'. To make this work I need to take the selected option from '1. Select State', pass it as an argument to a dynamic_options function call that would then connect to a database and retrieve the Towns for the provided State. These Towns would then be the options for '2. Select Town'. So, the second parameter '2. Select Town' should do something like this: <param name = "Town_id" type="select" dynamic_options="getTowns([user > selected value in 1. Select State])" label="Select Town" display="radio" > multiple="false"> The part in square brackets is pseudo code; this is the part that I cannot get to work . That part should also be automatically refreshed if the user changes the selection in '1. Select State'. I looked into the use of <conditional> but I don't see how I can use that if I don't know beforehand what values I can have (determined in real time based on content of database), so I cannot code the <when> tags. Any help is greatly appreciated. Thank you in advance, Hans

1 0

SLURM and hidden success
by Andrew Warren 11 Nov '13

11 Nov '13

Hello all, We are in the process of switching from SGE to SLURM for our galaxy setup. We are currently experiencing a problem where jobs that are completely successful (no text in their stderr file and the proper exit code) are being hidden after the job completes. Any job that fails or has some text in the stderr file is not hidden (note: hidden not deleted; they can be viewed by selecting 'Unhide Hidden Datasets'). Our drmaa.py is at changeset 10961:432999eabbaa Our drmaa egg is at drmaa = 0.6 And our SLURM version is 2.3.5 And we are currently passing no parameters for default_cluster_job_runner = drmaa:/// We have the same code base on both clusters but only observe this behavior when using SLURM. Any pointers or advice would be greatly appreciated. Thanks, Andrew

2 2

DRMAA/SGE job handling regression?
by Peter Cock 11 Nov '13

11 Nov '13

Hello all, On our main Galaxy tracking galaxy-dist using DRMAA/SGE, jobs submitted to the cluster and queued and waiting (qw) are correctly shown in Galaxy as "grey" pending entries in the history. With my test instance tracking galaxy-central (along with a new visual look and new icons), such jobs are wrongly shown as "yellow" (running). Is this a general regression affecting other people? There also seem to be issues where killing a job in Galaxy just hides it but it remains running (yellow once you tick show deleted datasets, and running on SGE too). This was working properly on galaxy-dist (the job was killed on the cluster, and shown as red if you ticked show deleted datasets). Thanks, Peter

1 0

Using input dataset names in output dataset names
by Peter Cock 11 Nov '13

11 Nov '13

Hi all, I'd like to change the output dataset labelling in Galaxy file format conversion tools. e.g. If the input is history entry 1 (e.g "My Genes") then the output from tabular_to_fasta.xml is currently named "FASTA-to-Tabular on data 1". I would prefer this was "FASTA-to-Tabular on data My Genes" or better "My Genes (as tabular)". I've just done this for my BLAST XML to tabular tool, using the .display_name trick: https://github.com/peterjc/galaxy_blast/commit/31e31c4b5deadd60828ce6e6a381… Would a pull request doing this to the built-in conversion tools be favourably received? Alternatively, would it be preferable to simply reused the input dataset's name unchanged for simple format conversion tools (without text about the conversion)? Related to this, would people prefer if the $on_string in the case of a single input file was the input file's name (e.g. "My Genes") rather than "data 1"? (When there are multiple input files, $on_string needs to be kept short). Regards, Peter

4 8

set_environment_for_install problem, seeking for ideas
by Bjoern Gruening 09 Nov '13

09 Nov '13

Hi Greg, I'm right now in implementing a setup_perl_environment and stumbled about a tricky problem (that is not only related to perl but also for ruby, python and R). The Problem: Lets assume a perl package (A) requires a xml parser written in C/C++ (Z). (Z) is a dependency that I can import but as far as I can see there is no way to call set_environment_for_install before setup_perl_environment, because setup_perl_environment defines an installation type. I would like to discuss that issue to get a few ideas. I can think about these solutions: - hackish solution: I can call install_environment.add_env_shell_file_paths( action_dict[ 'env_shell_file_paths' ] ) inside of the setup_*_environment path and remove it from action type afterwards - import all env.sh variables from every (package) definition. Regardless if set_environment_for_install is set or not. I must admit, I do not understand why set_environment_for_install is actually needed. I think we can assume that if I specify a <package name="R_3_0_1" version="3.0.1"> <repository name="package_r_3_0_1" owner="iuc" prior_installation_required="True" /> </package> I want the ENV vars sourced. Furthermore, that can solve an other issue: Namely, the need of ENV vars from a package definition in the same file. Lets imagine package P has dependency D and you want to download compile both in one tool_dependencies.xml file. You can either do it in one <package> definition or you need to split them up in 2 tool_dependencies.xml files, rigth? Maybe we can just assume a strict order in a tool_dependencies.xml file, where every ENV vars are sourced for the following one? Does that make sense? Any other ideas? Thanks, Bjoern

5 13

Job execution order mixed-up
by Jean-Francois Payotte 09 Nov '13

09 Nov '13

Dear Galaxy mailing-list, Once again I come seeking for your help. I hope someone already had this issue or will have an idea on where to look to solve it. :) One of our users reported having workflows failing because some steps were executed before all their inputs where ready. You can find a screenshot attached, where we can see that step (42) "Sort on data 39" has been executed while step (39) is still waiting to run (gray box). This behaviour has been reproduced with at least two different Galaxy tools (one custom, and the sort tool which comes standard with Galaxy). This behaviour seems to be a little bit random, as running two times a workflow where this issue occurs, only one time did some steps were executed in the wrong order. I could be wrong, but I don't think this issue is grid-related as, from my understanding, Galaxy is not using SGE job dependencies functionality. I believe all jobs stays in some internal queues (within Galaxy) until all input files are ready, and only then the job is submitted to the cluster. Any help or any hint on what to look at to solve this issue would be greatly appreciated. We have updated our Galaxy instance to August 12th distribution on October 1st, and I believe we never experienced this issue before the update. Many thanks for your help, Jean-François

2 1

multiple output tool in workflow
by Jennifer Jackson 09 Nov '13

09 Nov '13

Hi Jun, There is probably a problem with the tool design itself, but that may be what you are asking how to solve. I wouldn't think this is a problem with workflows at first pass. Is this your own tool? Or a tool from the tool shed (the repo developer is usually the one to make changes, unless you want to try)? This is the primary tool development wiki, the " <output> tag set" is where I would double check the tool first. http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax I am moving this over to the galaxy-dev(a)bx.psu.edu mailing list since it is a tool development question. Jen Galaxy team On 11/8/13 10:19 AM, Jun Fan wrote: > > Hi all, > > I am trying to creating a workflow from history. One of the tool > used generates multiple outputs in the format of gff3, fasta and sam. > Gff3 will be visualized in IGV and the fasta file is doing further > BLAST analysis. Now the problem is that the automatically generated > workflow does not connect the having-multiple-output tool and the > BLAST tool. I failed even I tried to connect these two tools in the > workflow by hand. I am guessing this is due to only the main output > type (gff3) is recognized in the workflow. How could I solve this problem? > > Best regards and have a nice weekend! > > Jun > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ -- Jennifer Hillman-Jackson http://galaxyproject.org

3 3