October 2012 - galaxy-dev - lists.galaxyproject.org

Column alignment in tabular display
by Peter Cock 30 Oct '12

30 Oct '12

Hello all, I'm curious about the logic used for column alignment when displaying tabular files (which for me was the headline feature of the last Galaxy Release - I'm very pleased to have this added at last). https://bitbucket.org/galaxy/galaxy-central/issue/553 Quoting from templates/dataset/tabular_chunked.mako function renderCell(cell_contents, index, colspan){ if (colspan !== undefined){ return $('<td>').attr('colspan', colspan).addClass('stringalign').text(cell_contents); } else if (DATASET_TYPES[index] == 'str'){ /* Left align all str columns, right align the rest */ return $('<td>').addClass('stringalign').text(cell_contents);; } else{ return $('<td>').text(cell_contents); } } It seems to me that the logic should be right align all numeric columns, left align the rest. At very least, columns of type 'list' should also be left aligned. Otherwise you can easily have a situation where on clicking the "eye" you get an apparently empty dataset (unless you scroll right or down). This can easily happen with FASTA to tabular (default settings), where the descriptions have commas in them (common with annotation) thus the first column is considered to be a list (and the second column type str for the sequence). If some of the later sequences in the file have very long descriptions, right aligning the description column makes the dataset appear empty. I can provide a screen shot if it would help to explain, but here is my proposed patch. Regards, Peter -- $ hg diff templates/dataset/tabular_chunked.mako diff -r 8657ada96fca templates/dataset/tabular_chunked.mako --- a/templates/dataset/tabular_chunked.mako Fri Jun 15 16:05:48 2012 +0100 +++ b/templates/dataset/tabular_chunked.mako Mon Jun 18 16:23:44 2012 +0100 @@ -17,8 +17,8 @@ if (colspan !== undefined){ return $('<td>').attr('colspan', colspan).addClass('stringalign').text(cell_contents); } - else if (DATASET_TYPES[index] == 'str'){ - /* Left align all str columns, right align the rest */ + else if (DATASET_TYPES[index] == 'str' || DATASET_TYPES[index] == 'list'){ + /* Left align all str and list columns, right align the rest */ return $('<td>').addClass('stringalign').text(cell_contents);; } else{

2 3

Need help on encoding error with the database.
by Jian Ma 30 Oct '12

30 Oct '12

Hi Galaxy-team, I'm running Galaxy on my local system, but recently get an coding error. The database I'm using is postgresql, I also modified part of the code, hope that will not make things complex. Looks like the tool generate a pdf file, and a geek of the file contains some binary code which is failed to store into the database. Below is the error message. Appreciate for suggestions. galaxy.jobs.runners.local ERROR 2012-10-30 10:47:28,855 Job wrapper finish method failed Traceback (most recent call last): File "/data/CistromeAP/lib/galaxy/jobs/runners/local.py", line 133, in run_job job_wrapper.finish( stdout, stderr ) File "/data/CistromeAP/lib/galaxy/jobs/__init__.py", line 454, in finish tool=self.tool, stdout=stdout, stderr=stderr ) File "/data/CistromeAP/lib/galaxy/tools/__init__.py", line 2207, in call_hook return code( *args, **kwargs ) File "./tools/expression/norm_code.py", line 103, in exec_after_process app.model.context.flush() File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 773, in execute_save_steps self.execute_cyclical_dependencies(trans, task, False) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 789, in execute_cyclical_dependencies self.execute(trans, [t], isdelete) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1424, in _save_obj c = connection.execute(statement.values(value_params), params) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 952, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/data/CistromeAP/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 933, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) DataError: ('E', 'r', 'r', 'o', 'r', ' ', 'i', 'n', ' ', "'", 'G', 'e', 'n', 'e', ' ', 'e', 'x', 'p', 'r', 'e', 's', 's', 'i', 'o', 'n', ' ', 'i', 'n', 'd', 'e', 'x', "'", ' ', 'h', 'o', 'o', 'k', ' ', "'", 'e', 'x', 'e', 'c', '_', 'a', 'f', 't', 'e', 'r', '_', 'p', 'r', 'o', 'c', 'e', 's', 's', "'", ',', ' ', 'o', 'r', 'i', 'g', 'i', 'n', 'a', 'l', ' ', 'm', 'e', 's', 's', 'a', 'g', 'e', ':', ' ', '(', 'D', 'a', 't', 'a', 'E', 'r', 'r', 'o', 'r', ')', ' ', 'i', 'n', 'v', 'a', 'l', 'i', 'd', ' ', 'b', 'y', 't', 'e', ' ', 's', 'e', 'q', 'u', 'e', 'n', 'c', 'e', ' ', 'f', 'o', 'r', ' ', 'e', 'n', 'c', 'o', 'd', 'i', 'n', 'g', ' ', '"', 'U', 'T', 'F', '8', '"', ':', ' ', '0', 'x', '8', '1', '\n', 'H', 'I', 'N', 'T', ':', ' ', ' ', 'T', 'h', 'i', 's', ' ', 'e', 'r', 'r', 'o', 'r', ' ', 'c', 'a', 'n', ' ', 'a', 'l', 's', 'o', ' ', 'h', 'a', 'p', 'p', 'e', 'n', ' ', 'i', 'f', ' ', 't', 'h', 'e', ' ', 'b', 'y', 't', 'e', ' ', 's', 'e', 'q', 'u', 'e', 'n', 'c', 'e', ' ', 'd', 'o', 'e', 's', ' ', 'n', 'o', 't', ' ', 'm', 'a', 't', 'c', 'h', ' ', 't', 'h', 'e', ' ', 'e', 'n', 'c', 'o', 'd', 'i', 'n', 'g', ' ', 'e', 'x', 'p', 'e', 'c', 't', 'e', 'd', ' ', 'b', 'y', ' ', 't', 'h', 'e', ' ', 's', 'e', 'r', 'v', 'e', 'r', ',', ' ', 'w', 'h', 'i', 'c', 'h', ' ', 'i', 's', ' ', 'c', 'o', 'n', 't', 'r', 'o', 'l', 'l', 'e', 'd', ' ', 'b', 'y', ' ', '"', 'c', 'l', 'i', 'e', 'n', 't', '_', 'e', 'n', 'c', 'o', 'd', 'i', 'n', 'g', '"', '.', '\n') 'INSERT INTO history_dataset_association (id, history_id, dataset_id, create_time, update_time, state, copied_from_history_dataset_association_id, copied_from_library_dataset_dataset_association_id, hid, name, info, blurb, peek, tool_version, extension, metadata, parent_id, designation, deleted, purged, visible) VALUES (%(id)s, %(history_id)s, %(dataset_id)s, %(create_time)s, %(update_time)s, %(_state)s, %(copied_from_history_dataset_association_id)s, %(copied_from_library_dataset_dataset_association_id)s, %(hid)s, %(name)s, %(info)s, %(blurb)s, %(peek)s, %(tool_version)s, %(extension)s, %(_metadata)s, %(parent_id)s, %(designation)s, %(deleted)s, %(purged)s, %(visible)s)' {'info': 'makeAffyBatch, Normalized_rma_Raw_Plots.pdf', 'peek': "%PDF-1.4\n%\x81\xe2\x81\xe3\x81\xcf\x81\xd3\\r\n1 0 obj\n<<\n/CreationDate (D:20121030103557)\n/ModDate (D:20121030103557)\n/Title (R Graphics Output)\n/Producer (R 2.15.1)\n/Creator (R)\n>>\nendobj\n2 0 obj\n<< /Type /Catalog /Pages 3 0 R >>\nendobj\n7 0 obj\n<< /Type /Page /Parent 3 0 R /Contents 8 0 R /Resources 4 0 R >>\nendobj\n8 0 obj\n<<\n/Length 990 /Filter /FlateDecode\n>>\nstream\nx\x9c\xa5W\xcb\x8eT9\x0c\xdd\xe7+\xbc\xec^\x10\x928\xcf--\x06\xa9\x05HP%\xb1@,\x80\x194B]\xbc\x1a\t>\x1f'vr\x1f}\x9b\xaaRon\xe9D\xc7'\xceI\xe2\xb8,\\\x83\x85\xcf\xf0]\xbd\x82\xef`\x8d\xd71\x83\xf3Nc\x06o\xbdN\x0e0Y\x1d\x1c\xfc\xf8\x0f\xde\xc0\x17\xf5\xf8\xf6\xf5\xb3'p\xb5SF\x1bc`\xfe\xdd]\xbdTN\xbb\x00\xbf\xd4\xdbw`\xe0_e\xe0Z\x91\xba\xb2\x95\x00/\x94uQ{\x0f\xe8H\x18\xe1\x006%\x1d\x07\xbe\x81\xaa\x9aZ<4!$\xc4J\xb6*\x05\xa7\x8d\x07\x97\x8a\xb6\xb6F3", 'update_time': datetime.datetime(2012, 10, 30, 14, 36, 51, 684), 'name': 'Normalized_rma_Raw_Plots.pdf', 'extension': 'pdf', 'tool_version': None, 'deleted': False, 'history_id': None, 'copied_from_library_dataset_dataset_association_id': None, '_state': None, 'designation': None, 'purged': False, 'parent_id': None, 'create_time': datetime.datetime(2012, 10, 30, 14, 36, 51, 587), 'hid': None, '_metadata': <psycopg2._psycopg.Binary object at 0x49231b8>, 'visible': True, 'dataset_id': 727L, 'id': 739L, 'blurb': None, 'copied_from_history_dataset_association_id': None} -- best， Jian Ma,

1 0

Best Practice - tool development
by Björn Grüning 30 Oct '12

30 Oct '12

Hi all, I have a few general questions about best practices in tool development and would like to collect a few opinions. Maybe the Intergalactic Utilities Commission can discuss some points at their first meeting. 1. How to handle dependencies that are not guaranteed to exist for a large time. Sometimes I'm very sceptical about whether upstream will keep every software version for the next X years, which I will define in my tools, e.g. some ftp server for binaries or some beta-versions of actually used software (bowtie2?). Is it possible/wished to create a repository for such software? 2. How to handle different compiler versions or other system level changes? For example with the new gcc-4.7 some 'include'-definitions changed and because of that bowtie2 is not compiling any more. Should we add a feature to ship patches in our tools for such simple workarounds? Or should we rather wait for a new upstream version and not support our tool on such systems? 3. Should we standardize the version numbering of tools to reflect if a update is important or not? For example something like foo.bar.baz. With: foo = major tool-dependency changing: expected result changes bar = minor tool-dependency changing due to bug fixing: results can change but it is not expected baz = no expected changes in the result due to wrapper changes (help-text, adding of new parameters, etc.). I think in the end we need to decide how much effort do we put into reproducibility over the next X years and how do we guarantee that, either as Community or as Intergalactic Utilities Commission. What did you think? Björn [1] http://gcc.gnu.org/gcc-4.7/porting_to.html -- Björn Grüning Albert-Ludwigs-Universität Freiburg Institute of Pharmaceutical Sciences Pharmaceutical Bioinformatics Hermann-Herder-Strasse 9 D-79104 Freiburg i. Br. Tel.: +49 761 203-4872 Fax.: +49 761 203-97769 E-Mail: bjoern.gruening(a)pharmazie.uni-freiburg.de Web: http://www.pharmaceutical-bioinformatics.org/

1 0

Hiding a tool via tool_conf.xml
by Peter Cock 30 Oct '12

30 Oct '12

Hi all, I'm aware that in an individual tool's XML wrapper you can add hidden="true" to the <tool> element to hide the tool from the listing shown in Galaxy's left hand panel (but the tool is still loaded and can be called by old workflows etc). Can this be done via the tool_conf.xml file as well? This would seem useful from a system configuration point of view where it seems wrong to edit the tool wrappers themselves. In fact, it might also be nice to add hidden="true" support to <section> as well as <tool> in tool_conf.xml - although I'm not so sure how useful that would be. Peter

1 2

Re: [galaxy-dev] download dataset from the proxy-ed Galaxy failed with broken link
by Derrick Lin 29 Oct '12

29 Oct '12

On Tue, Oct 30, 2012 at 4:03 PM, Ross <ross.lazarus(a)gmail.com> wrote: > Doesn't make sense that you can wget directly from the paste process > if you have security properly configured! > That's what you can do with the main Galaxy site. Not sure if that's intentional though. You can try wget --no-check-certificate https://main.g2.bx.psu.edu/datasets/851fbe5b7ff576a6/display?to_ext=tabular See if you can download my own dataset without providing any credential. > Do you have Apache authenticating and passing headers through to the > paste process - this section in universe_wsgi.ini > > # User authentication can be delegated to an upstream proxy server (usually > # Apache). The upstream proxy should set a REMOTE_USER header in the > request. > # Enabling remote user disables regular logins. For more information, see: > # http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy > use_remote_user = True > > If so, you should not be able to access anything via the paste process > directly without adding authentication headers. Once that's fixed (you > do NOT want anyone to be able to do what you can do - it bypasses all > security!) the apache configuration will probably need tweaking. It's > hard to advise - it's mostly voodoo IMHO - do you have an apache > fluent sysadmin? We don't use Apache for authentication. Thus use_remote_user = False. Will read more about this remote user thing. Cheers Derrick

1 0

Re: [galaxy-dev] download dataset from the proxy-ed Galaxy failed with broken link
by Derrick Lin 29 Oct '12

29 Oct '12

Hi Ross, Good point and it makes sense in term of security. But I just tried it, and it didn't help. I captured these two from the Paste log (not apache log): xx.xx.1xx.xx - - [30/Oct/2012:15:42:17 +1100] "GET /galaxy_dev/datasets/10b866a1b258d1f1/display?to_ext=fastqsanger HTTP/1.1" 302 - "-" "Wget/1.12 (linux-gnu)" galaxy.web.framework DEBUG 2012-10-30 15:42:17,676 Error: this request returned None from get_history(): http://pwbc.garvan.unsw.edu.au/galaxy_dev/root xx.xx.1xx.xx - - [30/Oct/2012:15:42:17 +1100] "GET /galaxy_dev/root?dataset_id=10b866a1b258d1f1&filename=None HTTP/1.1" 200 - "-" "Wget/1.12 (linux-gnu)" Another observation is, I can wget my own dataset from the main Galaxy straight away without providing any credential. Maybe something wrong with my universe ini file. Derrick On Tue, Oct 30, 2012 at 3:32 PM, Ross <ross.lazarus(a)gmail.com> wrote: > That url requires a login from here - have you tried supplying it - eg > curl -i -u <user:password> <target> > > On Tue, Oct 30, 2012 at 3:27 PM, Derrick Lin <klin938(a)gmail.com> wrote: > > If I downloaded with curl -i, the page it returned was: > > > > pwbcad@gamma01:tmp$ curl -i > > > http://galaxy-qa.garvan.unsw.edu.au/datasets/dc097fa4e7ad53f6/display?to_ex… > > HTTP/1.1 302 Found > > Date: Tue, 30 Oct 2012 04:25:21 GMT > > Server: PasteWSGIServer/0.5 Python/2.6.6 > > content-type: text/html > > location: /root?dataset_id=dc097fa4e7ad53f6&filename=None > > Set-Cookie: > > > galaxysession=c6ca0ddb55be603a4107d5e4793b9ce28e93ce1e646977c209d1af46a242147b57953de37776fcc2; > > expires=Mon, 28-Jan-2013 15:25:21 GMT; httponly; Max-Age=7776000; Path=/; > > Version=1 > > Vary: Accept-Encoding > > Transfer-Encoding: chunked > > > > <html> > > <head><title>Found</title></head> > > <body> > > <h1>Found</h1> > > <p>The resource was found at <a > > > href="/root?dataset_id=dc097fa4e7ad53f6&filename=None">/root?dataset_id=dc097fa4e7ad53f6&filename=None</a>; > > you should be redirected automatically. > > > > </p> > > <hr noshade> > > <div align="right">WSGI Server</div> > > </body> > > </html> > > > > Don't know why the apache returned 302 only when access via wget or curl. > > > > Derrick > > > > On Sat, Oct 27, 2012 at 7:44 AM, Derrick Lin <klin938(a)gmail.com> wrote: > >> > >> Hi Nate, > >> > >> I checked and what downloaded is a webpage, I opened it on browser, and > >> screenshot attached. > >> > >> Basically it displays error saying File not found. But dataset download > >> works fine via browser. > >> > >> Cheers, > >> Derrick > >> > >> On Fri, Oct 26, 2012 at 11:44 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: > >>> > >>> On Oct 24, 2012, at 11:15 PM, Derrick Lin wrote: > >>> > >>> > Hi guys, > >>> > > >>> > On our Galaxy is proxy-ed via Apache according to the wiki guide, the > >>> > normal download dataset via browser is working fine. > >>> > > >>> > However, when I tried download from command line via wget or curl -O > (I > >>> > copied the dataset URL by right clicking the floppy disk icon). > >>> > > >>> > And here is the wget output: > >>> > > >>> > pwbcad@gamma01:tmp$ wget > >>> > > http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=f… > >>> > --2012-10-25 10:57:42-- > >>> > > http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=f… > >>> > Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 > >>> > Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. > >>> > HTTP request sent, awaiting response... 302 Found > >>> > Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None > >>> > [following] > >>> > --2012-10-25 10:57:42-- > >>> > > http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=… > >>> > Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. > >>> > HTTP request sent, awaiting response... 200 OK > >>> > Length: unspecified [text/html] > >>> > Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1" > >>> > > >>> > [ <=> > >>> > ] 28,476 --.-K/s in 0.001s > >>> > > >>> > 2012-10-25 10:57:42 (31.4 MB/s) - > >>> > "root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476] > >>> > > >>> > It seems like the Apache returned another URL (returned 302, probably > >>> > due to proxy) and the new URL is broken. > >>> > >>> Hi Derrick, > >>> > >>> What are the contents of the file that wget saved? This should give > you > >>> a clue as to what's going on. > >>> > >>> --nate > >>> > >>> > > >>> > Thanks in advance. > >>> > > >>> > Cheers, > >>> > Derrick > >>> > ___________________________________________________________ > >>> > Please keep all replies on the list by using "reply all" > >>> > in your mail client. To manage your subscriptions to this > >>> > and other Galaxy lists, please use the interface at: > >>> > > >>> > http://lists.bx.psu.edu/ > >>> > >> > > > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > > > http://lists.bx.psu.edu/ > > > > -- > Ross Lazarus MBBS MPH; > Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 > http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ >

1 0

download dataset from the proxy-ed Galaxy failed with broken link
by Derrick Lin 29 Oct '12

29 Oct '12

Hi guys, On our Galaxy is proxy-ed via Apache according to the wiki guide, the normal download dataset via browser is working fine. However, when I tried download from command line via wget or curl -O (I copied the dataset URL by right clicking the floppy disk icon). And here is the wget output: pwbcad@gamma01:tmp$ wget http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=f… --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/datasets/ccc5aff28d2c8616/display?to_ext=f… Resolving galaxy.demo.edu.au... 1xx.x4.1xx.xx5 Connecting to galaxy.demo.edu.au|1xx.x4.1xx.xx5|:80... connected. HTTP request sent, awaiting response... 302 Found Location: /galaxy/root?dataset_id=ccc5aff28d2c8616&filename=None [following] --2012-10-25 10:57:42-- http://galaxy.demo.edu.au/galaxy/root?dataset_id=ccc5aff28d2c8616&filename=… Connecting to galaxy.demo.edu.au|129.94.136.205|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: "root?dataset_id=ccc5aff28d2c8616&filename=None.1" [ <=> ] 28,476 --.-K/s in 0.001s 2012-10-25 10:57:42 (31.4 MB/s) - "root?dataset_id=ccc5aff28d2c8616&filename=None.1" saved [28476] It seems like the Apache returned another URL (returned 302, probably due to proxy) and the new URL is broken. Thanks in advance. Cheers, Derrick

2 3

Fastq splitter
by Scooter Willis 29 Oct '12

29 Oct '12

Working with some fastq files that were exported from CLC Genomics Workbench and trying to split them. After running groomer on the files to convert to sanger running splitter and results in 0 split. I have included the first X sequences from the groomer file. Any thoughts on why it isn't splitting? Is it expecting a particular data pattern for the sequence id? The files originated as qseq on casava 1.7 and were converted to fastq using CLC and the reason the sequence identifier is a little different. [cid:F65701C9-A06D-4F15-A281-9DFCC74B8CF2] @HWUSI-EAS1775_1_1_6763_945 NGGGCCCCTCCTGAGCCCCAGCCCAGCCCCTCAGGGTGCTTANATCAAGGAGCCTGCCTGTCTGCATGGACCACACGAAAATCTCAAAGGAGTCCTGCGCC + #33239988222B@BBBBBBBBB@@@@BBB@22@@@@B@@66#3566666@BBBBB@BB@@@@@B@@B@@B@BB@@@B@@@@@;@@@@@;<<:<<@@@@@< @HWUSI-EAS1775_1_1_6763_945 CACCTCTCCACGCCCCACCCCGTTCTCCTGGGTGGCGGGGGATGCGAGCGGTGGGATCGTCTCGGCAGGGGGGCAGGACTCCTTTGGGATTTTCGTGTGGT + B@B0=?:=0?BB######################################################################################### @HWUSI-EAS1775_1_1_10085_933 NACCATGTTAGGCTGGTCTCTTAACTCCTGACCTCAGGTGATNTGCTTGCCTCGGCCTCCCAAAGTGCTGGGATTACAGGCGTGAGCCACCGTGCCCAGCC + #333366666@BBB@BBBBBB@BBB@BB@BBBB@B@B@@@24#4466666BBB@@BBBBB22B@@@@BB@@@@@@@@@B@@BB@@@@@@@<<<<<@@@@@5 @HWUSI-EAS1775_1_1_10085_933 CCTAATTCATCCACAAGCACCAGAGAATGGGCTGGTGCATGCATGAGTATGCTGGCAGTTTCACTTAATTCAACAAAAAATGGACTTTCACCTTAAAAAAA + B::=<9B@?-=7=8+@8@2?91+168@::+5=911B2B:-=:A9:422-,8=42@AB-3BBB3BB=88@@BBBBB?=?==4,+;>BBBB=BB,B3BBBBA# @HWUSI-EAS1775_1_1_12647_946 NCAGGCAGTGCCCCATAATCCCCTGGGTTTTCTGCATCCCCTNAGTGTCGCCACCGGGTCGCACCTGGCAGGTGCCTTGGCTGCAGCCGCGCCCCACTCCC + #211178877@BBBB@BB@B@BBB@B@@BBB@22B@B@B@87#7763666BBBB@@BBBB@BB@B@@@@@:::::82858<<::<@@@@@########### @HWUSI-EAS1775_1_1_12647_946 CACCCAGCAGGCGCTGCAGCGGGACCTGAAGACCCGGGAGAAGGGGGGGGGGGCGGGGGGGGGGGGGGCCGGGCTGCGGCCCAGGCACCCGCCAGGCGAGA + >87-*1:91+=;28+=16:=:5(7:?-B8?:@;=BBBB336=3?######################################################### @HWUSI-EAS1775_1_1_14406_953 NCTCATATCTGCAAAGAACAGACCGTTCAGTATATACTAACTATGGTGGATGATATGCTGCAGGTAGGTATATTCTTTACTTAACACATTTGGTGACTTCA + #2.,.77777B@BBB@@22@@@B@BBBB@B@@@@B@@@@@@B@B@@@@@@@@@@@@B@@@BBBBB@@@8@@@@@@@B22@@@@@@BBB@@@@@@@@B@@@@ @HWUSI-EAS1775_1_1_14406_953 TCTCCCCAAAAATGCTGCTACTGGGAGGGATGCAAAAAAATGAAGTCACCAAATGGGTTAAGTAAAGAATATACCTACCTGCAGCATATCATCCACCATAG + A88?1B><?>BB<B=*;6*;=2=+=/:4'18=@@BBBBBB>BBBB6=28@>BB>B57=BBBB<B<B?><;4*1==5>BB>AB3B31+01:B8@B?BBB8@2 @HWUSI-EAS1775_1_1_15514_936 NAAGTACACAGCTACATCCTCAAAGGTTACCAGCTCCTGAAANAACACGTGCTGGCATGCAATTTCAGGGTGAGGAATGTGAATGGCGGCACAAAAAAAAT + #/2//87777BBBB@BBB@@B@B@@B@@@@@@@@B@B@@@54#2-,1313@BB@BB@B@BB@BB@@@@@@<<<<<<:<<<@@@@@@@@@@@@@@@222220 @HWUSI-EAS1775_1_1_15514_936 TCTATCCTGGACTCCTGTCTCCCTTCCTCCACCCTGGCCTCCCTTCCTCCCTTCCCAAATGGGCCACTTTCACCCCATCCTCAGTGATTTTTTTTGTGCCG + ?61-+:*?:;1;1=8);4:-@,@8ABB8BB,?B<2=,;@8BB<;ABBABBB?BBBBBBBA1'-.+%)5;9414:8@93B>+:.73?+?B?<A@@=302BB# @HWUSI-EAS1775_1_1_16607_936 NGTCTCTCTCCTTTCTGAACAGAGATGAGCCCTAAGATACCTNGAAGATTTTGTTTTTAACCTCACATTAAGGAAAGCACACATCACCTTGAAAGGTTGCA + #111/87787@BBB@@@@B@@@@@@?@@@@@@@@@@B@B@66#6655225@@B222222@@B@B@B@@@@@@@B@@@@@@BB@@@B@@@@@B@@@@@@@@: @HWUSI-EAS1775_1_1_16607_936 TCTGTGCAATGGCACCCTTCTAGGAGGAAGTGTGCAACCTTTCAAGGGGATGTGTGCTTTCCTTAATGGGGGGTTAACAAACACATCTTCCAGGGATTTTA + B38?;=:8;5?B?4?3+3++45;>3@1=;>17*1=4BBB?BBB3>BB+3+;@7;1>=ABBBBBB3BB=-31330=BBBB82.B,?BB@=BB?B########

1 1

Illumina adaptor sequences in tools - copyright?
by Clare Sloggett 29 Oct '12

29 Oct '12

Hi all, We are looking at wrapping trimmomatic ( http://www.usadellab.org/cms/index.php?page=trimmomatic ). However to run, it requires the Illumina adaptor sequences, which are copyright. I was wondering if anyone has already dealt with this issue when wrapping other tools and putting them up on a public galaxy instance or in the Toolshed. For instance, I think that FastQC requires the same sequences. I would imagine that Illumina wouldn't want to stop people using these sequences for analysis purposes, but I'm still thinking we might need some sort of permission. Have others dealt with this? Thanks, Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759

1 0

How to remove a broken toolshed install
by Clare Sloggett 29 Oct '12

29 Oct '12

Hi all, I managed to break a toolshed-installed tool by fiddling with the files under shed_tools. This led to a situation in which the Galaxy admin interface claims the tool is still installed, but can't find any files for it. I manually put the repository files where I think they should go, but this didn't fix the situation, so what I really want to do is just get rid of it altogether and reinstall cleanly. I'm not certain that the tool was working properly before I fiddled with it, either. Galaxy won't let me uninstall, deactivate or update it (because it can't find it properly) and it won't let me install it (because it thinks it's installed). It also seems (judging by the last of the errors below) to be unable to find some config information that it expects, but I don't really understand what's going on there. So my question is: given a messy, screwed up install, how can I completely remove it and start from scratch? What are the different components and config files I need to remove it from and are they all manually accessible? Thanks in advance for any help! If it's relevant to my question, here are some of the behaviours I see currently: The tool appears as "Installed" under Admin -> Manage installed tool shed repositories, but doesn't show up in the tools panel. If I try Repository Actions -> Get repository updates , I get the error: The directory containing the installed repository named 'bcftools_view' cannot be found. But if I try Repository Actions -> Reset repository metadata , it apparently works, I get Metadata has been reset on repository bcftools_view. And, if I try to 'Deactivate or uninstall' the apparently-installed repository, I get: URL: http://galaxy-tut.genome.edu.au/admin_toolshed/deactivate_or_uninstall_repo… Module paste.exceptions.errormiddleware:144 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ >> environ, self.app) Module paste.wsgilib:543 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ >> body = method( trans, **kwargs ) Module galaxy.web.framework:205 in decorator >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:452 in deactivate_or_uninstall_repository >> remove_from_tool_panel( trans, tool_shed_repository, shed_tool_conf, uninstall=remove_from_disk_checked ) Module galaxy.util.shed_util:1781 in remove_from_tool_panel >> tool_panel_dict = generate_tool_panel_dict_from_shed_tool_conf_entries( trans, repository ) Module galaxy.util.shed_util:942 in generate_tool_panel_dict_from_shed_tool_conf_entries >> tree = util.parse_xml( shed_tool_conf ) Module galaxy.util:135 in parse_xml >> tree = ElementTree.parse(fname) Module elementtree.ElementTree:859 in parse Module elementtree.ElementTree:576 in parse TypeError: coercing to Unicode: need string or buffer, NoneType found Thanks, Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759

2 4