- galaxy-dev - lists.galaxyproject.org

Problem in Json datatype?
by Keith Suderman 04 Jun '15

04 Jun '15

Greetings, I was adding datatype classes to galaxy.datatype.text.py for our custom JSON formats and I noticed what I think is a problem with the Json datatype class, in particular the _looks_like_json method. The basic algorithm is (in pseudo code): if the_file_is_small_enough: try to load the the file as json return True on success, False otherwise else while True find first non-blank line if line starts with ‘{‘ or ‘[‘: return True else return False However, JSON is typically compressed by stripping whitespace (including newlines), particularly when it is fetched from a web service. This means that the first call to readLine is going to load the entire file, which defeats the purpose of checking the file size in the first place. I would submit a pull request with our fix, but since I am not a Python programmer I thought I would simply post the code here for review. class Json(Text): # Unchanged bits of the class have been omitted. # Add a function that reads a single character skipping over whitespace. def read(self, fileHandle): ch = fileHandle.read(1) while ch.isspace(): ch = fileHandle.read(1) return ch # Only the “else-part” has changed def _looks_like_json(self, filename): # Pattern used by SequenceSplitLocations if os.path.getsize(filename) < 50000: # If the file is small enough - don't guess just check. try: json.load(open(filename, "r")) return True except Exception: return False else: with open(filename, "r") as fh: ch = self.read(fh) return ch == '{' or ch == '[' We then use the read() method to sniff out our JSON formats. Once we can sniff a header subclasses only need to specify the header. class Lapps( Json ): header = ‘’’{“discriminator”:”http://vocab.lappsgrid.org''' def sniff(self, filename) with open(filename) as fh: for c in header: if c != self.read(fh) return False return True class Lif ( Lapps ): header = ‘’’{“discriminator”:”http://vocab.lappsgrid.org/ns/media/jsonld#lif”''' class Gate( Lapps ): header = ‘’’{“discriminator”:”http://vocab.lappsgrid.org/ns/media/gate”''' … Regardless of how the JSON is rendered (pretty printed or not) we can match the “magic” strings used to identify our formats. Being new to Python I don’t know how expensive it is to read a file one character at a time, but it has to be cheaper than potentially reading the entire file. Cheers, Keith ------------------------------ Research Associate Department of Computer Science Vassar College Poughkeepsie, NY

3 2

Functional Test Failures with 15.05
by Ryan G 04 Jun '15

04 Jun '15

Hi - Are the functional tests for Galaxy supposed to run out of the box? I'm getting lots a failures with a fresh install in a virtual python environment.

2 1

API access from Tool script
by Raffaele Montella 04 Jun '15

04 Jun '15

Dears, I’m working on a tool script (a Python script) that given a dataset id host and port, for example “http://web.uniparthenope.it:28080/api/datasets/3068f10db18abc2b” get the json dateset description in order to extract the file_name field as "file_name": "/home/0124000066/galaxy-python/galaxyTools/scratch/galaxy/files/001/dataset_1369.dat” The motivation is accessing to the actual filesystem path from the dataset id. There is any clean way to transfer the galaxysession cookie value to the tools script as an environment variable or any form of the authentication of the user that is running the tool? Bests, Raffaele****************************************************************************** IL MERITO DEGLI STUDENTI VIENE RICONOSCIUTO Il 5 per mille all'Universita' degli Studi di Napoli "Parthenope" incrementa le borse di studio agli studenti - codice fiscale 80018240632 http://www.uniparthenope.it/index.php/it/5-per-mille Questa informativa e' inserita in automatico dal sistema al fine esclusivo della realizzazione dei fini istituzionali dell'ente.

1 0

LEfSe on local Galaxy instance
by Romain Maleysson 04 Jun '15

04 Jun '15

Hello, I recently installed Galaxy and among other tools I would like to have LEfSe on my local instance. I get the latest LEfSe's repository from the galaxy main tool shed, and I also install it's requirements packages and R libraries on the server (btw it's Debian based). I have a problem with the first tool, "A) Format Data for LEfSe", as you can see in the picture I attached "galaxy_lefse_error.png". In this step, I'm supposed to indicate which row Galaxy has to use to label the data. My issue is that the box "Select which row to use as class" remains saying "No options available", which blocks me to process the step. After checking the xml file of the tool, I found that a "dynamic option" is involved and normally have to generate the options list. I got helped by George Weingart (thanks to him and his patience) and I can say the following things : - My file is correct : I can run LEfSe on it with the public server Huttenhower-lab. I did the same manipulations with the example file provided by LEfSe's authors (found at this link http://www.huttenhower.org/webfm_send/73) and same result on my own Galaxy instance, while it's working on Huttenhower-lab; - This issue is not related to LEfSe : I installed the program outside of Galaxy (as stand alone) on the server and I get correct results. I tried to uninstall/reinstall LEfSe in Galaxy from the tool shed. I also replaced the LEfSe's scripts in the tool shed folder by the ones of my stand alone LEfSe. I checked the Galaxy log and didn't find any error possibly related to LEfSe. And finally, the other tools of LEfSe in my Galaxy instance are working well and give me correct results, only the "format data" tool has a problem; - With the same file (e.g. the example one), a difference occurs in the data loading between my Galaxy instance and Huttenhower-lab. After uploading, the datatype detected by my Galaxy instance is "axes", whereas it's "tabular" with Huttenhower-lab. Manually forcing the datatype to tabular (on upload or after upload) doesn't solve my issue. Does anyone have a clue to explain this difference in datatype detection? Many thanks in advance, Romain

1 0

Multi-level datatype conversions
by David Kelly 03 Jun '15

03 Jun '15

Hello, Suppose I have three custom datatypes defined: a, b, and c I have a datatype converter that converts from a -> b I have a datatype converter that converts from b -> c Can I then send 'a' files as input into tools that accept type 'c'? This doesn't seem to work in my test case, but I thought it might be worth asking to see if anyone has tried this before. Thanks! Regards, David

1 0

Migrating Galaxy 14.02 to 15.05 : error with tool migration
by Jean-Philippe Villemin 03 Jun '15

03 Jun '15

I'm trying to make an update ... I' have done the following commands hg pull hg update -C latest_15.05 ./run.sh "Exception: Your database has version '118' but this code expects version '129'. Please backup your database and then migrate the schema by running 'sh manage_db.sh -c ./universe_wsgi.ini upgrade'." sh manage_db.sh -c ./universe_wsgi.ini upgrade sh ./scripts/migrate_tools/0012_tools.sh -c universe_wsgi.ini install_dependencies I have the following error and i don't know what it means : No handlers could be found for logger "galaxy.tools.data" Traceback (most recent call last): File "./scripts/migrate_tools/migrate_tools.py", line 21, in <module> app = MigrateToolsApplication( sys.argv[ 1 ] ) File "/Users/galaxy_dev_user/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py", line 41, in __init__ self._configure_toolbox() File "/Users/galaxy_dev_user/galaxy-dist/lib/galaxy/config.py", line 748, in _configure_toolbox self.toolbox = tools.ToolBox( tool_configs, self.config.tool_path, self ) File "/Users/galaxy_dev_user/galaxy-dist/lib/galaxy/tools/__init__.py", line 106, in __init__ app=app, File "/Users/galaxy_dev_user/galaxy-dist/lib/galaxy/tools/toolbox/base.py", line 78, in __init__ self._load_tool_panel() File "/Users/galaxy_dev_user/galaxy-dist/lib/galaxy/tools/toolbox/base.py", line 312, in _load_tool_panel self.__add_tool_to_tool_panel( val, self._tool_panel, section=False ) File "/Users/galaxy_dev_user/galaxy-dist/lib/galaxy/tools/toolbox/base.py", line 251, in __add_tool_to_tool_panel tool = self._tools_by_id[ tool_id ] KeyError: 'tool_test' -- *Jean-Philippe Villemin *- Bioinformatics, Software Engineer - IURC (Institut Universitaire de Recherche Clinique) 641 avenue du Doyen Gaston Giraud 34093 Montpellier Cedex 5, France *Jean-Philippe.VILLEMIN(a)inserm.fr <Jean-Philippe.VILLEMIN(a)inserm.fr>*

1 0

Problem sharing histories
by Poole, Richard 02 Jun '15

02 Jun '15

Hi Nate, Although it was working fine in the past (and older shared histories are still shared) we are currently experiencing an issue sharing histories on our local Galaxy server. I really don’t understand the error messages at all, other than it looking like its possible a proxy/webserver issue. After entering an email (which is a list for admin users but blank for non-admin users) I get the Internal Server Error page. Here is the log: 127.0.0.1 - - [02/Jun/2015:10:41:35 +0100] "GET /api/users?f_email=&_=1433238093664 HTTP/1.1" 200 - "http://****.****.ucl.ac.uk:8080/history/share?use_panels=False&id=93d12697461a7974" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.6.3 (KHTML, like Gecko) Version/8.0.6 Safari/600.6.3" 127.0.0.1 - - [02/Jun/2015:10:41:46 +0100] "GET /api/users?f_email=&_=1433238093665 HTTP/1.1" 200 - "http://****.****.ucl.ac.uk:8080/history/share?use_panels=False&id=93d12697461a7974" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.6.3 (KHTML, like Gecko) Version/8.0.6 Safari/600.6.3" 127.0.0.1 - - [02/Jun/2015:10:41:48 +0100] "POST /history/share HTTP/1.1" 500 - "http://****.****.ucl.ac.uk:8080/history/share?use_panels=False&id=93d12697461a7974" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.6.3 (KHTML, like Gecko) Version/8.0.6 Safari/600.6.3" Error - <type 'exceptions.TypeError'>: append() takes exactly one argument (0 given) URL: http://****.****.ucl.ac.uk:8080/history/share File '/Users/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/Library/Python/2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/Library/Python/2.7/site-packages/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/Users/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 133 in __call__ return self.handle_request( environ, start_response ) File '/Users/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 191 in handle_request body = method( trans, **kwargs ) File '/Users/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 64 in decorator return func( self, trans, *args, **kwargs ) File '/Users/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/history.py', line 783 in share histories = self._get_histories( trans, id ) File '/Users/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/history.py', line 902 in _get_histories histories.append( ) TypeError: append() takes exactly one argument (0 given) CGI Variables ------------- CONTENT_LENGTH: '-1' CONTENT_TYPE: 'application/x-www-form-urlencoded' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-us' HTTP_CONNECTION: 'close' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603affb9e7f3db8ec24e38c21f8e92c63c0423b0bca935c4829106a35253947071a3; __utma=156947391.1608730577.1432903437.1432903437.1432903437.1; __utmz=156947391.1432903437.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' HTTP_DNT: '1' HTTP_HOST: '****.****.ucl.ac.uk<http://ucl.ac.uk>:8080' HTTP_ORIGIN: 'http://****.****.ucl.ac.uk:8080' HTTP_REFERER: 'http://****.****.ucl.ac.uk:8080/history/share?use_panels=False&id=93d12697461a7974' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.6.3 (KHTML, like Gecko) Version/8.0.6 Safari/600.6.3' HTTP_X_FORWARDED_FOR: '127.0.0.1' HTTP_X_FORWARDED_HOST: '****.****.ucl.ac.uk<http://ucl.ac.uk>:8080' HTTP_X_FORWARDED_SERVER: '****.****.ucl.ac.uk<http://ucl.ac.uk>' ORGINAL_HTTP_HOST: 'localhost:8081' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/history/share' REMOTE_ADDR: '127.0.0.1' REQUEST_METHOD: 'POST' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8081' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x10cc3e290> is_api_request: False paste.cookies: (<SimpleCookie: __utma='156947391.1608730577.1432903437.1432903437.1432903437.1' __utmz='156947391.1432903437.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' galaxysession='c6ca0ddb55be603affb9e7f3db8ec24e38c21f8e92c63c0423b0bca935c4829106a35253947071a3'>, 'galaxysession=c6ca0ddb55be603affb9e7f3db8ec24e38c21f8e92c63c0423b0bca935c4829106a35253947071a3; __utma=156947391.1608730577.1432903437.1432903437.1432903437.1; __utmz=156947391.1432903437.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x10cc3e1d0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x10d21f690> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: '96fdc0dc090b11e5aa28685b359a2712' webob._parsed_post_vars: (MultiDict([('id', '93d12697461a7974'), ('email', '33b43b4e7093c91f'), ('share_button', 'Submit')]), <FakeCGIBody at 10d4be2d0 viewing MultiDict([('id...t')])>) webob._parsed_query_vars: (MultiDict([]), '') wsgi process: 'Multithreaded' ------------------------------------------------------------ Any help with this greatly appreciated :) Richard Richard J Poole PhD Wellcome Trust Fellow Department of Cell and Developmental Biology University College London 21 University Street, London WC1E 6DE Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133)

3 2

unable to install from toolshed. bounced e-mail
by Leontovich, Alexey A., Ph.D. 01 Jun '15

01 Jun '15

Hello galaxy team, This e-mail was bounced, so I am sending it again. When I try to install tools (bismark, bisulfighter, bscall) to my local instance of Galaxy, I get an error message: Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. The first time I tried to install was Friday, May 31, 2015. I have installed many tools before, about 6 months ago and it worked just fine. Could you please help with current installation problems? Thank you. Alexey Leontovich Alexey Leontovich, Ph.D. | Assistant Professor of Medical Informatics | Biomedical Statistics and Informatics |Phone: 507-284-4850 | Fax: 507-538-0850 | leontovich.alexey(a)mayo.edu Mayo Clinic | 200 First Street SW | Rochester, MN 55905 | mayoclinic.org<http://www.mayoclinic.org>

2 1

Error bootstrapping a toolshed
by Marius van den Beek 01 Jun '15

01 Jun '15

Hello everyone, I had some trouble bootstrapping a new toolshed with the 15_05 release (worked fine in the past), and I thought the solution might be interesting. ~/toolshed2/galaxy$ sh run_tool_shed.sh -bootstrap_from_tool_shed http://toolshed.g2.bx.psu.edu Bootstrapping from tool shed at http://toolshed.g2.bx.psu.edu. Creating database... done. Starting tool shed in order to populate users and categories... done. Retrieving admin user's API key from http://127.0.0.1:9009... done. Creating users... usage: create_users.py [-h] -a API -f FROM_TOOL_SHED -t TO_TOOL_SHED create_users.py: error: unrecognized arguments: 8f754ebd516ff576c3b30b6a3402a518 } done. Creating categories... usage: create_categories.py [-h] -a API -f FROM_TOOL_SHED -t TO_TOOL_SHED create_categories.py: error: unrecognized arguments: 8f754ebd516ff576c3b30b6a3402a518 } done. Bootstrap complete, shutting down temporary tool shed process. A log has been saved to tool_shed_bootstrap.log The solution is to exclude the curl braces coming from the API response. A quick fix is to change the line api_key=`curl -s --user $admin_user_email:$admin_user_password $local_shed_url/api/authenticate/baseauth/ | sed 's/..*api_key[^0-9a-f][^0-9a-f]*$[0-9a-f]*$..*/\1/'` to api_key=`curl -s --user $admin_user_email:$admin_user_password $local_shed_url/api/authenticate/baseauth/ | sed 's/..*api_key[^0-9a-f][^0-9a-f]*$[0-9a-f]*$..*/\1/'| sed 's/[{}]//g'` in lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh . Hope it’s useful, Marius

2 2

unable to install tools from the toolshed
by Leontovich, Alexey A., Ph.D. 01 Jun '15

01 Jun '15

Hello galaxy team, When I try to install tools (bismark, bisulfighter, bscall) to my local instance of Galaxy, I get an error message: Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. The first time I tried to install was Friday, May 31, 2015. I have installed many tools before, about 6 months ago and it worked just fine. Could you please help with current installation problems? Thank you. Alexey Leontovich Alexey Leontovich, Ph.D. | Assistant Professor of Medical Informatics | Biomedical Statistics and Informatics |Phone: 507-284-4850 | Fax: 507-538-0850 | leontovich.alexey(a)mayo.edu Mayo Clinic | 200 First Street SW | Rochester, MN 55905 | mayoclinic.org<http://www.mayoclinic.org>

1 0