July 2016 - galaxy-dev - lists.galaxyproject.org

Running jobs as real user and extra_file_path
by Louise-Amélie Schmitt 29 Apr '17

29 Apr '17

Hi everyone, I just wanted to ask how the extra_file_path is handled in case of job running as the real user since the file_path is only writable by the galaxy user. Any clue? Thanks, L-A

3 3

Python script can't can't import BioPython modules in Galaxy 16.01
by Makis Ladoukakis 18 Jan '17

18 Jan '17

Hello everyone, I've recently upgraded to Galaxy 16.01 and i have some issues with a python wrapper from a tool. When I run it via Galaxy I get the following error: Traceback (most recent call last): File "/home/galaxy_user/galaxy/tools/myTools/know_parser/know_parser.py", line 12, in <module> from Bio import SeqIO ImportError: No module named Bio However when I click on the "view details" icon to copy the command from the tool and run it on my command line the tool works perfectly. In addition this issue occurs while I already have installed BioPython. When I type Python and write "from Bio import SeqIO" I get no error. Can this be an issue related to the virtual environment that is initialized each time Galaxy starts? Thank you in advance, Makis Ladoukakis

5 8

Python ValueError: too many values to unpack
by Matthias De Smet 19 Oct '16

19 Oct '16

Hi all, I’m trying to develop a tool that lets you select a file on the server. I’m using the dynamic_options setting to let Galaxy render a dropdown list of available file. However, when I try to test the tool in its current form, I get the following error: <pre> galaxy.web.framework.decorators ERROR 2016-01-26 13:08:35,649 Uncaught exception in exposed API method: Traceback (most recent call last): File "lib/galaxy/web/framework/decorators.py", line 260, in decorator rval = func( self, trans, *args, **kwargs) File "lib/galaxy/webapps/galaxy/api/tools.py", line 98, in build return tool.to_json(trans, kwd.get('inputs', kwd)) File "lib/galaxy/tools/__init__.py", line 2602, in to_json populate_state(trans, self.inputs, state_inputs, state_errors, params.__dict__) File "lib/galaxy/tools/__init__.py", line 2471, in populate_state state[input.name] = input.get_initial_value(trans, context, history=history) File "lib/galaxy/tools/parameters/basic.py", line 984, in get_initial_value value = [ optval for _, optval, selected in options if selected ] ValueError: too many values to unpack </pre> My tool xml looks like this: <pre> <tool id="archivetolibrary" name="Copy files" version="0.1.0"> <description>from the archive to a data library</description> <code file="list_files.py"/> <command interpreter="bash"> test.sh > $log </command> <inputs> <param name="archive_source" type="select" display="radio" label="Choose source archive"> <options from_file="archives.loc"> <column name="name" index="0"/> <column name="value" index="1"/> </options> </param> <param name="query" type="text" label="Search for files"/> <param name="selected_file" type="select" label="Choose your file" dynamic_options="list_files(archive_source,query)"/> </inputs> <outputs> <data name="log" format="txt" label="${tool.name} on ${on_string}"></data> </outputs> <tests> <test> </test> </tests> <help> This tool copies data from and archive folder to your history. Keep in mind this folder MUST be readable by Galaxy in order for this tool to work. </help> </tool> </pre> and the script called in the “code” tag looks like this <pre> def list_files(dir,query): listing=[] for fname in listdir(dir): fullpath = path.join(dir, fname) #only select datafiles, not md5 checksums if path.isfile(fullpath) and not fullpath.endswith(".md5"): listing.append( fname ) return listing </pre> This script should return a list with filenames that should be displayed in the dropdown list. I’m tracking Galaxy release 15.10, running on a Ubuntu 14.04.3 server Thanks for your help! Matthias

3 3

Storing data in the user session
by Steve Cassidy 18 Oct '16

18 Oct '16

Hi again, I’m looking for the right way to store some user credentials in the galaxy session so that tools can work on behalf of the user with our repository. Currently users have an API key and they need to upload it as a data item to that is then passed to each tool that needs it as input. This doesn’t seem like the right solution since the API key becomes part of the history and so would be shared if the history were shared. What would be better would be a way of storing the API key in the user session and then being able to pass that into a tool. I note that there are a few user session variables available in the tool xml file: __user__, __user_email__, __user_name__ and __user_id__. There is also a user preferences page where they can fill out a few details. However, I can’t see a mechanism to extend this in any way and have extra properties in the preferences pane that would then be available to tools via the template file. I can see that the standard practice for the data sources on usegalaxy.org<http://usegalaxy.org> is to send the user off to a repository website to find data that is then posted back to the galaxy server. This isn’t appropriate for us since the interaction with the repository is not just for downloading a single dataset - tools will query the repository, download various kinds of data and possibly upload new data, so we need to store user credentials in some way. Is there already a good way to achieve this or is this an enhancement to Galaxy? Thanks, Steve — Department of Computing, Macquarie University http://web.science.mq.edu.au/~cassidy

5 17

workflow import error
by Tony Schreiner 30 Sep '16

30 Sep '16

We have 2 galaxy instances running one is v15.05-74-gf6b894a the other is v15.10-30-g6a47d2d a user is trying to import a workflow file, it works on the older one, but raises an error on the newer one. URL: http://redacted.edu:8080/workflow/import_workflow <http://prince.bc.edu:8080/workflow/import_workflow> File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/middleware/error.py', line 151 in __call__ app_iter = self.application(environ, sr_checker) File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 133 in __call__ return self.handle_request( environ, start_response ) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line 191 in handle_request body = method( trans, **kwargs ) File '/cluster/home/galaxy/galaxy/lib/galaxy/webapps/galaxy/controllers/workflow.py', line 921 in import_workflow workflow, missing_tool_tups = self._workflow_from_dict( trans, data, source=src, add_to_menu=add_to_menu ) File '/cluster/home/galaxy/galaxy/lib/galaxy/web/base/controller.py', line 1111 in _workflow_from_dict publish=publish File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line 159 in build_workflow_from_dict module, step = self.__module_from_dict( trans, step_dict, secure=False ) File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line 605 in __module_from_dict module.save_to_step( step ) File '/cluster/home/galaxy/galaxy/lib/galaxy/workflow/modules.py', line 592 in save_to_step step.tool_inputs = self.tool.params_to_strings( self.state.inputs, self.trans.app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/__init__.py', line 1980 in params_to_strings return params_to_strings( self.inputs, params, app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/__init__.py', line 92 in params_to_strings value = params[ key ].value_to_basic( value, app ) File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/grouping.py', line 574 in value_to_basic rval[ self.test_param.name ] = self.test_param.value_to_basic( value[ self.test_param.name ], app ) KeyError: 'uncol' i'm not able conclude much from this dump. Can anyone suggest some steps to debug this. Cheers Tony Schreiner

2 1

Re: [galaxy-dev] how to create users through Galaxy API?
by Martin Čech 03 Sep '16

03 Sep '16

Hi, you specify username, password and email in the body (payload) of the POST as Key:Value pairs. Code from the API method: ( https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/ga…) username = payload[ 'username' ] email = payload[ 'email' ] password = payload[ 'password' ] There are also other conditions that need to be fulfilled (e.g. user creation has to be turned on in the configuration) - you will find these when you look at the source code of the method (because the documentation is not perfect yet, sorry). M. On Sun, Dec 29, 2013 at 11:22 PM, xlwang <xlwang_0903(a)163.com> wrote: > hi, > I still have some confusion. > post /api/users , when i use postman, it seems like this: > > > No other parameters? How to confirm the new user's information?Such as > user name, password. > Can you give me a example please?Thank you! > > At 2013-12-30 06:04:15,"Martin Čech" <marten(a)bx.psu.edu> wrote: > > Hello, > > to create a user through API you send a POST request to /api/users > > More information about User API here: > > > https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galaxy.… > > Martin > > > On Fri, Dec 27, 2013 at 5:11 AM, xlwang <xlwang_0903(a)163.com> wrote: > >> hello, >> I want to create users through Galaxy API.For example, I can get users >> information through url like this(use postman): >> http://localhost:8080/api/users?key=my_key >> But how to create a user? >> >> >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > > > >

6 7

Consuming dataset collections
by Steve Cassidy 17 Aug '16

17 Aug '16

Hi all, I’m staring at the discussion of handling dataset collections: http://planemo.readthedocs.io/en/latest/_writing_collections.html but failing to see the solution to my problem. I have a tool that creates a dataset collection, a group of files with names like 1_1308_1_2_092-ch6-speaker16.TextGrid where the 1_1308_1_2_092 part is a unique identifier that I’d like to keep track of. I’ve used a discover_datasets tag in the tool xml file to match my output filenames and extract the designation (1_1308_1_2_092-ch6-speaker16.TextGrid) and the ext (TextGrid). I have another tool that runs a query over these files and generates a single tabular result that will ideally include the identifier in some form. Here’s the command section for that tool: query_textgrids.py --textgrid "${",".join(map(str, $textgrid))}" --tier $tier --regex '$regex' --output_path $output where ‘$textgrid’ is one of my input parameters that has multiple=“true” set so that it can be a dataset collection. That works ok but the input I get are the filenames (dataset_1.dat, etc.) not the name of the datasets. The page above mentions something called the ‘element_identifier’ and gives this funky example: merge_rows --name "${re.sub('[^\w\-_]', '_', $input.element_identifier)}" --file "$input" --to $output; I can’t see what this element_identifier thing is - the suggestion is that it might be the dataset name, but I’m not sure. Also I don’t understand why the command above is doing replacement of whitespace with underscores. If this is the name I’m after, it would seem that I’d need to pass these names along with the textgrid files and then pair them up inside my script - is that what I need to do? All of this cries out to me for a more explicit representation of a dataset collection that my tool can create and consume rather than this hacky treatment of filenames. If I could generate a manifest file of some kind describing my dataset collection then none of this parsing of filenames would be needed. I could also consume the manifest file as well and it could be used for collection level metadata. Is this a silly idea? Anyway, any help with my immediate problem would be appreciated. Thanks, Steve — Department of Computing, Macquarie University http://web.science.mq.edu.au/~cassidy

2 3

Error running JBrowse
by Raymond Wan 17 Aug '16

17 Aug '16

Dear all, I'm trying to run the JBrowse tool within Galaxy. The installation was fine (I think), but when I run the tool, I get this error: Fatal error: Exit code 1 () Traceback (most recent call last): File "/d15/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/d022577ff9cf/jbrowse/jbrowse.py", line 11, in <module> from Bio.Data import CodonTable ImportError: No module named Does this mean I'm missing something? In the history, if I click the "(i)" icon for the incomplete job (in red), everything looks fine, except that it says: Tool Version: python: can't open file 'jbrowse.py': [Errno 2] No such file or directory However, "jbrowse.py" does exist in the path above. Any idea what could be wrong? Thank you! Ray

2 2

Adding a download link to the "Download or Export Workflow" page with load balancing
by Alex Rodriguez 12 Aug '16

12 Aug '16

Dear Galaxy developers, I am interested in adding a download link to the “Download or Export Workflow” page of a workflow. I am able to add the link through the files in the templates directory and correctly add the necessary API functions in the lib/webapps/galaxy/controllers/workflow.py file. I wanted to ask for some help in figuring out what I might be missing when I try to download a file when I have load balancing feature activated. The page looks something like this: Download to File Download workflow to file so that it can be saved or imported into another Galaxy server. <https://dev.globusgenomics.org/u/arodri7/w/workflow-1-for-asms-2016-worksho…> New Test Download ( test file) URL for Importing to Another Galaxy This workflow must be accessible before it can be imported into another Galaxy. <https://dev.globusgenomics.org/workflow/sharing?id=a728237e0d32bb8c> As you can see I added the “New test Download” link. I have added and tested the function which gets me the information about the workflow I want correctly. For now it’s just a “hello world” string. When I don’t use load balancing I get the contents I want downloaded to the file correctly. However, when I activate load balancing, it only downloads an empty file. I can print the contents of the test download file to the galaxy log, but as I said nothing gets printed on the actual file. Is there some javascript code I also need to add for this functionality that I might be missing such that the download works with load balancing? As I mentioned I do get a proper download when load balancing is not activated. BTW, all other existing download links work properly with both load balancing and no load balancing. Thanks, Alex

2 4

Fwd: Galaxy Blast
by Peter Cock 02 Aug '16

02 Aug '16

Hello all, Harry (CC'd) emailed me about some teething trouble getting nucleotide BLAST databases to appear in the drop down lists (see below). I suspect this is partly due to the partial data table sample that ships with Galaxy: https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_con… Can any of the Galaxy Data Manager expects advise please? Thanks, Peter ---------- Forwarded message ---------- From: Harry Wright <Harry.Wright(a)moredun.ac.uk> Date: Tue, Jul 26, 2016 at 9:25 AM Subject: RE: Galaxy Blast To: Peter Cock <p.j.a.cock(a)googlemail.com> Hi Peter, Sorry for the delay in response. If you think it would be useful please post it. I have downloaded what I thought was the newest versions of NCIB blast+ 2.2.31 from the toolshed. The tool_data_table_conf.xml.sample I altered was in galaxy_dir/config/. If you need anything else please ask. Thanks Harry -----Original Message----- From: Peter Cock [mailto:p.j.a.cock@googlemail.com] Sent: 22 July 2016 16:28 To: Harry Wright Subject: Re: Galaxy Blast Hi Harry, Hmm. I'm a little puzzled. That file looks fine on our GitHub and on the Tool Shed: https://github.com/peterjc/galaxy_blast/blob/master/tool-data/tool_data_tab… http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus (browse tip files) Can you tell me exactly which version of things you got from the Tool Shed, and exactly which tool_data_table_conf.xml and tool_data_table_conf.xml.sample you had to edit? My guess is the problem is this partial example that ships with Galaxy: https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_con… (There is talk of moving datatypes like this back into the Galaxy core, which would simply this) May I forward this email to the public Galaxy developers mailing list please? Thanks, Peter On Fri, Jul 22, 2016 at 3:59 PM, Harry Wright <Harry.Wright(a)moredun.ac.uk> wrote: > > > Dear Peter, > > > > I have just installed a new instance of Galaxy here at Moredun and > have been busy setting up tools for us to use. One of the tools that I > have set up is blast with the NCBI nr and nt databases sitting in the > backend as a Locally installed database. I downloaded both files and > formatted them into databases. I then add lines to the blastdb_p.loc > and blastdb.loc. When I restarted the server I found that I could > access nr through the interface, for blastx, but not nt, using blastn. > For nt I was getting the message “No option available” in the > Nucleotide BLAST database drop down when I selected “Locally installed BLAST database”. > > > > After some searching I found reference to > config/tool_data_table_conf.xml.sample. When I look there I could only > find reference to blastdb_p and not blastdb. > > > > I added the lines below to the file and changed the file name to > tool_data_table_conf.xml. I restarted galaxy to find that I could now > use nt in blastn. > > > >  > > <table name="blastdb" comment_char="#"> > > <columns>value, name, path</columns> > > <file path="tool-data/blastdb.loc" /> > > </table> > > > > I was wondering if there was something that I have missed or has this > section of the code been missed out of the tool_data_table_conf.xml > file in the installation download? > > > > Regards > > Harry > > > > Dr. Harry Wright > > Moredun Research Institute > > Pentlands Science Park > > Bush Loan > > Penicuik > > Midlothian EH26 0PZ > > Tel - 0131 4455111 > > Fax - 0131 4456235 > > > > The Moredun Research Institute is a limited company registered in Scotland. > Registered number: SC149440 > > The Moredun Research Institute is a charity registered in Scotland. > Registered number: SC022353 > > Registered office: Moredun Research Institute, Pentlands Science Park, > Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK > > The Moredun Research Institute is a limited company registered in Scotland. Registered number: SC149440 The Moredun Research Institute is a charity registered in Scotland. Registered number: SC022353 Registered office: Moredun Research Institue, Pentlands Science Park, Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK

2 3