Running jobs as real user and extra_file_path
by Louise-Amélie Schmitt
Hi everyone,
I just wanted to ask how the extra_file_path is handled in case of job
running as the real user since the file_path is only writable by the
galaxy user. Any clue?
Thanks,
L-A
2 years, 2 months
Python script can't can't import BioPython modules in Galaxy 16.01
by Makis Ladoukakis
Hello everyone,
I've recently upgraded to Galaxy 16.01 and i have some issues with a python wrapper from a tool. When I run it via Galaxy I get the following error:
Traceback (most recent call last):
File "/home/galaxy_user/galaxy/tools/myTools/know_parser/know_parser.py", line 12, in <module>
from Bio import SeqIO
ImportError: No module named Bio
However when I click on the "view details" icon to copy the command from the tool and run it on my command line the tool works perfectly. In addition this issue occurs while I already have installed BioPython.
When I type Python and write "from Bio import SeqIO" I get no error.
Can this be an issue related to the virtual environment that is initialized each time Galaxy starts?
Thank you in advance,
Makis Ladoukakis
2 years, 6 months
Python ValueError: too many values to unpack
by Matthias De Smet
Hi all,
I’m trying to develop a tool that lets you select a file on the server. I’m using the dynamic_options setting to let Galaxy render a dropdown list of available file.
However, when I try to test the tool in its current form, I get the following error:
<pre>
galaxy.web.framework.decorators ERROR 2016-01-26 13:08:35,649 Uncaught exception in exposed API method:
Traceback (most recent call last):
File "lib/galaxy/web/framework/decorators.py", line 260, in decorator
rval = func( self, trans, *args, **kwargs)
File "lib/galaxy/webapps/galaxy/api/tools.py", line 98, in build
return tool.to_json(trans, kwd.get('inputs', kwd))
File "lib/galaxy/tools/__init__.py", line 2602, in to_json
populate_state(trans, self.inputs, state_inputs, state_errors, params.__dict__)
File "lib/galaxy/tools/__init__.py", line 2471, in populate_state
state[input.name] = input.get_initial_value(trans, context, history=history)
File "lib/galaxy/tools/parameters/basic.py", line 984, in get_initial_value
value = [ optval for _, optval, selected in options if selected ]
ValueError: too many values to unpack
</pre>
My tool xml looks like this:
<pre>
<tool id="archivetolibrary" name="Copy files" version="0.1.0">
<description>from the archive to a data library</description>
<code file="list_files.py"/>
<command interpreter="bash">
test.sh > $log
</command>
<inputs>
<param name="archive_source" type="select" display="radio" label="Choose source archive">
<options from_file="archives.loc">
<column name="name" index="0"/>
<column name="value" index="1"/>
</options>
</param>
<param name="query" type="text" label="Search for files"/>
<param name="selected_file" type="select" label="Choose your file" dynamic_options="list_files(archive_source,query)"/>
</inputs>
<outputs>
<data name="log" format="txt" label="${tool.name} on ${on_string}"></data>
</outputs>
<tests>
<test>
</test>
</tests>
<help>
This tool copies data from and archive folder to your history. Keep in mind this folder MUST be readable by Galaxy in order for this tool to work.
</help>
</tool>
</pre>
and the script called in the “code” tag looks like this
<pre>
def list_files(dir,query): listing=[]
for fname in listdir(dir):
fullpath = path.join(dir, fname) #only select datafiles, not md5 checksums
if path.isfile(fullpath) and not fullpath.endswith(".md5"):
listing.append( fname )
return listing
</pre>
This script should return a list with filenames that should be displayed in the dropdown list.
I’m tracking Galaxy release 15.10, running on a Ubuntu 14.04.3 server
Thanks for your help!
Matthias
2 years, 9 months
Storing data in the user session
by Steve Cassidy
Hi again,
I’m looking for the right way to store some user credentials in the galaxy session so that tools can work on behalf of the user with our repository.
Currently users have an API key and they need to upload it as a data item to that is then passed to each tool that needs it as input. This doesn’t seem like the right solution since the API key becomes part of the history and so would be shared if the history were shared.
What would be better would be a way of storing the API key in the user session and then being able to pass that into a tool.
I note that there are a few user session variables available in the tool xml file: __user__, __user_email__, __user_name__ and __user_id__. There is also a user preferences page where they can fill out a few details. However, I can’t see a mechanism to extend this in any way and have extra properties in the preferences pane that would then be available to tools via the template file.
I can see that the standard practice for the data sources on usegalaxy.org<http://usegalaxy.org> is to send the user off to a repository website to find data that is then posted back to the galaxy server. This isn’t appropriate for us since the interaction with the repository is not just for downloading a single dataset - tools will query the repository, download various kinds of data and possibly upload new data, so we need to store user credentials in some way.
Is there already a good way to achieve this or is this an enhancement to Galaxy?
Thanks,
Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy
2 years, 9 months
workflow import error
by Tony Schreiner
We have 2 galaxy instances running
one is
v15.05-74-gf6b894a
the other is
v15.10-30-g6a47d2d
a user is trying to import a workflow file, it works on the older one, but
raises an error on the newer one.
URL: http://redacted.edu:8080/workflow/import_workflow
<http://prince.bc.edu:8080/workflow/import_workflow>
File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/middleware/error.py',
line 151 in __call__
app_iter = self.application(environ, sr_checker)
File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py',
line 84 in __call__
return self.application(environ, start_response)
File '/cluster/home/galaxy/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py',
line 633 in __call__
return self.application(environ, start_response)
File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line
133 in __call__
return self.handle_request( environ, start_response )
File '/cluster/home/galaxy/galaxy/lib/galaxy/web/framework/base.py', line
191 in handle_request
body = method( trans, **kwargs )
File '/cluster/home/galaxy/galaxy/lib/galaxy/webapps/galaxy/controllers/workflow.py',
line 921 in import_workflow
workflow, missing_tool_tups = self._workflow_from_dict( trans, data,
source=src, add_to_menu=add_to_menu )
File '/cluster/home/galaxy/galaxy/lib/galaxy/web/base/controller.py', line
1111 in _workflow_from_dict
publish=publish
File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line
159 in build_workflow_from_dict
module, step = self.__module_from_dict( trans, step_dict, secure=False )
File '/cluster/home/galaxy/galaxy/lib/galaxy/managers/workflows.py', line
605 in __module_from_dict
module.save_to_step( step )
File '/cluster/home/galaxy/galaxy/lib/galaxy/workflow/modules.py', line 592
in save_to_step
step.tool_inputs = self.tool.params_to_strings( self.state.inputs,
self.trans.app )
File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/__init__.py', line 1980
in params_to_strings
return params_to_strings( self.inputs, params, app )
File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/__init__.py',
line 92 in params_to_strings
value = params[ key ].value_to_basic( value, app )
File '/cluster/home/galaxy/galaxy/lib/galaxy/tools/parameters/grouping.py',
line 574 in value_to_basic
rval[ self.test_param.name ] = self.test_param.value_to_basic( value[
self.test_param.name ], app )
KeyError: 'uncol'
i'm not able conclude much from this dump. Can anyone suggest some steps to
debug this.
Cheers
Tony Schreiner
2 years, 9 months
Re: [galaxy-dev] how to create users through Galaxy API?
by Martin Čech
Hi,
you specify username, password and email in the body (payload) of the POST
as Key:Value pairs.
Code from the API method: (
https://galaxy-central.readthedocs.org/en/latest/_modules/galaxy/webapps/...)
username = payload[ 'username' ] email = payload[ 'email' ] password =
payload[ 'password' ]
There are also other conditions that need to be fulfilled (e.g. user
creation has to be turned on in the configuration) - you will find these
when you look at the source code of the method (because the documentation
is not perfect yet, sorry).
M.
On Sun, Dec 29, 2013 at 11:22 PM, xlwang <xlwang_0903(a)163.com> wrote:
> hi,
> I still have some confusion.
> post /api/users , when i use postman, it seems like this:
>
>
> No other parameters? How to confirm the new user's information?Such as
> user name, password.
> Can you give me a example please?Thank you!
>
> At 2013-12-30 06:04:15,"Martin Čech" <marten(a)bx.psu.edu> wrote:
>
> Hello,
>
> to create a user through API you send a POST request to /api/users
>
> More information about User API here:
>
>
> https://galaxy-central.readthedocs.org/en/latest/lib/galaxy.webapps.galax...
>
> Martin
>
>
> On Fri, Dec 27, 2013 at 5:11 AM, xlwang <xlwang_0903(a)163.com> wrote:
>
>> hello,
>> I want to create users through Galaxy API.For example, I can get users
>> information through url like this(use postman):
>> http://localhost:8080/api/users?key=my_key
>> But how to create a user?
>>
>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/
>>
>
>
>
>
2 years, 10 months
Consuming dataset collections
by Steve Cassidy
Hi all,
I’m staring at the discussion of handling dataset collections:
http://planemo.readthedocs.io/en/latest/_writing_collections.html
but failing to see the solution to my problem.
I have a tool that creates a dataset collection, a group of files with names like 1_1308_1_2_092-ch6-speaker16.TextGrid where the 1_1308_1_2_092 part is a unique identifier that I’d like to keep track of. I’ve used a discover_datasets tag in the tool xml file to match my output filenames and extract the designation (1_1308_1_2_092-ch6-speaker16.TextGrid) and the ext (TextGrid).
I have another tool that runs a query over these files and generates a single tabular result that will ideally include the identifier in some form. Here’s the command section for that tool:
query_textgrids.py --textgrid "${",".join(map(str, $textgrid))}" --tier $tier --regex '$regex' --output_path $output
where ‘$textgrid’ is one of my input parameters that has multiple=“true” set so that it can be a dataset collection. That works ok but the input I get are the filenames (dataset_1.dat, etc.) not the name of the datasets.
The page above mentions something called the ‘element_identifier’ and gives this funky example:
merge_rows --name "${re.sub('[^\w\-_]', '_', $input.element_identifier)}" --file "$input" --to $output;
I can’t see what this element_identifier thing is - the suggestion is that it might be the dataset name, but I’m not sure. Also I don’t understand why the command above is doing replacement of whitespace with underscores.
If this is the name I’m after, it would seem that I’d need to pass these names along with the textgrid files and then pair them up inside my script - is that what I need to do?
All of this cries out to me for a more explicit representation of a dataset collection that my tool can create and consume rather than this hacky treatment of filenames. If I could generate a manifest file of some kind describing my dataset collection then none of this parsing of filenames would be needed. I could also consume the manifest file as well and it could be used for collection level metadata. Is this a silly idea?
Anyway, any help with my immediate problem would be appreciated.
Thanks,
Steve
—
Department of Computing, Macquarie University
http://web.science.mq.edu.au/~cassidy
2 years, 11 months
Error running JBrowse
by Raymond Wan
Dear all,
I'm trying to run the JBrowse tool within Galaxy. The installation
was fine (I think), but when I run the tool, I get this error:
Fatal error: Exit code 1 () Traceback (most recent call last): File
"/d15/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/jbrowse/d022577ff9cf/jbrowse/jbrowse.py",
line 11, in <module> from Bio.Data import CodonTable ImportError: No
module named
Does this mean I'm missing something?
In the history, if I click the "(i)" icon for the incomplete job (in
red), everything looks fine, except that it says:
Tool Version: python: can't open file 'jbrowse.py': [Errno 2] No such
file or directory
However, "jbrowse.py" does exist in the path above. Any idea what
could be wrong?
Thank you!
Ray
2 years, 11 months
Adding a download link to the "Download or Export Workflow" page with load balancing
by Alex Rodriguez
Dear Galaxy developers,
I am interested in adding a download link to the “Download or Export Workflow” page of a workflow. I am able to add the link through the files in the templates directory and correctly add the necessary API functions in the lib/webapps/galaxy/controllers/workflow.py file.
I wanted to ask for some help in figuring out what I might be missing when I try to download a file when I have load balancing feature activated. The page looks something like this:
Download to File
Download workflow to file so that it can be saved or imported into another Galaxy server. <https://dev.globusgenomics.org/u/arodri7/w/workflow-1-for-asms-2016-works...>
New Test Download ( test file)
URL for Importing to Another Galaxy
This workflow must be accessible before it can be imported into another Galaxy. <https://dev.globusgenomics.org/workflow/sharing?id=a728237e0d32bb8c>
As you can see I added the “New test Download” link.
I have added and tested the function which gets me the information about the workflow I want correctly. For now it’s just a “hello world” string.
When I don’t use load balancing I get the contents I want downloaded to the file correctly.
However, when I activate load balancing, it only downloads an empty file. I can print the contents of the test download file to the galaxy log, but as I said nothing gets printed on the actual file.
Is there some javascript code I also need to add for this functionality that I might be missing such that the download works with load balancing? As I mentioned I do get a proper download when load balancing is not activated. BTW, all other existing download links work properly with both load balancing and no load balancing.
Thanks,
Alex
2 years, 11 months
Fwd: Galaxy Blast
by Peter Cock
Hello all,
Harry (CC'd) emailed me about some teething trouble getting
nucleotide BLAST databases to appear in the drop down lists
(see below).
I suspect this is partly due to the partial data table sample that
ships with Galaxy:
https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_c...
Can any of the Galaxy Data Manager expects advise please?
Thanks,
Peter
---------- Forwarded message ----------
From: Harry Wright <Harry.Wright(a)moredun.ac.uk>
Date: Tue, Jul 26, 2016 at 9:25 AM
Subject: RE: Galaxy Blast
To: Peter Cock <p.j.a.cock(a)googlemail.com>
Hi Peter,
Sorry for the delay in response. If you think it would be useful please post it.
I have downloaded what I thought was the newest versions of NCIB
blast+ 2.2.31 from the toolshed.
The tool_data_table_conf.xml.sample I altered was in galaxy_dir/config/.
If you need anything else please ask.
Thanks
Harry
-----Original Message-----
From: Peter Cock [mailto:p.j.a.cock@googlemail.com]
Sent: 22 July 2016 16:28
To: Harry Wright
Subject: Re: Galaxy Blast
Hi Harry,
Hmm. I'm a little puzzled. That file looks fine on our GitHub and on
the Tool Shed:
https://github.com/peterjc/galaxy_blast/blob/master/tool-data/tool_data_t...
http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus (browse tip files)
Can you tell me exactly which version of things you got from the Tool
Shed, and exactly which tool_data_table_conf.xml and
tool_data_table_conf.xml.sample you had to edit?
My guess is the problem is this partial example that ships with Galaxy:
https://github.com/galaxyproject/galaxy/blob/dev/config/tool_data_table_c...
(There is talk of moving datatypes like this back into the Galaxy
core, which would simply this)
May I forward this email to the public Galaxy developers mailing list please?
Thanks,
Peter
On Fri, Jul 22, 2016 at 3:59 PM, Harry Wright
<Harry.Wright(a)moredun.ac.uk> wrote:
>
>
> Dear Peter,
>
>
>
> I have just installed a new instance of Galaxy here at Moredun and
> have been busy setting up tools for us to use. One of the tools that I
> have set up is blast with the NCBI nr and nt databases sitting in the
> backend as a Locally installed database. I downloaded both files and
> formatted them into databases. I then add lines to the blastdb_p.loc
> and blastdb.loc. When I restarted the server I found that I could
> access nr through the interface, for blastx, but not nt, using blastn.
> For nt I was getting the message “No option available” in the
> Nucleotide BLAST database drop down when I selected “Locally installed BLAST database”.
>
>
>
> After some searching I found reference to
> config/tool_data_table_conf.xml.sample. When I look there I could only
> find reference to blastdb_p and not blastdb.
>
>
>
> I added the lines below to the file and changed the file name to
> tool_data_table_conf.xml. I restarted galaxy to find that I could now
> use nt in blastn.
>
>
>
> <!-- Locations of nucleotide (mega)blast databases -->
>
> <table name="blastdb" comment_char="#">
>
> <columns>value, name, path</columns>
>
> <file path="tool-data/blastdb.loc" />
>
> </table>
>
>
>
> I was wondering if there was something that I have missed or has this
> section of the code been missed out of the tool_data_table_conf.xml
> file in the installation download?
>
>
>
> Regards
>
> Harry
>
>
>
> Dr. Harry Wright
>
> Moredun Research Institute
>
> Pentlands Science Park
>
> Bush Loan
>
> Penicuik
>
> Midlothian EH26 0PZ
>
> Tel - 0131 4455111
>
> Fax - 0131 4456235
>
>
>
> The Moredun Research Institute is a limited company registered in Scotland.
> Registered number: SC149440
>
> The Moredun Research Institute is a charity registered in Scotland.
> Registered number: SC022353
>
> Registered office: Moredun Research Institute, Pentlands Science Park,
> Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK
>
>
The Moredun Research Institute is a limited company registered
in Scotland. Registered number: SC149440
The Moredun Research Institute is a charity registered in
Scotland. Registered number: SC022353
Registered office: Moredun Research Institue, Pentlands
Science Park, Bush Loan, Penicuik, Midlothian, EH26 0PZ, UK
2 years, 11 months
