- galaxy-dev - lists.galaxyproject.org

use of copied history while original user account deleled (and purged)
by Olivia Doppelt-Azeroual 04 May '15

04 May '15

Dear Developers, We manage a Galaxy instance at Institut Pasteur where there are more than a hundred users. Among them there are postdocs that are meant to leave the institute. Our Galaxy instance is configured with LDAP authentication. The LDAP entry is suppressed shortly after the end of contracts. We are facing the following problem: A user left a few month ago, 1/ before leaving, she shared interesting histories with another colleague of the lab. 2/ before leaving, the colleague created a copy of every shared history in her own Galaxy account. 3/ The colleague made a few tests to check that the data were really transfered, by displaying them or downloading them and everything was ok. 4/ The time passed and the ldap user account of the user who left was suppressed. We deleted then purged her Galaxy account. 5/ The colleague has tried to relaunch some analysis using the copied histories. The data are there as her colleague is able to download the files or display them, but the jobs haven't been launched. They have remained grey in the copied history. The problem is that there are no logs at all on the galaxy side. No command line is generated either. On the reporting side of Galaxy, we see that the jobs have been created but their status remains "NEW' We know that this will be a recurrent problem if we can't resolve it. Have someone already complained about something like that? Best regards, -- Olivia Doppelt-Azeroual, PhD Fabien Mareuil, PhD Bioinformatics Engineer Galaxy Team - CIB/C3BI Institut Pasteur, Paris

2 1

Possible to pass hostName to a tool?
by Ben Bimber 01 May '15

01 May '15

Hello, I apologize for the second email as this is related to a question from two days ago. I am writing a galaxy tool that runs code that will query the galaxy server. Within the tool wrapper XML, is there a way to write the API URL of the current server? I dont see anything along these lines documented. Thanks in advance for any help, Ben

2 1

Fwd: Failure to set attributes_type metadata for interval files / error in gff_filter_by_attribute tool
by Marius van den Beek 01 May '15

01 May '15

Hello all, Not sure if the mailing list or trello would be appropriate for this. I had a problem using the gff_filter_by_attribute tool, both locally and on usegalaxy.org. Neither filtering for GTF or GFF were working. The error message was: Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 158, in prepare_job job_wrapper.prepare() File "/home/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 832, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/home/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 408, in build self.__build_command_line( ) File "/home/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 424, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/home/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/home/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "DynamicallyCompiledCheetahTemplate.py", line 92, in respond NotFound: cannot find 'attribute_types' while searching for 'input.metadata.attribute_types' I could fix it by importing safe_dumps from galaxy.util.json, which apparently was not imported. The next problem occurred because gff_filter_by_attribute expects GTF attribute name-value separators (" ") but allowed GFF input, which uses ("=") as separator. I added a logic to switch between seperators based on the file extension. Everything works smoothly now, the code changes are here: https://github.com/mvdbeek/galaxy Should I create a pull request? -- Marius van den Beek, PhD Drosophila Genetics and Epigenetics Institut de Biologie Paris Seine <http://www.ibps.upmc.fr/fr/Recherche/umr-biologie-developpement/genetique-e…> 9, Quai St Bernard, Boîte courrier 24 75252 Paris Cedex 05 Tel +33 01 44 27 34 01

2 2

May 2015 Galactic News (and 2 GCC2015 deadlines today)
by Dave Clements 01 May '15

01 May '15

1 0

Configuring galaxy to run on LSF
by Asma Riyaz 30 Apr '15

30 Apr '15

Hi, Configuration of Galaxy for an LSF server led me to this link: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#DRMAA The issue being the /lsf folder under galaxy does not exist. I am sort of stuck at this point and would highly appreciate anybody's help in this regard. Thank you, Asma

2 1

Migrating user account from galaxy specific to LDAP
by Teshome Dagne Mulugeta 30 Apr '15

30 Apr '15

?Hi, We are currently using galaxy specific login using email address. We changed the authentication to LDAP. Most users email address doesn't match with their registered email address in LDAP. Now, users are not able to see their previous works which is expected of course but I have no clue how to fix it. Please help. Cheers, Teshome

4 5

Galaxy for Natural Language Processing
by Keith Suderman 30 Apr '15

30 Apr '15

Hello, Our group is investigating using Galaxy as a workflow engine for NLP (Natural Language Processing) tasks. I have installed a local Galaxy instance and created wrappers for the services we use and so far everything is working great. I do have a few questions and they all fall under the “Advanced Topics” section as defined at the end of the tutorial for creating a Histogram [1] 1. parameter validation: Many of our tools rely on additions made by previous tools in the workflow; for example, a tool that identifies noun phrases may require that the input has been run through a part of speech (POS) tagger, the POS tagger may require that the input has been run through a tokenizer, etc. Our tools can do this validation, I am just looking for a way to wire this into Galaxy so a user can only connect tools in the workflow editor if this validation passes. I have been looking looking at the code for lib/galaxy/tools/parameters/validation.py and I don’t see anything that I can (easily) bend to our use case. What I was hoping for was something like: <input type=“data” format=“our_custom_format” name=“input”> <validator type=“dataset_custom”> <command interpreter=“bash”>validate.sh $input</command> <!— OR —> <tool file=“custom_validator.xml”/> </validator> </input> I also see the tantalizing sentence, "Custom code execution at various time points of the workflow that allows a fine grained control over the execution process", but I can't find any examples of how this is done. 2. data repositories / data collections I need to be able to process collections of data pulled from remote servers. I have been looking at DataManagers and data collections in Galaxy, but everything seems to assume the data is local to the server, or can be copied/uploaded to the server. For practical and legal reasons beyond my pay grade this is not a solution in our case. For example, an organization may be willing to allow our users to query their service for documents, run the documents through our workflow, and store the intermediate results; but they will not allow us to copy their data to another server verbatim. There are possibilities for me to cache data, but the general use case is that I have to call an external service to fetch documents one at a time and then run the same workflow on each document. Any suggestions on how to accomplish this in Galaxy? I can do single documents, I just need to expand this to include collections of documents. A typical workflow might look something like: a) Query Tool -> Server, find all documents that contain the word “cheese” b) Server -> Here is the list of document IDs [ id1, id2, …, idn ] c) WorkFlow -> for each id in the list do c1) Download document c2 ) Work work work work… c3) Persist output I can do all of the above except the most important bit; iterating… 3. format conversion: Is it possible for Galaxy to automatically convert between formats when designing a workflow? I see the <change_format/> tag, but that seems to change the output format of a tool based on the input (or some other condition) in the same tool; I need to be able to change the format based on the input requirements of the next tool in the workflow. For example, if Tool A produces format X, Tool B requires format Y, and a converter from X to Y has been defined in the datatypes_conf.xml; I would like for Galaxy to implicitly insert the converter from X to Y when I drag the output noodle from Tool A to Tool B in the designer. Is this possible? 4. OAuth 2.0 / OpenID Connect: I need to be able to fetch documents from data providers that require an OAuth 2.0 access token. Currently, I use a separate service to go through the OAuth authentication/authorization process and then have the user copy/paste their access token into a text field in Galaxy. Is there a way to perform the OAuth authentication dance required by the remote service inside Galaxy itself? I’ve looked at the Trello site for Galaxy and see that both OAuth 2.0 and OpenID Connect are on the radar, hopefully this use case is being considered as well. I’m sure to have more questions after working through some visualization examples, but this should keep me busy for now. Sincerely, Keith Suderman REFERENCES 1. https://wiki.galaxyproject.org/Admin/Tools/AddingTools ------------------------------ Research Associate Department of Computer Science Vassar College Poughkeepsie, NY

4 9

new tool works correctly but galaxy thinks it failed
by Ryan G 30 Apr '15

30 Apr '15

Hi - I implemented a tool I us into Galaxy but when I run the tool view Galaxy, Galaxy thinks the tool failed. I checked the output file the tools create, and it succeeded. How do I tell Galaxy that a tool it ran was successful? Is it something in my XML file? The tool is outputting some info to stderr. Could this be why?

6 10

Re: [galaxy-dev] <code> tagset in tools
by Simon Gladman 30 Apr '15

30 Apr '15

Hi Dan, Thanks for the reply and the suggestions. I think the data libraries would work much better than the get_microbial_data tool as then it would be more easily kept up to date. Cheers, Simon. On 30 April 2015 at 01:01, Daniel Blankenberg <dan(a)bx.psu.edu> wrote: > The <code> tag won’t be going away any time soon, but in many cases there > are other ways to do what this tag does, using the built-in dynamic options > (in particular, currently for tab-delimited files), that are much > preferred. So it is deprecated, and it should be used sparingly, but there > are times when there isn’t a better way for tools to dynamically build > their options without using a code file. > > In the case of the get microbial data tool, if it was working prior to > 15.03, it should still be working, afaik we haven’t changed the handling of > these recently (with the exception of not loading them inside of a ToolShed > application instance). Its been a while since I’ve played with that > specific tool (and we deprecated it ourselves and removed it from main), so > I can’t confirm off-the-cuff whether it should still be working or not. If > something really did break with respect to the code files, we should take a > look though. > > > I think using a Data Library is the perfect replacement for for the > microbial import tool, and would cause less file duplication when used. But > what exactly is no longer working? > > > Thanks for using Galaxy, > > Dan > > On Apr 29, 2015, at 10:38 AM, Peter Cock <p.j.a.cock(a)googlemail.com> > wrote: > > > On Wed, Apr 29, 2015 at 6:33 AM, Simon Gladman > > <simon.gladman(a)unimelb.edu.au> wrote: > >> Hi dev, > >> > >> Now that the <code> tagset is deprecated, how do we replace that > >> functionality. For example the "get microbial data" tool runs the script > >> listed in the <code> tagset to load the list of available microbe data > when > >> the tool is selected. This doesn't seem to work in the latest stable > 15.03.1 > >> or 15.03.2. > >> > >> How do we configure tools like this now? Or is there an updated version > of > >> the microbial_import tool? > >> > >> Cheers, > >> > >> Simon. > > > > Good question Simon. > > > > I am still using <code> for advanced parameter validation, e.g. > > > https://github.com/peterjc/pico_galaxy/blob/64aecfa6041b3b7fe2c70a30236a7aa… > > > > i.e. Validation before the job gets submitted to the cluster or run. > > If the <code> tag suddenly goes away, my tool will still work but > > would be a little less user friendly (some errors would only be > > spotted after the tool starts running). > > > > Peter > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > > > -- Simon Gladman | Bioinformatician Victorian Life Sciences Computation Initiative - LSCC 187 Grattan Street, The University of Melbourne | Victoria 3010 Australia T +61 3 9035 5822 | M 0418 421 077 Mail: simon.gladman(a)unimelb.edu.au Web: http://www.vlsci.org.au

1 0

Data Libraries
by Ryan G 29 Apr '15

29 Apr '15

Hi all - We are trying to use Galaxy as a mechanism for our sequencing lab to create data libraries for data they generate. I noticed in the docs, only Admins are able to create data libraries. Is there a way to change this? I'd like to give specific users in our group this ability without giving them admin rights. Ryan

3 4