July 2013 - galaxy-dev - lists.galaxyproject.org

Support for Galaxy tools on Python
by Hyungro Lee 18 Jul '13

18 Jul '13

Hi, I was wondering if Galaxy provides python library for Galaxy tools. For example, if it did happen, filter and sort tools can be expressed like this: galaxy.filter(param1, param2, ...) and galaxy.sort(param1, param2, ...) instead of using command line tools like filtering.py $param1, $param2, ... I thought it might be useful for python developers with bioblend if they want to extend developments on python after launching galaxy and importing workflows through bioblend. Any plans and thoughts? Best, Hyungro

1 0

possible bug
by Berner, Thomas 18 Jul '13

18 Jul '13

Hey guys, we found a curios bug, but it is a little bit tricky to explain. Here is an example: - Click on "Concatenate datasets tail-to-head" - add a new data set - click on the first data field and select your data set of choice - press "tab" to get to the next data set an select the second data set by typing the number of the data set via the numpad (might only be possible if you have a long history with many files) => the first number is written as an "character", e.g., 1=a, 2=b, and so on. This bug only shows up if you use the numpad to type in the numbers. We tested it with different tools in our local Galaxy instance and in the Galaxy Main instance. Can anybody confirm this behavior? Best regards, Thomas

1 0

Re: [galaxy-dev] Cann't restart server after updating to the latest version
by Joachim Jacob | VIB | 18 Jul '13

18 Jul '13

Hi, Please keep questions on the galaxy-dev list. Can you paste the part of your universe_wsgi.ini that is under # --- HTTP server -------------------- ? Can you also provide the output (as user galaxy) of $ lsof -Pni | grep galaxy I think this error can happen if you try to start Galaxy when another instance is already running; or when you have misconfigured your universe_wsgi.ini. Cheers, Joachim Joachim Jacob Contact details: http://www.bits.vib.be/index.php/about/80-team On 07/18/2013 10:46 AM, Karen Chait wrote: > > Hello Jacob, > > Thanks for your help solving restart problem > http://dev.list.galaxyproject.org/Cann-t-restart-server-after-updating-to-t… > > I am facing the same problem again but this time when running ./run.sh > I get the following error: > > socket.error: [Errno 98] Address already in use > > galaxy.jobs.handler INFO 2013-07-18 03:28:46,328 sending stop signal > to worker thread > > galaxy.jobs.handler INFO 2013-07-18 03:28:46,328 job handler queue stopped > > galaxy.jobs.runners INFO 2013-07-18 03:28:46,328 LWRRunner: Sending > stop signal to monitor thread > > galaxy.jobs.runners INFO 2013-07-18 03:28:46,328 LWRRunner: Sending > stop signal to 3 worker threads > > galaxy.jobs.runners INFO 2013-07-18 03:28:46,328 LocalRunner: Sending > stop signal to 5 worker threads > > galaxy.jobs.handler INFO 2013-07-18 03:28:46,329 sending stop signal > to worker thread > > galaxy.jobs.handler INFO 2013-07-18 03:28:46,329 job handler stop > queue stopped > > Exception in thread JobHandlerQueue.monitor_thread (most likely raised > during interpreter shutdown): > > Do you know how to solve the problem? > > Thanks > > Karen >

1 0

Dynamically Generated Tools
by Michael E. Cotterell 17 Jul '13

17 Jul '13

TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool "tool shed" compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an "hg pull" or if it's more complicated than that)? Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however, if that's not the recommended course of action, then I'd like some advice on what to do? Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) mepcott(a)uga.edu (mailto:mepcott@uga.edu) mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) http://michaelcotterell.com/

4 6

Java JAXB for Tool Configuration Files
by Michael E. Cotterell 17 Jul '13

17 Jul '13

Hey guys, I don't know if there will be much interest in this, but in the course of revising code that my predecessors wrote, I discovered that they were hand crafting tool XML configuration files. Since we all know that makes reading and writing such files hard, I started work on creating a Java model for the tool config XML using JAXB. In short, I can create and populate some Java objects and then marshal them directly to XML, knowing that they'll be in the correct format. Also, it makes reading/unmarshalling tool config files easy as well. If there's any interest in this, I can probably separate out the model code into a separate project. At the moment, it's a subset of the tool config XML model that I need for my Web service tools, but I could foresee completing it in hopes of it being more useful to others. Imagine: define your tool using Java and have it print out the pretty XML! Also, there should be an XSD for the tool config files. I can generate one once my Java model is complete, but it's kind of important. In the wiki, it says something to the effect that all is needed is valid XML… That's not entirely correct. It should valid XML that conforms to an XML schema definition (now I'm ranting :P). If one already exists, then all the better! Anyway, let me know if there's any interest. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) mepcott(a)uga.edu (mailto:mepcott@uga.edu) mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) http://michaelcotterell.com/

1 0

Galaxy load problem
by Auerbach, Kenneth R. 17 Jul '13

17 Jul '13

Hello, I'm running Galaxy on a Linux server and have been getting warning messages about heavy loads on the CPU due to Galaxy. This is a dedicated Galaxy server and is currently used very lightly. There seem to be six idle postgres processes running, but only two jobs running. Any idea why the load is so high? Thank you. Ken. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

2 1

Re: [galaxy-dev] Galaxy Local Admin user
by Hans-Rudolf Hotz 17 Jul '13

17 Jul '13

Please keep all replies on the list by using "reply all" ! for my reply see below... On 07/17/2013 06:58 AM, Chadi Abdul Kader El Farran wrote: > Dear Hans-Rudolf, > > Thank you for your reply. > > The problem was solved by restarting. > > Now my problem is that I am unable to upload files into my Galaxy. > > I have put the following code in Universe_wsgi.ini: > > "# Directories of files contained in the following directory can be uploaded to a library from the Admin view > library_import_dir = /upload" > > However when I upload files to a data library, all files will seem to have no information (0 bytes) > That is very cryptic....and you need to provide more details - did you get an error? - how do you know the files are empty - what happens if you work with the files make sure to follow the wiki page: http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries Hans-Rudolf > Can you kindly help me? > > Thanks a lot for the support. With warmest regards > > > Chadi EL Farran (Post Graduate Student) > Dr. Jonathan Yuin-Han LOH's Laboratory > #08-01, Institute of Molecular and Cell Biology (IMCB) > 61 Biopolis Drive, Proteos > Singapore 138673 > Contact No: +65-91944793 > E-mail: chadi(a)nus.edu.sg > ________________________________________ > From: Hans-Rudolf Hotz [hrh(a)fmi.ch] > Sent: Tuesday, July 16, 2013 9:39 PM > To: Chadi Abdul Kader El Farran > Cc: galaxy-dev(a)bx.psu.edu > Subject: Re: [galaxy-dev] Galaxy Local Admin user > > Hi > > Just double checking: Have you restarted Galaxy? > > > Regards, Hans-Rudolf > > > On 07/16/2013 07:23 AM, Chadi Abdul Kader El Farran wrote: >> Dear Madam/Sir, >> >> I have installed Galaxy Local in our system. I registered as a user. However when I try to give admin status to some users, I am unable despite following the steps mentioned in the help. >> >> >> "To give a user Galaxy admin privileges, add their Galaxy login ( email ) to the list in the following config setting in the Galaxy configuration file universe_wsgi.ini. >> >> # this should be a comma-separated list of valid Galaxy users >> admin_users = user1@bx.psu.edu,user2@bx.psu.edu" >> >> I have done what is mentioned above and the universe_wsgi.ini is edited accordingly. Still when I log on to Galaxy Local I don't see the admin tab. >> >> Can you kindly help me. >> >> With warmest regards >> >> >> Chadi EL Farran (Post Graduate Student) >> Dr. Jonathan Yuin-Han Loh's Laboratory >> #08-01, Institute of Molecular and Cell Biology (IMCB) >> 61 Biopolis Drive, Proteos >> Singapore 138673 >> Contact No: +65-91944793 >> E-mail: chadi(a)nus.edu.sg >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> >

2 2

Postgres renaming to postgresql
by Nikos Sidiropoulos 17 Jul '13

17 Jul '13

Hi all After updating to the last stable galaxy when I'm trying to upgrade the database I'm getting the following message: /steno-internal/projects/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/url.py:105: SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to 'postgresql'. The new URL format is postgresql[+driver]://<user>:<pass>@<host>/<dbname> If I change the line in universe_wsgi.ini from database_connection = postgresql://galaxy:password@localhost:5432/galaxydb to

1 1

repeat tag enhancement?
by Berner, Thomas 17 Jul '13

17 Jul '13

Hey guys, i was searching the mailing list and wiki for some hints if there is a possibility to enhance the repeat tag. We want to give the option to our users to add more than one additional set of the contained parameters at one go e.g. 10 or 20 or 50, because it can be very frustrating to click the "add new dataset" button 50 times. Is there any option to do this with a text field or preset like "add 20 new datasets" or something like that? I would be very thankful for an advice. Greetings, Thomas

4 4

Galaxy-less Tool Installing
by John Chilton 16 Jul '13

16 Jul '13

One of my goals for the GCC was to sell the idea that tool shed repositories need to be installable without a database present. I talked with James Taylor and Enis Afgan about this idea briefly and they seemed to believe this was a good idea - I kept meaning to discuss it with Greg but I never got a good opportunity. Though in past Greg has made this sound potentially doable and has never objected to the goal overtly. I have two specific use cases in mind (CloudBioLinux and LWR), but perhaps the higher-level justification is something along the lines that a lot of effort from Greg and others (Dave, Bjorn, Peter, Nate) has gone into building a modular dependency system that could very easily be leveraged by applications other than Galaxy, so the extra steps that could be taken to make this possible should to make the codebase as broadly useful and to encourage adoption. The Galaxy community could benefit from other applications potentially utilizing and populating the tool shed and Galaxy tool developers would be further incentized to write good, modular dependencies and publish them to the tool shed. A high-level task decomposition would be something like this: 1. Rework installing tool shed repositories to not require a database. A kind of messy way to do this might be adding a use_database flag throughout. A cleaner way might be to use allow the core functionality to work with callbacks or plugins that performed the database interactions. 2. Separate the core functionality out of the Galaxy code base entirely into a reusable, stand-alone library. I would love buy in from the Galaxy team on item 2 above, but it is not strictly needed for my goals - I imagine I could write a script to pull it out Galaxy and build the library automatically or even just have the Galaxy codebase present when using Galaxy-less tool shed dependencies. Buy in on item 1 by the Galaxy team (specifically Greg and Dave B.) however is needed, are there any objections to this idea? Do you have any broad advice on how to approach this to ensure the changes make sense, work with your long term vision, and end up in Galaxy? Of all the things on my TODO list for the next year, this is probably the most potentially broadly interesting to this weeks BOSC codefest attendees, so I was going to attempt to sell this as something to work on. The sales pitch would include building a little tool shed version of the module command - http://linux.die.net/man/1/module to demonstrate this work and have something immediately useful produced. The idea would be to create a command-line tool for utilizing tool shed dependencies. # Unlike standard module, install procedure is available. Probably could # default to main tool shed and latest installable revision % tsmodule repo:install galaxyp/tint % tsmodule repo:install toolshed.g2.bx.psu.edu/galaxyp/tint/ab43b5ba7a4e # module lets you list packages, I guess tool shed version would need # repository and package listings: % tsmodule repo:list toolshed.g2.bx.psu.edu/galaxyp/tint/ab43b5ba7a4e % tsmodule package:list tint_proteomics_scripts/1.19.19/galaxyp/tint/ab43b5ba7a4e # Finally, a use command would source the env.sh script and make dependency # available in the command-line (might require starting new shell?): % tsmodule package:use tint_proteomics_scripts % tsmodule package:use tint_proteomics_scripts/1.19.19 % tsmodule package:use int_proteomics_scripts/1.19.19/galaxyp/tint/ab43b5ba7a4e # use apps that would be available to tools with valid requirements tags. % iQuantCLI This would be different from using the API scripts because there would be no API, Galaxy instance, or Galaxy database involved - just the Galaxy code. If this was able to split into its own Python library, one could imagine even allowing something like tsmodule to be installable right from pip and recursively fetch a toolshed_client library or something like that.

5 5