February 2014 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Database and FTP setup
by Conrad, Turner Allen 20 Feb '14

20 Feb '14

Thanks, Nate! Got it to run after playing with the postgresql url, table was created, and I began populating it with my admin account. I also prepared the database for ftp by creating a user for lookup. As for proftpd, I have it installed, but am not quite sure what do now. How must I edit the postgresql and proftpd configuration files before changing the galaxy ini? ---------------------------------------------------- 1LT Turner A Conrad, USAR IRR Ph.D. Candidate Graduate School of Biomedical Sciences MED 4.017V Dept. of Microbiology and Immunology 7703 Floyd Curl Drive San Antonio, TX 78229 Email: conradt(a)livemail.uthscsa.edu Office: (210) 567-3947 Cell: (210) 954-0893 On Feb 20, 2014 10:23 PM, Nate Coraor <nate(a)bx.psu.edu> wrote: > > Hi Turner, > > You should be connecting to the database that Galaxy is using, specified in the database_connection option in universe_wsgi.ini. If you have not yet started Galaxy at least once, then the tables will not have been created yet. Once you run Galaxy and the tables have been created, you should be able to connect to that database (galaxydb, or whatever you have set in database_connection) with psql and the GRANT should succeed. > > --nate > > > On Thu, Feb 20, 2014 at 9:49 AM, Conrad, Turner Allen <ConradT(a)livemail.uthscsa.edu> wrote: >> >> Hi, >> >> >> I'm not experienced with postgres and ftp and am thus having difficulties setting up an ftp server on my local Galaxy instance, but it is essential as our files are all over 2GB. The admin support page (https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP) is a little fuzzy on the details. I'm currently running the server out of Ubuntu 13.10 desktop both physically and via SSH. So far, I've created a local account postgres with a psql database galaxydb and user galaxyftp, but I can't figure out whether the galaxy_user table is supposed to exist, needs to be created, nor how to add this information to the database_connection option in universe_wsgi.ini. ALTER ROLE has been completed successfully, but I get the error ERROR: relation "galaxy_user" does not exist upon attempting GRANT SELECT ON. Is there a simple explanation of how to complete this task and get the proftpd server up and running for Ubuntu? Please assume proftpd is installed, but not configured, the database is in the state described above, and I am starting with sudo su - postgres followed by psql galaxydb . >> >> >> >> Turner Conrad >> UTHSCSA >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > >

2 1

Database and FTP setup
by Conrad, Turner Allen 20 Feb '14

20 Feb '14

Hi, I'm not experienced with postgres and ftp and am thus having difficulties setting up an ftp server on my local Galaxy instance, but it is essential as our files are all over 2GB. The admin support page (https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP) is a little fuzzy on the details. I'm currently running the server out of Ubuntu 13.10 desktop both physically and via SSH. So far, I've created a local account postgres with a psql database galaxydb and user galaxyftp, but I can't figure out whether the galaxy_user table is supposed to exist, needs to be created, nor how to add this information to the database_connection option in universe_wsgi.ini. ALTER ROLE has been completed successfully, but I get the error ERROR: relation "galaxy_user" does not exist upon attempting GRANT SELECT ON. Is there a simple explanation of how to complete this task and get the proftpd server up and running for Ubuntu? Please assume proftpd is installed, but not configured, the database is in the state described above, and I am starting with sudo su - postgres followed by psql galaxydb . Turner Conrad UTHSCSA

2 1

package_r_xx problem: X11 is not available
by Geert Vandeweyer 20 Feb '14

20 Feb '14

Hi all, I installed to package_r_2.11 and r_3x from the devteam to use as tool dependency. All installation goes fine, but when I run my tool I get the following error: Error in png(file = "../Plots/outname.png", bg = "white", width = 480, : X11 is not available The tool uses the installed R : /galaxy/galaxy_tool_binaries/R/2.11.0/devteam/package_r_2_11_0/8d0a55bf7aaf/lib/R/bin/R If I force the tool to use the system installed version of R, it works fine. Are there packages available in the toolsheds that allow the 'png' function to work? I already tried the R-packages from boris, but they gave compilation errors... I also tried to install the R-png package from cran as a galaxy dependency, but that didn't help either. Thanks for the help, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

3 7

Error in SICER tool
by Nikos Sidiropoulos 20 Feb '14

20 Feb '14

Hi all, I'm getting the following error whenever I'm trying to run SICER Traceback (most recent call last): File "/steno-internal/projects/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/sicer/82a8234e03f2/sicer/sicer_wrapper.py", line 158, in <module> if __name__=="__main__": __main__() File "/steno-internal/projects/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/sicer/82a8234e03f2/sicer/sicer_wrapper.py", line 154, in __main__ raise e IOError: [Errno 2] No such file or directory: '/NextGenSeqData/galaxy-data/database/tmp/tmpHPGhcN/input_bed_file-W200-G600-E0.01.scoreisland' Do you know if there's a workaround for this error? I have installed SICER from the toolshed (devteam) Bests, Nikos

2 1

Database and ftp setup
by Conrad, Turner Allen 20 Feb '14

20 Feb '14

Hi, I'm not experienced with postgres and ftp and am thus having difficulties setting up an ftp server on my local Galaxy instance, but it is essential as our files are all over 2GB. The admin support page (https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP) is a little fuzzy on the details. I'm currently running the server out of Ubuntu 13.10 desktop both physically and via SSH. So far, I've created a local account "postgres" with a psql database "galaxydb" and user "galaxyftp", but I can't figure out whether the "galaxy_user" table is supposed to exist, needs to be created, nor how to add this information to the "database_connection" option in universe_wsgi.ini.ALTER ROLE has been completed successfully, but I get the error "ERROR: relation "galaxy_user" does not exist" upon attempting GRANT SELECT ON. Is there a simple explanation of how to complete this task and get the proftpd server up and running for Ubuntu? Please assume proftpd is installed, but not configured, the database is in the state described above, and I am starting with "sudo su - postgres" followed by "psql galaxydb." Turner Conrad UTHSCSA

1 0

Fix for need_late_validation error when running workflow with XY_Plot_1 tool
by Jim Johnson 20 Feb '14

20 Feb '14

Attempting to run a workflow with the tool XY_Plot_1 results in an error when trying to display the workflow steps. The local dep_value variable is undefined. The XY_Plot_1 tool has contains a repeat with a DataToolParameter "input" and 2 ColumnListParameter "xcol", "ycol" dependent on the input. In the current lib/galaxy/tools/parameters/basic.py code, the context values are expected to be of type dict, but in the XY_Plot_1 example the value we need to traverse is a list from the repeat. Although the preferred solution would be to scope parameters within conditional and repeat tags, the following will traverse the context in search of any match to the param name + dependency name combination. $ hg diff lib/galaxy/tools/parameters/basic.py diff -r 3789ce7ddf1a lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py Thu Feb 20 10:52:31 2014 -0500 +++ b/lib/galaxy/tools/parameters/basic.py Thu Feb 20 10:05:32 2014 -0600 @@ -886,15 +886,30 @@ # If we got this far, we can actually look at the dependencies # to see if their values will not be available until runtime. for dep_name in self.get_dependencies(): - if dep_name in context: - dep_value = context[ dep_name ] - else: - # Quick hack to check deeper in the context. - # TODO: Context should really be scoped and the correct subset passed along. - # This happens specifically in all the GATK tools, the way the reference genome is handled. - for layer in context.itervalues(): - if isinstance( layer, dict ) and self.name in layer and dep_name in layer: - dep_value = layer[dep_name] + # Quick hack to check deeper in the context. + # TODO: Context should really be scoped and the correct subset passed along. + # This happens specifically in all the GATK tools, the way the reference genome is handled. + # Also needed ito run workflows using the XY_Plot_1 tool + def get_dep_value(param_name, dep_name, layer): + dep_val = None + if isinstance( layer, dict ) and param_name in layer and dep_name in layer: + dep_val = layer[dep_name] + elif isinstance( layer, list): + for l in layer: + dep_val = get_dep_value(param_name, dep_name, l) + if dep_val: + break + elif isinstance( layer, dict): + for l in layer.itervalues(): + dep_val = get_dep_value(param_name, dep_name, l) + if dep_val: + break + return dep_val + dep_value = get_dep_value(self.name, dep_name, context) + # If dep_name not found, assume + if not dep_value: + log.warn("Unable to find %s dependency value for param: %s", dep_name, self.name) + return True # Dependency on a dataset that does not yet exist if isinstance( dep_value, DummyDataset ): return True -- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota

1 0

Running out of memory on data download
by Michael Römer 20 Feb '14

20 Feb '14

Dear all, We run our Galaxy server in a virtual machine with less than 1GB RAM. When I'm trying to download results via the Download button in the corresponding history entry that are larger than 1GB, the results file is loaded into memory. This leads to the galaxy process being killed, due to memory issues. Is there a way to omit loading the result into memory? Regards, Michael -- Michael Römer, MSc. Bioinf. Abteilung Kognitive Systeme Fachbereich Informatik Wilhelm-Schickard-Institut Sand 1, Raum A312 72076 Tübingen Tel.: +497071/2978970 E-Mail: michael.roemer(a)uni-tuebingen.de

2 2

Re: [galaxy-dev] Running out of memory on data download
by Dannon Baker 20 Feb '14

20 Feb '14

My first guess is that it's likely this is due to 'debug = True' being set in your universe_wsgi.ini. If you set this to false and restart, the entire request will no longer be loaded into memory. -Dannon

1 0

select with a value preselected
by Ketan Maheshwari 19 Feb '14

19 Feb '14

Hello, In my tool UI, I have a select tool as follows: <param name="site" type="select" multiple="true" label="Execution Location" help="Multi-select list - hold the appropriate key while clicking to select multiple items"> <option value="localhost">Localhost</option> <option value="midway">Midway</option> <option value="uc3">UC3</option> <option value="stampede">Stampede</option> <option value="tukey">Tukey</option> </param> How do I tell it to preselect value "localhost" by default? Currently, user has to explicitly select the a value but if she forgets to do so the tool breaks because nothing is selected by default. Thanks, -- Ketan

2 2

Re: [galaxy-dev] Tools with multiple input files, and multiple output files
by Jim Johnson 19 Feb '14

19 Feb '14

I suggested augmenting the tool_conf syntax as part of the DataCollection development. To replace the need for the the multiple output determined at runtime, I suggest being able to declare data collections within the outputs tags, and being able to use regular expressions in the from_work_dir param to populate the collections. In workflows, one would want to be able to hook a data collection output to a data input. Mothur Metagenomics tool that has an output per distance label and calculator method An example of declaring a list of outputs, which will determined at run time based on from_work_dir regular expression: <tool id="mothur_classify_otu" name="Classify.otu" version="1.20.0" force_history_refresh="True"> ... <outputs> <dataset_collection type="list" label="${tool.name} on ${on_string} consensus taxonomies"> <data format="cons.taxonomy" name="splicing_diff" label="${tool.name} on ${on_string}: ${file_name}" from_work_dir="^\S+?\.(unique|[0-9.]*\.cons\.taxonomy)$" /> </dataset_collection> <dataset_collection type="list" label="{tool.name} on ${on_string} taxomy summaries"> <data format="cons.taxonomy" name="splicing_diff" label="${tool.name} on ${on_string}: ${file_name}" from_work_dir="^\S+?\.(unique|[0-9.]*\.cons\.tax\.summary)$" /> </dataset_collection> </outputs> > Hey Peter, > > Have you seen this solution? > > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number… > > It always seems to get mentioned when this topic is brought up. It has > serious limitations in terms or workflow running, but it can probably > be made to work for individual tool executions. > > Otherwise I would wait for future feature sets, maybe other people > have some good ideas however. > > -John > > On Wed, Feb 19, 2014 at 7:16 AM, Peter Cock<p.j.a.cock(a)googlemail.com> wrote: >> Hi all, >> >> I'm looking for examples of tools which take multiple input >> files (one or more, determined at run time) and produce >> multiple output files (one for each input file). Any >> specific suggestions? >> >> I have a number of sequence filtering/renaming tools >> where this might be useful - in some cases taking >> multiple input files and producing a single output is >> fine, but in general I'd like to know how to preserve a >> one to one mapping from input files to output files. >> >> I realise this may overlap slightly with the work John is >> doing on dataset collections, but for now I'd like to target >> the current Galaxy feature set. >> >> In some of the simpler cases, if I have N input datasets >> and want N output files, I can just run the tool N times . >> This means more steps in the Galaxy GUI, but it isn't >> very complicated. >> >> However, for the current problem I need access to all >> the inputs at once for setting overall data derived >> parameters. >> >> Regards, >> >> Peter >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ -- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota

1 0