May 2010 - galaxy-dev - lists.galaxyproject.org

stupid logging question
by Michael Pheasant 07 Sep '10

07 Sep '10

Using the run.sh script, and the default 'log_level = DEBUG' universe_wsgi.ini file setting gives a huge amount of output. Changing the ini to either log_level = INFO, WARNING or ERROR and i get the same output, including the INFO messages every second. I assumed that WARNING or ERROR would not give the INFO messages; is there some way to not see these? Cheers m -- Michael Pheasant Software Engineer Queensland Facility for Advanced Bioinformatics Level 6, QBP University of Queensland, QLD 4072 T: +61 (0)7 3346 2070 F: +61 (0)7 3346 2101 www.qfab.org

4 8

adding another UCSC mirror to display
by Davide Cittaro 05 Aug '10

05 Aug '10

Hi again, this may be a FAQ, sorry for that... How can I add an alternate UCSC mirror site (our local mirror) to galaxy so that everything that could be displayed at UCSC-main will have the UCSC-localmirror option? Besides... how can I remove the bx main mirror? I've tried to add a ${GALAXYROOT}/tool-data/shared/campus/campus_build_sites.txt but that's not enough... I've modified the ucsc/bam.xml file adding another dynamic_link section but that's not enough... any hint? Thanks again d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

2 1

Dynamically adding tools to Galaxy
by Sumedha Ganjoo 26 Jul '10

26 Jul '10

Hello, I am a student at the University Of Georgia. We had developed a tool for Galaxy 1.3 that enables accessing various web services as tools from Galaxy. We are currently working on a similar tool for the current version of Galaxy. This requires adding a tool dynamically without having to restart Galaxy. I was wondering if someone has already implemented that feature? We do have most of the code for this in place. Is there a way, once complete, this code can be added to the main Galaxy source code? Also I would really appreciate if someone could tell me where can I find the older versions ( v 1.3 ) of Galaxy? Thanks in advance. Regards, Sumedha Sumedha Ganjoo Graduate Assistant Computer Science Department University Of Georgia

4 7

Incorporating Velvet and Interproscan into Galaxy
by Paszkiewicz, Konrad 08 Jun '10

08 Jun '10

I'd like to incorporate some short-read genome assembly programs into Galaxy. The Velvet assembler is a particular favourite of our facility and ideally I would like to give our users the option of running Velvet assemblies on FASTQ datasets. The ideal scenario would be for the contigs generated by velvet to then be piped into an interface to interproscan. However, in the first instance, I'd like to focus on the velvet interface. My main question centres around whether the format velvet uses to store information is representable as a composite datatype in Galaxy. Velvet comprises of two elements - a prep step (velveth) followed by the actual assembly (velvetg). Velveth generates a directory with files (the files always have the same names). Velvetg is then run with the directory containing the relevant files as a parameter. I have read through the composite datatypes wiki page but I'm not sure if this form of data storage can be represented in galaxy at the moment. Any advice would be much appreciated. All the very best, Konrad. Dr Konrad Paszkiewicz Exeter Sequencing Service, Biosciences, Stocker Road, University of Exeter, Exeter EX4 4QD, UK. http://biosciences.exeter.ac.uk/facilities/sequencing/

4 5

display_application
by Davide Cittaro 02 Jun '10

02 Jun '10

Hi all, I've noticed that when I try to visualize a BAM file on UCSC (with display_application) the galaxy web process drains RAM and never releases it... I easly go in OutOfMemory error. Is it trying to load BAM (or any other custom file defined to be visualized in the same way) in RAM? Can anybody explain how display_application works and how to debug it? d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

4 9

Galaxy with non-SGE/non-Torque distributed jobs
by Andrey Tovchigrechko 01 Jun '10

01 Jun '10

We have decided to use a local Galaxy install as a front-end to our metagenomic binning tool MGTAXA ( http://andreyto.github.com/mgtaxa/ ) I need some guidance from the Galaxy developers for the best way to proceed: 1) The server will be on a DMZ, with no direct access to the internal network, where the computes will be running on a local SGE cluster. The best that our IT allowed is for some script on the internal cluster to monitor a directory on the web server, pull input/tasks from where when they appear, put the results back. My current idea is to have the Galaxy "local runner" to start "proxy jobs": each proxy job is a local process that does "put the input into watched dir; until results appear in the watched dir; sleep(30); loop; finish". In other words, Galaxy thinks that it is running jobs locally, but it fact those jobs are just waiting for the remote results to come back. Does that look like a sane solution? How will it scale on the Galaxy side? E.g. how many such simultaneous tasks can the local runner support? Any anticipated gotchas? Additionally, we will be also trying to run computes on our TeraGrid account. I was thinking that the solution above can be applied to that scenario also, except that now the proxy job would be polling qsub on TeraGrid through ssh, or call Globus API. Here one problem is that a job often has to wait in a TeraGrid queue for 24 hours or so. Will my proxy jobs on Galaxy time out/get killed by any chance? The alternatives are 1) write another runner (in addition to local, sge, torque) - how much work it will be? 2) write a fake SGE python interface and make Galaxy think it is using local SGE 2) What repo is best to clone, given the scope of our activity described above? We will likely need to mess a bit with the Galaxy internals, not just the tool definition. Should we clone galaxy-central or galaxy-dist? What workflow would you recommend for updating, submitting patches etc? I will be very grateful for answers to the above, and also to any alternative recommendations. Andrey

4 3

how to activate account to allow remote access?
by Edward Kirton 01 Jun '10

01 Jun '10

Hello, i recently switched to ldap authentication such that user accounts are automatically created. Everything works as it should on the intranet site, but when i try to log-on via our internet site, i get the following error: "This account was created for use with an external authentication method, contact your local Galaxy administrator to activate it." Even if i create a password for the user, this message remains. How do i activate the account? Thanks, Ed

2 1

Integrating Galaxy into our Kerberos auth
by Peter van Heusden 01 Jun '10

01 Jun '10

Hi everyone I'm busy integrating Galaxy (installed locally on galaxy.sanbi.ac.za) with our Kerberos authentication. To do this, in the Apache virtualhost configuration, I install mdo_auth_kerb and specify: <Location /> AuthType Kerberos Krb5Keytab /etc/krb5.keytab KrbServiceName http KrbAuthRealms SANBI Require valid-user </Location> in my virtualhost config. Now, our Kerberos users show up as <user>@SANBI (SANBI is our realm). Their email addresses are of the form <user>@sanbi.ac.za. So this causes a problem with the default Galaxy config. The solution I've got is to throw away that @SANBI, then set remote_user_maildomain to sanbi.ac.za. And add this to the virtualhost config: RewriteEngine on <Proxy http://localhost:8080> Order deny,allow Allow from all </Proxy> RewriteCond %{IS_SUBREQ} ^false$ RewriteCond %{LA-U:REMOTE_USER} ^([^@]+)@ RewriteRule . - [E=RU:%1] RequestHeader set REMOTE_USER %{RU}e Note the ^([^@]+)@ - this is different to the (.*) which is on the wiki, and has the effect that %1 in the RewriteRule only contains the <user> part. The remote_user_maildomain is then used to get <user>@sanbi.ac.za. So far this setup seems to work. BTW. why, in templates/webapps/galaxy/base_panels.mako does it only show the "Logged in as" if you are not using remote_user ? Since user_email is always generated somehow, surely it is safe to show it whether you're logged in via remote_user or not. And finally, I see that after the first few uploads, there aren't any new sets of slides being put up on the dev2010 page. I'm looking forward to seeing a full set of slides so I can show them to people at SANBI! Peter

2 1

Running multiple files automatically within workflows
by Juan Carlos Perin 31 May '10

31 May '10

We have a problem that is probably not unique and may have a solution already. We have a script that looks at a directory and identifies all pileup files in that directory and performs a process on each of those files. We would like to integrate this script into galaxy but we're not sure how to handle the input of a directory (rather than a single file) into a workflow? One way (I am guessing) is if the generation of a data library could be automated. This then could somehow automatically import into an analysis workflow and run on every sample imported. I suppose I could manually have a directory of files imported and then run each workflow on the file, one by one, but in the case of having 400 or more samples, this just isn't practical. Is there any way to have a single application simply use a directory as input, instead of a single file, and then have a workflow execute on each of these files automatically? Aside from doing this within our script, it'd be nice to be able to do this for NGS alignments as well, in the case that we have an input set of multiplexed data, where the workflow needs to execute on (potentially) a few hundred samples repeatedly? I'm sure we could probably write a perl script wrapper that does this in the background somehow, but would prefer not to have to hack galaxy into performing this type of function. Thanks in advance. -Juan Perin

2 1

Input type 'data' sorting - sometimes text, sometimes numeric
by Dennis Gascoigne 31 May '10

31 May '10

I have tried to figure out what makes some data input select boxes sort the files numerically (like the Merge Columns tool) and some on text (like the add columns tool) but to me the configs seem identical. The correct behaviour is to sort numerically. Can you please include in your next update a fix to ensure that all input data selects are sorted numerically? Cheers Dennis

3 4