- galaxy-dev - lists.galaxyproject.org

Why are Custom Builds disabled when
by Peter van Heusden 29 Oct '13

29 Oct '13

In galaxy.masthead.mako, from line 194, the following code appears: if app.config.use_remote_user: if app.config.remote_user_logout_href: menu_options.append( [ _('Logout'), app.config.remote_user_logout_href, "_top" ] ) else: menu_options.append( [ _('Preferences'), h.url_for( controller='/user', action='index', cntrller='user' ), "galaxy_main" ] ) menu_options.append( [ 'Custom Builds', h.url_for( controller='/user', action='dbkeys' ), "galaxy_main" ] ) logout_url = h.url_for( controller='/user', action='logout' ) menu_options.append( [ 'Logout', logout_url, "_top" ] ) menu_options.append( None ) Why are Custom Builds only enabled if you're *not* using remote_user ? Is this because they're related to user preferences (as per the code in lib/galaxy/webapps/galaxy/controllers/user.py)? And then a followup, if I have to add a build using either a FASTA file or a lenfile and editing the Galaxy install on the server, where do I need to make changes? Thanks, Peter

2 1

Upload via URL returns empty output
by Nikos Sidiropoulos 28 Oct '13

28 Oct '13

Hi all, I have created a Data library and uploaded (linked) some fastq files. After applying the bar code splitter tool, when I click on the output links I can get the output on my browser. I also get them by copy pasting the address in a new tab. But when I try the upload via URL by copy pasting the address I get an empty output. I haven't setup an ftp server neither I'm using and apache proxy. Could this be the problem? Bests, Nikos

1 0

trackster: missing graphs for large number of contigs
by Berner, Thomas 28 Oct '13

28 Oct '13

Hey guys, we want to use trackster in our local galaxy instance (rev. 10392:1ae95b3aa98d release_2013.08.12) and followed the instructions at http://wiki.g2.bx.psu.edu/Learn/Visualization . We tested it with a custom Build of 96 sequences in a fasta file, where all bams were mapped against before, everything worked fine. But now we want to do the same for a much larger file (about 29.000 contigs) and no graph is displayed at all, no matter which contig is used. No error appears, only the graphs are missing. Any ideas what this could be? We would be grateful for any advice. Thanks Thomas --- Thomas Berner Julius Kühn-Institut (JKI) - Federal Research Centre for Cultivated Plants - Erwin Baur-Straße 27 06484 Quedlinburg - Germany - Phone: ++49 ( 0 ) 3946 47 562 EMail: thomas.berner(a)jki.bund.de

2 3

Galaxy and ProFTP
by Jurgens de Bruin 28 Oct '13

28 Oct '13

I am having problems getting ProFTP to work: This is the output of ProFTP: FTP session opened. manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'USER username' to mod_core manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'USER username' to mod_core manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'USER username' to mod_delay manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'USER username' to mod_auth manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching CMD command 'USER username' to mod_auth manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching POST_CMD command 'USER username' to mod_sql manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching POST_CMD command 'USER username' to mod_delay manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching LOG_CMD command 'USER username' to mod_sql manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching LOG_CMD command 'USER username' to mod_log manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_core manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_core manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_sql_passwd manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_sql manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_delay manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching PRE_CMD command 'PASS (hidden)' to mod_auth manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): dispatching CMD command 'PASS (hidden)' to mod_auth manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): no supplemental groups found for user 'username' manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): mod_sql_passwd/0.4: expected '982673e732e72d0646575025d72ee4c811a547f6', got 'c4233ae0617d48182fdca25e1488871a008ad783' manager proftpd[33551] manager (155.240.187.111[155.240.187.111]): USER username (Login failed): No such user found. This is proftpd.conf: This is a basic ProFTPD configuration file (rename it to # 'proftpd.conf' for actual use. It establishes a single server # and a single anonymous login. It assumes that you have a user/group # "nobody" and "ftp" for normal operation and anon. ServerName "Galaxy FTP" ServerType standalone DefaultServer on # Port 21 is the standard FTP port. Port 21 # Don't use IPv6 support by default. UseIPv6 off # Umask 022 is a good standard umask to prevent new dirs and files # from being group and world writable. Umask 022 # To prevent DoS attacks, set the maximum number of child processes # to 30. If you need to allow more than 30 concurrent connections # at once, simply increase this value. Note that this ONLY works # in standalone mode, in inetd mode you should use an inetd server # that allows you to limit maximum number of processes per service # (such as xinetd). MaxInstances 30 # Set the user and group under which the server will run. User galaxy Group galaxy # To cause every FTP user to be "jailed" (chrooted) into their home # directory, uncomment this line. DefaultRoot ~ # Automatically create home directory if it doesn't exist CreateHome on dirmode 700 # Normally, we want files to be overwriteable. AllowOverwrite on # Allow users to resume interrupted uploads AllowStoreRestart on # Bar use of SITE CHMOD by default <Limit SITE_CHMOD> DenyAll </Limit> # Bar use of RETR (download) since this is not a public file drop <Limit RETR> DenyAll </Limit> # Do not authenticate against real (system) users AuthPAM off # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxy_prod@localhost:5432 galaxyftp ******* SQLAuthTypes SHA1 SQLAuthenticate users # An empty directory in case chroot fails SQLDefaultHomedir /scratch/sysusers/galaxy/ftp #Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email,password,'600','601','/scratch/sysusers/galaxy/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" # Create log ExtendedLog /var/log/proftpd.log read,write Include /etc/proftpd/modules.conf AuthOrder mod_sql.c SQLDefaultGID 601 SQLDefaultUID 600 I have added use_pbkdf2 = false to my universe_wsgi.ini and the password have been reset. Any help would be appreciated!! -- Regards/Groete/Mit freundlichen Grüßen/recuerdos/meilleures salutations/ distinti saluti/siong/duì yú/привет Jurgens de Bruin

1 0

pass regular expression
by Jun Fan 27 Oct '13

27 Oct '13

Hi all, I am trying to develop a tool which allows the user to use regular expression to only keep the lines matching the given pattern in the output. There are many special characters, e.g. \ in \d. In the perl script the wrapper invokes, the regular expression is printed as Xd+ for the regular expression \d+. My question is how to pass these special characters including \, (, ), ^ etc. correctly into the perl script. Best regards! Jun

4 4

toolshed contact not working (varscan)
by Geert Vandeweyer 26 Oct '13

26 Oct '13

Hi, I tried to contact the owner of the varscan repository through the main toolshed ( http://toolshed.g2.bx.psu.edu/) "repository actions, contact owner function. Upon submission I got the following error : An error occurred sending your message by email: 'Configuration' object has no attribute 'smtp_ssl' Is this a known issue? The option is not available if I browse the repository from my local galaxy admin pages. In case fcaramia reads this, please contact me, I have some updates to your varscan somatics script, as the default scripts failed to run for me. Best, Geert -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

Toolshed reset all repository metadata
by Simon Guest 25 Oct '13

25 Oct '13

We've noticed a funny about resetting repository metadata from the toolshed web interface. My understanding is that this is used to get the toolshed back in sync with the underlying mercurial repo, for example after pushing updates from hg on the command line. It seems that only the person who created the repository can reset metadata. I granted authority to make changes to a colleague, which enables him to push updates, but he doesn't see the reset metadata entry in the menu. This seems like a bug to me. Or is it by design? Also, is there a reason why this function is necessary at all? Wouldn't it be better if the toolshed just noticed updates to the underlying mercurial repo, and reset its own metadata automatically? cheers, Simon

3 3

Defining $GALAXY_CPUS for use in tool wrappers
by Peter Cock 24 Oct '13

24 Oct '13

Hello all, Re: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html Something I raised during the GCC2013, and we talked about via Twitter as well was a Galaxy environment variable for use within Tool Wrappers setting the number of threads/CPUs to use. The idea is that you can configure a default value, and then override this per runner or per tool etc. James Taylor had suggested calling this environment variable $GALAXY_CPUS which seem fine to me (personally I'd say threads not CPUs but I don't really mind). e.g. <command>my_tool --threads "\$GALAXY_CPUS" --input "$input" --output "$output"</command> Everyone I spoke to about this seemed positive about the idea. This would/should be integrated into the various cluster back ends, for example for SGE/OGE the number of threads is already configurable via the DRMAA settings and available as the environment variable $NSLOTS for non-MPI jobs, so my guess is all Galaxy needs to do is something like this: $ hg diff diff -r ce0d758bb995 lib/galaxy/jobs/runners/drmaa.py --- a/lib/galaxy/jobs/runners/drmaa.py Tue Jul 30 12:30:30 2013 +0100 +++ b/lib/galaxy/jobs/runners/drmaa.py Tue Jul 30 16:10:40 2013 +0100 @@ -43,6 +43,7 @@ # - execute the command # - take the command's exit code ($?) and write it to a file. drm_template = """#!/bin/sh +export GALAXY_CPUS="$NSLOTS" GALAXY_LIB="%s" if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then Is there an open Trello card for this? Thanks, Peter

5 11

Re: [galaxy-dev] Tool Shed Tests failing with sniffer/ftype problem
by Peter Cock 24 Oct '13

24 Oct '13

Retitling thread to focus on a regression in the (Tool Shed) test framework to do with input file types and sniffers. On Thu, Aug 1, 2013 at 11:47 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > On Thu, Aug 1, 2013 at 11:43 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: >> On Mon, Jul 29, 2013 at 5:03 PM, Dave Bouvier <dave(a)bx.psu.edu> wrote: >>> Peter, >>> >>> The issue with seq_rename incorrectly failing functional tests has been >>> resolved in 10266:fe04978dadac, and the test results will be corrected the >>> next time the automated testing framework runs. I'm still looking into the >>> other issues you reported. >>> >>> --Dave B. >> >> Hi Dave, >> >> The seq_rename test results (apparently) from last night are still failing, >> http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename/e440681dd17c >> >> AssertionError: Attempting to set field 'new_column' to value '['2']' >> in form 'tool_form' threw exception: cannot find value/label "2" in >> list control control: <SelectControl(new_column=[1])> If the above >> control is a DataToolparameter whose data type class does not include >> a sniff() method, make sure to include a proper 'ftype' attribute to >> the tag for the control within the <test> tag set. > > Same issue here?, > > http://testtoolshed.g2.bx.psu.edu/view/peterjc/nlstradamus/2be36fa7565e > > AssertionError: Attempting to set field 'fasta_file' to value > '['empty.fasta']' in form 'tool_form' threw exception: cannot find > value/label "empty.fasta" in list control control: > <SelectControl(fasta_file=[])> If the above control is a > DataToolparameter whose data type class does not include a sniff() > method, make sure to include a proper 'ftype' attribute to the tag for > the control within the <test> tag set. And another case which looks the same, http://testtoolshed.g2.bx.psu.edu/view/peterjc/effectivet3/392279f2e120 AssertionError: Attempting to set field 'fasta_file' to value '['empty.fasta']' in form 'tool_form' threw exception: cannot find value/label "empty.fasta" in list control control: <SelectControl(fasta_file=[])> If the above control is a DataToolparameter whose data type class does not include a sniff() method, make sure to include a proper 'ftype' attribute to the tag for the control within the <test> tag set. Note that for the nlstradamus and effectivet3 tests this is using an empty FASTA file with no sequences in it, so the sniffer will not be able guess this is a FASTA file - which is why the tool text gives the ftype explicitly. I originally used a literally empty fasta file (0 bytes) but as I recall the (non-Tool Shed) framework at the time didn't like me doing that in the test framework, so this empty.fasta file just has a couple of blank lines in it. I am testing with empty FASTA files deliberately - they can often occur in gene/protein selection pipelines where an early part of the pipeline is stringent enough to reject all the sequences. However, the seq_rename sniffer problem is not to do with an 'empty' fasta file - it uses a non-empty tabular file. My guess is somewhere in the test framework the ftype attribute is being lost. This is still working locally, $ ./run_functional_tests.sh -id seq_rename $ ./run_functional_tests.sh -id nlstradamus $ ./run_functional_tests.sh -id effectiveT3 Regards, Peter

2 7

Re: [galaxy-dev] SeqPrep wrapper?
by Lionel Guy 23 Oct '13

23 Oct '13

Hi Brad, (cc-ing the list) I'm more interested in the merging part, and after a quick search on the toolshed, there are no tools to merge reads. Seems that there is a need for a read merger wrapper then... Lionel On 14 Oct 2013, at 13:46 , "Langhorst, Brad" <Langhorst(a)neb.com> wrote: > I would also like to see a seqprep wrapper… > if you need something sooner, cutdadapt is already wrapped and i'm pretty sure there is an existing tool to merge reads as well. > > Brad > -- > Brad Langhorst > Applications and Product Development Scientist > > On Oct 14, 2013, at 6:59 AM, Lionel Guy <guy.lionel(a)gmail.com> > wrote: > >> Hi, >> >> I'm interested in a wrapper for SeqPrep: >> https://github.com/jstjohn/SeqPrep >> Anyone preparing one/aware of someone preparing one? Otherwise I'll put one together. >> >> From its description: "SeqPrep is a program to merge paired end Illumina reads that are overlapping into a single longer read. It may also just be used for its adapter trimming feature without doing any paired end overlap." >> >> Cheers, >> >> Lionel >> >> Lionel Guy >> Uppsala University >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > -- Lionel Guy Fålhagsleden 13, SE-75324 Uppsala | email: guy.lionel(a)gmail.com | mobile: +46 (0)73 9760618 | phone: +46 (0)18 410 7398

1 1