July 2013 - galaxy-dev - lists.galaxyproject.org

FTP Upload configuration
by Federico Zambelli 20 Jul '13

20 Jul '13

Dear all, I'm installing a local version of Galaxy but I'm having issues configuring the FTP upload functionality that I need. I assume the documentation in the wiki about configuring proftpd is outdated since Galaxy has recently changed authentication method. I apologize in advance for the length of this e-mail, thanks to everyone who will bother to reply. My Galaxy set up uses Postgresql as db and nginx and it seems working fine. I installed the latest available version of Proftpd (1.3.5rc3) and compiled it with all the needed (I think) modules. Here they are: mod_core.c mod_xfer.c mod_rlimit.c mod_auth_unix.c mod_auth.c mod_ls.c mod_log.c mod_site.c mod_delay.c mod_facts.c mod_sql.c mod_sql_postgres.c mod_sql_passwd.c mod_auth_pam.c mod_cap.c *********************** My proftpd.conf looks like this: # This is a basic ProFTPD configuration file (rename it to # 'proftpd.conf' for actual use. It establishes a single server # and a single anonymous login. It assumes that you have a user/group # "nobody" and "ftp" for normal operation and anon. ServerName "ProFTPD 4 Galaxy" ServerType standalone DefaultServer on AuthPAM off SyslogFacility DAEMON SyslogLevel debug # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex AuthOrder mod_sql.c RequireValidShell off # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxy_prod@localhost:5432 galaxy password #SQLAuthTypes SHA1 SQLAuthTypes SHA1 SHA256 pbkdf2 SQLPasswordPBKDF2 SHA256 1000 24 SQLAuthenticate users SQLDefaultHomedir /var/opt/local/proftpd SQLUserInfo custom:/LookupGalaxyUser SQLPasswordUserSalt sql:/GetUserSalt #SQLNamedQuery LookupGalaxyUser SELECT "email,password,'497','495','/home/galaxy/galaxy_dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN substring(password from 1 for 6) = 'PBDKF2' THEN substring(password from 38 for 69) ELSE password END) AS password2,'galaxy','galaxy','/home/galaxy/galaxy-dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 for 6) = 'PBDKF2' THEN SUBSTRING (password from 21 for 36) END) AS salt FROM galaxy_user WHERE email='%U'" SQLLogFile /var/log/proftpd.log SQLDefaultGID 1002 SQLDefaultUID 1002 # Port 21 is the standard FTP port. Port 21 PassivePorts 30000 40000 # Don't use IPv6 support by default. UseIPv6 off # Umask 022 is a good standard umask to prevent new dirs and files # from being group and world writable. Umask 077 # To prevent DoS attacks, set the maximum number of child processes # to 30. If you need to allow more than 30 concurrent connections # at once, simply increase this value. Note that this ONLY works # in standalone mode, in inetd mode you should use an inetd server # that allows you to limit maximum number of processes per service # (such as xinetd). MaxInstances 30 # Set the user and group under which the server will run. User galaxy Group galaxy # To cause every FTP user to be "jailed" (chrooted) into their home # directory, uncomment this line. DefaultRoot ~ CreateHome on dirmode 700 # Normally, we want files to be overwriteable. AllowOverwrite on AllowStoreRestart on # Bar use of SITE CHMOD by default <Limit SITE_CHMOD> DenyAll </Limit> # Bar use of RETR (download) since this is not a public file drop <Limit RETR> DenyAll </Limit> ************************************************* I have found the SQLNamedQuery directives by googling around and I have figured out they should work both with old and new password encryption methods used by Galaxy. My installation however is fresh and thus I should have only password encrypted with the new method. When I try to ftp the server this is what shows up in the log: 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: >>> sql_sess_init 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: entering postgres cmd_defineconnection 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: name: 'default' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: user: 'galaxy' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: host: 'localhost' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: db: 'galaxy_prod' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: port: '5432' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: ttl: '0' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: exiting postgres cmd_defineconnection 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: connection 'default' successfully established 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: mod_sql engine : on 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: negative_cache : off 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: authenticate : users 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: usertable : users 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: userid field : userid 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: password field : passwd 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: UID field : uid 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: GID field : gid 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: homedir field : homedir 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: homedir(default) : '/var/opt/local/proftpd' 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: shell field : shell 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: SQLMinUserUID : 999 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: SQLMinUserGID : 999 2013-07-20 16:35:54,618 mod_sql/4.3[4127]: <<< sql_sess_init 2013-07-20 16:36:01,533 mod_sql/4.3[4127]: >>> sql_escapestr 2013-07-20 16:36:01,533 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,533 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: Postgres server version: 9.2.4 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' opened 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: <<< sql_escapestr 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: >>> sql_lookup 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: >>> process_named_query 'GetUserSalt' 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_select 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,537 mod_sql/4.3[4127]: query "SELECT (CASE WHEN SUBSTRING (password from 1 for 6) = 'PBDKF2' THEN SUBSTRING (password from 21 for 36) END) AS salt FROM galaxy_user WHERE email='federico.zambelli(a)gmail.com'" 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_select 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: <<< process_named_query 'GetUserSalt' 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: <<< sql_lookup 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: >>> sql_pre_pass 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: <<< sql_pre_pass 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: >>> cmd_getpwnam 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: cache miss for user 'federico.zambelli(a)gmail.com' 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: >>> sql_lookup 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: >>> process_named_query 'LookupGalaxyUser' 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_select 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,540 mod_sql/4.3[4127]: query "SELECT email, (CASE WHEN substring(password from 1 for 6) = 'PBDKF2' THEN substring(password from 38 for 69) ELSE password END) AS password2,'galaxy','galaxy','/home/galaxy/galaxy-dist/database/ftp/federico.zambelli(a)gmail.com','/bin/bash' FROM galaxy_user WHERE email='federico.zambelli(a)gmail.com'" 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: exiting postgres cmd_select 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: <<< process_named_query 'LookupGalaxyUser' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: <<< sql_lookup 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: custom SQLUserInfo query 'LookupGalaxyUser' returned 6 columns for user 'federico.zambelli(a)gmail.com' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: user UID 0 below SQLMinUserUID 999, using SQLDefaultUID 1002 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: user GID 0 below SQLMinUserGID 999, using SQLDefaultGID 1002 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: cache miss for user 'federico.zambelli(a)gmail.com' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: user 'federico.zambelli(a)gmail.com' cached 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: + pwd.pw_name : federico.zambelli(a)gmail.com 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: + pwd.pw_uid : 1002 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: + pwd.pw_gid : 1002 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: + pwd.pw_dir : /home/galaxy/galaxy-dist/database/ftp/federico.zambelli(a)gmail.com 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: + pwd.pw_shell : /bin/bash 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: <<< cmd_getpwnam 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: >>> cmd_auth 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: entering postgres cmd_escapestring 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: entering postgres cmd_open 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: connection 'default' count is now 2 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: exiting postgres cmd_open 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: entering postgres cmd_close 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: connection 'default' count is now 1 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: exiting postgres cmd_close 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: exiting postgres cmd_escapestring 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: cache hit for user 'federico.zambelli(a)gmail.com' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: >>> cmd_check 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: checking password using SQLAuthType 'sha1' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: 'sha1' SQLAuthType handler reports failure 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: checking password using SQLAuthType 'sha256' 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: 'sha256' SQLAuthType handler reports failure 2013-07-20 16:36:01,541 mod_sql/4.3[4127]: checking password using SQLAuthType 'pbkdf2' 2013-07-20 16:36:01,547 mod_sql/4.3[4127]: 'pbkdf2' SQLAuthType handler reports failure 2013-07-20 16:36:01,547 mod_sql/4.3[4127]: <<< cmd_check 2013-07-20 16:36:01,547 mod_sql/4.3[4127]: <<< cmd_auth 2013-07-20 16:36:02,861 mod_sql/4.3[4127]: entering postgres cmd_exit ********************************************************************* It looks like it is able to retrieve the user from the db but then it somewhat fails to check the password validity. Thanks again. Bye and have a nice w.e. -- ============================================ Federico Zambelli, Ph.D. Bioinformatics, Evolution and Comparative Genomics Lab Dept. of Biosciences University of Milano - Italy What can be asserted without proof can be dismissed without proof. ============================================

1 0

nglims, error pages and permissions
by Lee Katz 19 Jul '13

19 Jul '13

Hi, I am using nglims, and I am getting Internal Error pages sometimes. I am not sure if it is because of Galaxy itself or nglims though. The latest is when I create a new role and hit submit. It is repeatable if I fill in the name as "test", no description, I add in a single role (others are present but excluded from being associated), if I check "create a new group..." and then click "save." However, when I created a group and checked "create a new role ...," it did not create an error. Next, I tried associating data with this new user. However, her new roles (both the new role I created and her username-role) did not show up, and so I could not give the data to her. Is there any reason why a new user's roles would not show up? -- Lee Katz, Ph.D.

2 3

Functional Test Errors for Tool Shed Tools
by Nilesh Kavthekar 19 Jul '13

19 Jul '13

Hi everyone, I'm running into a bit of trouble with running functional tests on tool wrappers for rseqc that I am writing (testing according to the wiki page<http://wiki.galaxyproject.org/TestingInstalledTools>). The tools are listed in shed_tool_conf.xml. Though I'm working with ~15 tool wrappers, I'm attaching the error for read_quality.py, which is recurring for each of the other tools I'm writing. I'm attaching the link to the xml wrapper for read_quality<https://gist.github.com/nkav/abae3a22c1e4c881c2d8>to show you how I write the tests. Here is the specific error message (this was the same message returned for the 12 other rseqc tools I had wrapped): ====================================================================== ERROR: test_tool_000000 ( functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/nilesh… ) Read Quality ( testtoolshed.g2.bx.psu.edu/repos/nilesh/rseqc_dev2/read_quality/1.0.0 ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nilesh/galaxy-dist/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/home/nilesh/galaxy-dist/test/functional/test_toolbox.py", line 43, in do_it shed_tool_id=shed_tool_id ) File "/home/nilesh/galaxy-dist/test/base/twilltestcase.py", line 204, in upload_file filename = self.get_filename( filename, shed_tool_id=shed_tool_id ) File "/home/nilesh/galaxy-dist/test/base/twilltestcase.py", line 179, in get_filename return os.path.abspath( os.path.join( file_dir, filename ) ) File "/usr/lib/python2.7/posixpath.py", line 77, in join elif path == '' or path.endswith('/'): AttributeError: 'NoneType' object has no attribute 'endswith' -------------------- >> begin captured stdout << --------------------- form 'login' contains the following controls ( note the values ) --------------------- >> end captured stdout << ---------------------- I'm using the latest version of Galaxy and the RSeQC package, and the tool dependencies for rseqc install correctly after downloading from the test tool shed. Am I forgetting to do something here? Thanks so much for your help.

2 2

Problem in uploading file
by Himanshu Gupta 19 Jul '13

19 Jul '13

Hello everyone, I had configured galaxy with MySQL database and for some time it was working fine. But suddenly it started showing following warning whenever I start galaxy. ".../galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs2.egg/sqlalchemy/sql/expression.py:1927: SAWarning: The IN-predicate on "request_type.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate. Consider alternative strategies for improved performance." And now I am not able to upload any file. Please help me to resolve it. Thanks

1 0

FastQC files are not being recognized by
by Ricardo Perez 19 Jul '13

19 Jul '13

Dear all, We are doing some processes in our server that use fastq files. We have obtained the data in .sra format and then convert them into fastq format using the sra tool kit. However, when we upload this files into galaxy, some tools does not recognize the file as fastq. When we apply FasQC:Read QC, under summary statistics it says that encoding is Sanger / Illumina 1.9. Is this an error of our part while installing galaxy in our server? Thank you for your time, --Ricardo Perez

2 1

Fwd: Dynamically Generated Tools
by Ross 18 Jul '13

18 Jul '13

AFAIK it's not yet possible, but a tool shed API to expose subversion hooks for repository creation and update is a logical development which is already on the radar AFAIK. For the bigger picture of complexity for the user, I think some key clarifications required include: which 'user' is experiencing complexity; how big a barrier it is to generating tools; how readily can we make it go away; how much will it help? Tool builders (even 'users' of the TF) might reasonably be expected to have a capacity for substantial complexity that the end user of the generated tools should not need to possess - after all, once a generated tool is installed in Galaxy, the Galaxy user can't tell the difference and sees no more complexity from the generated tool than a hand written one. On absent local tool sheds: it's trivial to run a toolshed on a dev laptop and not hard to run in production - but tool makers can always use the public test and main TS if they want to distribute new tools so I'm not sure how big a barrier a missing local toolshed really is. On Thu, Jul 18, 2013 at 12:08 PM, Michael E. Cotterell < mepcotterell(a)gmail.com> wrote: > I didn't mean to imply it was unclear. It definitely made sense. > > I completely agree with you. However, is there for a way (maybe via the > toolshed api) for my tool to push these tools to a TS repo? Also, doesn't > that add some more complexity for the user? Not everyone runs a local tool > shed. > > Sincerely, > Michael E. Cotterell > > Ph.D. Student in Computer Science, University of Georgia > Instructor of Record, Graduate RA & TA, University of Georgia > Faculty Liaison, CS Graduate Student Association, University of Georgia > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) > mepcott(a)uga.edu (mailto:mepcott@uga.edu) > mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) > http://michaelcotterell.com/ > > > On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote: > > > Sorry if my post was rambling and unclear. > > > > Here's the executive summary for what it's worth: > > IMHO, a Galaxy tool that generates tools should emit them as tool shed > compatible artefacts, ready for uploading to a TS repository for automated > installation to any target instance. > > > > > > > > > > > > On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell < > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com)> wrote: > > > Ross, > > > > > > Thanks for the comments. > > > > > > I've already made my main tool "tool_shed" compatible. What I'm > wondering about is what do I do with the tools that my tool creates? > > > > > > Sincerely, > > > Michael E. Cotterell > > > > > > Ph.D. Student in Computer Science, University of Georgia > > > Instructor of Record, Graduate RA & TA, University of Georgia > > > Faculty Liaison, CS Graduate Student Association, University of Georgia > > > mepcotterell(a)gmail.com (mailto:mepcotterell@gmail.com) (mailto: > mepcotterell(a)gmail.com) > > > mepcott(a)uga.edu (mailto:mepcott@uga.edu) (mailto:mepcott@uga.edu) > > > mec(a)cs.uga.edu (mailto:mec@cs.uga.edu) (mailto:mec@cs.uga.edu) > > > http://michaelcotterell.com/ > > > > > > > > > On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: > > > > > > > Hi, Michael, > > > > > > > > As others have said, in the long term, tool wrappers and their > dependencies will be distributed and installed through tool sheds rather > than being distributed with Galaxy source, so you might want to plan to > generate and upload tool shed compatible archives from the get-go. The test > and main tool sheds contain plenty of examples and Björn's tools exercise > pretty much all of the functionality - you can browse the repo structure > which is identical to the gz archive structure you need to upload to a new > repo, or download the repo as a gz, unpack it and get exactly the kind of > directory structure and contents you need to emulate for your tools. Once > you have a working tool packaging the archive up is straightforward. > > > > > > > > The format for tool shed repo uploads and syntax for the tags used > to define dependencies is very well documented in the tool shed section of > the wiki. As Björn points out, the tool factory python wrapper might be a > useful source of ideas and perhaps code. Your tool generator will need to > do something similar to write the content and generate complete tool shed > archives. When generating a new tool, the TF uses an ugly XML wrapper > generator (contributed improvements would be very welcome!) and (probably > more usefully) the few lines of code you need to package up the functional > test data and the XML and wrapper if you need one in toolshed archive > format. > > > > > > > > > > > > On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning < > bjoern.gruening(a)pharmazie.uni-freiburg.de (mailto: > bjoern.gruening(a)pharmazie.uni-freiburg.de) (mailto: > bjoern.gruening(a)pharmazie.uni-freiburg.de)> wrote: > > > > > Hi Michael, > > > > > > > > > > I think you will enter new ground with your tool. The closest tool > that > > > > > will do something similar is Ross toolfactory, I think: > > > > > > > > > > http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory > > > > > > > > > > For me one question is, do you really want to offer that service to > > > > > every user and is it save? > > > > > > > > > > > TL;DR I have a tool that generates other tools. Where should > those tools go, and how should I add them (without modifying Galaxy code)? > > > > > > > > > > > > I apologize, in advance, for the wall of text. I would like some > advice on what the best way to go about setting up a tool that generates > other tools... > > > > > > > > > > > > As some of you (especially those who saw my boss's talk at GCC) > already know, I'm working on a tool that lets galaxy users add web service > operations as tools to Galaxy. > > > > > > We have a general purpose client for invoking web services, and > we have a tool that let's a user enter a WSDL/WADL URL, select the > operations they want to add, and generates tool config XML files for Galaxy. > > > > > > > > > > > > Right now, I'm in the process of making our tool "tool shed" > compatible, and that's going along great > > > > > > (I especially like the tool dependency installation stuff), but > I'm trying to determine the best course of action on the following: > > > > > > > > > > > > 1. Where is the best place to put the tool config (XML) files > that are generated by my tool? My predecessors just placed them in a > subdirectory under our tool's directory. > > > > > > I'm ok with this, but does that mean that all the generated > files will get removed if they upgrade our tool via the tool shed (haven't > taken the time to see if > > > > > > Galaxy just does an "hg pull" or if it's more complicated than > that)? > > > > > > > > > > > > > > > > > > > > It is mainly a hg pull. But if you deinstall that repository and > > > > > reinstall it, all your files are lost. > > > > > > > > > > > Would it be better to have my tool create a local tool shed > repository > > > > > > (just for these generated tools), and if so, can the tool shed > API let my tool install this repository? Having not looked at > install_tool_shed_repositories.py yet, > > > > > > it's unclear from the wiki documentation whether or not POST > /api/tool_shed_repositories/install_repository_revision requires that the > tool shed already registered in tool_sheds_conf.xml)? > > > > > > > > > > > > > > > > > > > > I think that is a better way to do it. The API can be used to > install > > > > > your tools. But it need to be configured beforehand from the user, > > > > > administrator. > > > > > > > > > > > 2. What's the best way to add the generated tools to Galaxy? My > predecessors modified the tool_config.xml directly and required users to > restart Galaxy. > > > > > > Now, If these tools are in a tool shed repo, I know that I can > use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if > these tools are in a tool shed repo, however, > > > > > > if that's not the recommended course of action, then I'd like > some advice on what to do? > > > > > > > > > > > > > > > > > > > > There is a feature request, that you can specify in your > > > > > toolshed-repository the preferred location in the tool panel. For > now I > > > > > would ignore it and just install the tools somewhere and hope that > > > > > feature request is quiet high in Gregs ToDo list :) > > > > > > > > > > Cheers, > > > > > Bjoern > > > > > > > > > > -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=en&user=UCUuEM4AAAAJ

2 2

permissions
by Lee Katz 18 Jul '13

18 Jul '13

Hi, I buried this into another email but I would like to get some help on it and therefore have to make it visible. Newer users to my Galaxy instance are not showing up under permissions when I try to give them access to MiSeq data. Is there any kind of troubleshooting steps I can take to understand why? The new users and their new roles do not show up under "roles associated" when I try to edit permissions. However, they do show up when I try to edit folder permissions. Thank you! -- Lee Katz, Ph.D.

1 0

tool inheritance
by Lee Katz 18 Jul '13

18 Jul '13

One more question for the Galaxy gurus. I have a series of tools that 1) interleave raw reads, 2) assemble them, and 3) predict genes from the assembly. How can I display inheritance on the data produced from these tools? It would be really nice if a set of genes showed that they came from an assembly, which came from a set of raw reads. I can't find this on the site, and so I am sure I am using the wrong keywords or something... Thank you! -- Lee Katz, Ph.D.

2 2

Re: [galaxy-dev] FastQC files are not being recognized by
by Ambrose Andongabo (RRes-Roth) 18 Jul '13

18 Jul '13

Try the fastqsanger datatype if it works fine if not try the others such as fastqillumina etc Thanks Ambrose -----Original Message----- From: galaxy-dev-bounces(a)lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Ricardo Perez Sent: 18 July 2013 20:41 To: galaxy-dev(a)lists.bx.psu.edu Subject: [galaxy-dev] FastQC files are not being recognized by Dear all, We are doing some processes in our server that use fastq files. We have obtained the data in .sra format and then convert them into fastq format using the sra tool kit. However, when we upload this files into galaxy, some tools does not recognize the file as fastq. When we apply FasQC:Read QC, under summary statistics it says that encoding is Sanger / Illumina 1.9. Is this an error of our part while installing galaxy in our server? Thank you for your time, --Ricardo Perez ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- This message has been scanned for viruses and dangerous content by MailScanner, and we believe but do not warrant that this e-mail and any attachments thereto do not contain any viruses. However, you are fully responsible for performing any virus scanning. -- This message has been scanned for viruses and dangerous content by MailScanner, and we believe but do not warrant that this e-mail and any attachments thereto do not contain any viruses. However, you are fully responsible for performing any virus scanning.

1 0

Strange behaviour with galaxy history
by Groot, Philip de 18 Jul '13

18 Jul '13

Hi, First of all, I apologize for interrupting into this discussion, but I have exactly the same problem and did not find a solution yet. And no: prefix and cookies are set properly in our particular setting (as "/")! This is not the cause... I think that the problem occurred with the May Galaxy update, but it was unnoticed for a while. Problem is that the histories of all users are not listed when being asked for it within Galaxy. Only the current history is present for every user. In addition, when I login using another computer, a different history is being depicted (which suggest a cookie issue but I just cannot find it). Copying the current history works but the copy is not listed either. In the postgres database, the histories are properly listed. If I type in the proper history name in Galaxy, I even get the stored history back! So this all works, but users cannot see their histories. The java console does not reveal any problem. prefix and cookie are set properly (I think). So I don't know. Does anyone have any clue on how to tackle this issue? I am happy to provide more information if needed! Thanks! Dr. Philip de Groot Ph.D. Bioinformatics Researcher Wageningen University / TIFN Nutrigenomics Consortium Nutrition, Metabolism & Genomics Group Division of Human Nutrition PO Box 8129, 6700 EV Wageningen Visiting Address: Erfelijkheidsleer: De Valk, Building 304 Dreijenweg 2, 6703 HA Wageningen Room: 0052a T: +31-317-485786 F: +31-317-483342 E-mail: Philip.deGroot(a)wur.nl<mailto:Philip.deGroot@wur.nl> Internet: http://www.nutrigenomicsconsortium.nl<http://www.nutrigenomicsconsortium.nl/> http://humannutrition.wur.nl/ https://madmax.bioinformatics.nl/

3 5