January 2014 - galaxy-dev - lists.galaxyproject.org

ProFFTPd-1.3.4d no longer able to authenticate users from Postgresql database
by Hazard, E. Starr 27 Jan '14

27 Jan '14

I was able to query my Postgresql Db for user authentication at one time. I deleted and then rebuilt the postgresql DB and rebuilt the entire Galaxy instance in early December. Since then I cannot get ProFFTPd-1.3.4d to authenticate users from Postgresql database. At that time my existing universe_wsgi.ini I changed the following from postgres to postgresql #database_connection = postgres://galaxy:Galaxy2013@localhost:5434/galaxy_prod database_connection = postgresql://galaxy:galaxy2013@localhost:5434/galaxy_prod What ftp interaction behavior changes were included in this database_connectio method renaming? (And how can I fix them?) My Galaxy users can login to Galaxy just fine and execute programs if I manually load files into the designated FTP dir and then move these to database/files . Screen session from interactive test mode FTP session opened. dispatching PRE_CMD command 'USER [valid email]' to mod_core . . . dispatching CMD command 'PASS (hidden)' to mod_auth no supplemental groups found for user '[valid email]' ROOT PRIVS at mod_auth_pam.c:338 RELINQUISH PRIVS at mod_auth_pam.c:508 mod_sql_passwd/0.4: expected 'PBKDF2$sha256$10000$1xxxxxxxxxx',got Œyyyyyy¹ I am not using either PBKDF2 or sha256 see below why are these prefixes appended? Xxx and yyyy are NOT the same alphanumeric strings SQLLogFile.txt excerpt: Jan 06 17:27:49 mod_sql/4.3[29179]: query "SELECT email,password,' <mailto:'irish(a)musc.edu>[valid email]','{UNKNOWN TAG}',¹/PATH/galaxy-dist/database/files/ <mailto:'/shared/app/Galaxy/galaxy-dist/database/files/irish(a)musc.edu>[vali d email]','/bin/ bash' FROM galaxy_user WHERE email=' <mailto:email='irish(a)musc.edu>[valid email]'" Jan 06 17:27:49 mod_sql/4.3[29179]: entering postgres cmd_close Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 1 Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_close Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_select Jan 06 17:27:49 mod_sql/4.3[29179]: <<< process_named_query 'LookupGalaxyUser' Jan 06 17:27:49 mod_sql/4.3[29179]: <<< sql_lookup Jan 06 17:27:49 mod_sql/4.3[29179]: user UID 0 below SQLMinUserUID 999, using SQLDefaultUID 65533 Jan 06 17:27:49 mod_sql/4.3[29179]: user GID 0 below SQLMinUserGID 999, using SQLDefaultGID 65533 Jan 06 17:27:49 mod_sql/4.3[29179]: cache miss for user ' <mailto:'irish(a)musc.edu>[valid email]' Jan 06 17:27:49 mod_sql/4.3[29179]: user ' <mailto:'irish(a)musc.edu>[valid email]' cached Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_name : [valid email] Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_uid : 65533 Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_gid : 65533 Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_dir : /PATH/galaxy-dist/database/files/ <mailto:/shared/app/Galaxy/galaxy-dist/database/files/irish@musc.edu>[valid email] Jan 06 17:27:49 mod_sql/4.3[29179]: + pwd.pw_shell : /bin/bash Jan 06 17:27:49 mod_sql/4.3[29179]: <<< cmd_getpwnam Jan 06 17:27:49 mod_sql/4.3[29179]: >>> cmd_auth Jan 06 17:27:49 mod_sql/4.3[29179]: entering postgres cmd_escapestring Jan 06 17:27:49 mod_sql/4.3[29179]: entering postgres cmd_open Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 2 Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_open Jan 06 17:27:49 mod_sql/4.3[29179]: entering postgres cmd_close Jan 06 17:27:49 mod_sql/4.3[29179]: connection 'default' count is now 1 Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_close Jan 06 17:27:49 mod_sql/4.3[29179]: exiting postgres cmd_escapestring Jan 06 17:27:49 mod_sql/4.3[29179]: cache hit for user '[valid email]' Jan 06 17:27:49 mod_sql/4.3[29179]: >>> cmd_check Jan 06 17:27:49 mod_sql/4.3[29179]: checking password using SQLAuthType 'sha1' Jan 06 17:27:49 mod_sql/4.3[29179]: 'sha1' SQLAuthType handler reports failure ProFTPd conf file excerpt: (based on https://wiki.galaxyproject.org/Admin/Config/Upload%20via%20FTP) # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex # Set up SQLLogfile SQLLogFile /PATH/ProFFTPd-1.3.4d/etc/SQLLogFile.txt # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxy_prod@:5xxx galaxy galpwd # port and password are changed here; they are correct in the working files. SQLAuthTypes SHA1 SQLAuthenticate users # An empty directory in case chroot fails SQLDefaultHomedir /shared/app/ProFFTPd-1.3.4d/default # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email,password,¹1233¹,'1234',¹/PATH/galaxy-dist/database/files/%U ','/bin/bash' FROM galaxy_user WHERE email='%U'"

2 3

Galaxy problem with DRMAA
by Jingchao Zhang 27 Jan '14

27 Jan '14

Hi everybody, I managed to configure Galaxy to work with DRMAA in our cluster but after the job finished, the dataset state is set to error. Here is the last few lines of the paster.log file: galaxy.jobs.runners.drmaa DEBUG 2014-01-27 10:52:50,449 (181) submitting with credentials: jing [uid: 3663] galaxy.jobs.runners.drmaa DEBUG 2014-01-27 10:52:50,490 (181) Job script for external submission is: /work/galaxy/galaxy/galaxy-dist/database/pbs/181.jt_json galaxy.jobs.runners.drmaa INFO 2014-01-27 10:52:51,175 (181) queued as 961147 galaxy.jobs DEBUG 2014-01-27 10:52:51,225 (181) Persisting job destination (destination id: drmaa) galaxy.jobs.runners.drmaa DEBUG 2014-01-27 10:52:51,781 (181/961147) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2014-01-27 10:52:56,004 (181/961147) state change: job finished normally galaxy.jobs DEBUG 2014-01-27 10:52:56,276 (181) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /work/galaxy/galaxy/galaxy-dist/database/job_working_directory/000/181 galaxy 11295 galaxy.jobs DEBUG 2014-01-27 10:52:57,161 (181) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /work/galaxy/galaxy/galaxy-dist/database/job_working_directory/000/181 galaxy 11295 galaxy.jobs DEBUG 2014-01-27 10:52:59,777 setting dataset state to ERROR galaxy.jobs DEBUG 2014-01-27 10:52:59,871 job 181 ended In the browser, the stderr file says: Traceback (most recent call last): File "/work/galaxy/galaxy/galaxy-dist/tools/data_source/data_source.py", line 6, in <module> from galaxy.util.json import from_json_string, to_json_string File "/work/galaxy/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", line 14, in <module> pkg_resources.require( 'docutils' ) File "/work/galaxy/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 414, in require raise EggNotFetchable( str( [ egg.name for egg in e.eggs ] ) ) galaxy.eggs.EggNotFetchable: ['docutils'] Traceback (most recent call last): File "./scripts/set_metadata.py", line 29, in <module> import galaxy.model.mapping #need to load this before we unpickle, in order to setup properties assigned by the mappers File "/work/galaxy/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 10, in <module> eggs.require("pexpect") File "/work/galaxy/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 414, in require raise EggNotFetchable( str( [ egg.name for egg in e.eggs ] ) ) galaxy.eggs.EggNotFetchable: ['pexpect']? The "docutils" and "pexpect" eggs are in the /work/galaxy/galaxy/galaxy-dist/eggs folder and I don't see why they are not fetchable. Any help is appreciated. Thanks. -Jing

1 0

Fwd: Galaxy tool is not running
by Eric Kuyt 27 Jan '14

27 Jan '14

Hi Greg or James, This morning I got a question about the Aggregate Datapoints tool which only returned nan results. in line 204@ aggregate_scores_in_intervals.py<https://bitbucket.org/galaxy/galaxy-dist/src/95bf71620d50c6d81248eef2001a7d…> I found out that the if not is NaN ( score ): should be if not isnan( score ): on https://usegalaxy.org/tool_runner?tool_id=aggregate_scores_in_intervals2 the tool also seems to be failing. I hope you could use this, Eric ---------- Forwarded message ---------- From: Sandra Gavaldá Martín Date: 26 January 2014 20:52< Subject: Galaxy tool is not running To: "Kuijt, Eric" <eric.kuijt(a)wur.nl> Hello Eric, My name is Sandra Gavaldá and I am very grateful for Galaxy tools. I usually use the one "Aggregate datapoints" for fusioning files with intervals and my .wig files. I noticed you made some updating and I do not why I can not obtain any result now. -- Central Veterinary Institute of Wageningen UR (CVI) Department of Infection Biology PO box 65, 8200 AB Lelystad, NL Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad Tel: +31-(0)320-293391 Fax: +31-(0)320-238153 E-mail: eric.kuijt(a)wur.nl Web: http://www.cvi.wur.nl

2 1

galaxy tries to access toolshed at http instead of using https
by Jager, Victor de 27 Jan '14

27 Jan '14

dear all, When I try to install a tool from our local toolshed Galaxy changes the repository url from HTTPS to HTTP and throws an error. I'm setting up a local (intranet) toolshed for some of our in house tools. We run galaxy (latest stable) on HTTPS://nioo0025.nioo.int/galaxy The toolshed is setup at HTTPS://nioo0025.nioo.int/toolshed My tool_sheds_conf.xml contains: (note the HTTPS protocol) <?xml version="1.0"?> <tool_sheds> <tool_shed name="Galaxy main tool shed" url="http://toolshed.g2.bx.psu.edu/"/> <tool_shed name="Galaxy test tool shed" url="http://testtoolshed.g2.bx.psu.edu/"/> <tool_shed name="NIOO Galaxy tool shed" url="https://nioo0025.nioo.int/toolshed/"/> </tool_sheds> So far so good. The toolshed works and I can upload tool tarballs I created. Galaxy sees the toolshed and the uploaded tool. When I try to install the tool I get the following message: paster.log:tool_shed.tool_shed_registry DEBUG 2014-01-27 16:00:13,485 Invalid url 'http://nioo0025.nioo.int/toolshed/' received by tool shed registry's password_manager_for_url method. paster.log:10.128.34.21 - - [27/Jan/2014:16:00:13 +0200] "GET /galaxy/admin_toolshed/prepare_for_install?tool_shed_url=http://nioo0025.nioo.int/toolshed/&repository_ids=6c73c198d6105528&changeset_revisions=d4127427fd4c HTTP/1.1" 500 - "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0" basically telling me that our Galaxy instance tries to access the toolshed repo via HTTP instead of HTTPS. How can I instruct Galaxy to use the https location of the toolshed as specified in tool_sheds_conf.xml? Thanks for any help Victor

1 0

Help with two questions about transfering data to galaxy local instance
by Huayan Gao 27 Jan '14

27 Jan '14

Hi, I am doing a data transferring interface in galaxy. I want to accomplish two things. First, when user enters a number, the interface will show some related information or search results (eg, related files) by this unique number. Then user can select the files they wanted to put into galaxy history panel for further data analysis. So my question is, can add a button in xml interface or python code to show the search results? The second question is that the files could be big. How can I transfer the files faster to my local galaxy instance? The current copy is just too slow. Thanks!

2 1

Remote User Logout
by Alper Kucukural 25 Jan '14

25 Jan '14

Hi, I am using LDAP for authentication in galaxy mirror so remoteUser is enabled. But logout is not working. How can I use standard logout in this case? Thanks, Alper Kucukural, PhD Bioinformatics Core, University of Massachusetts Medical School 368 Plantation St.Room AS4.2067 Worcester, MA 01605-2324 Phone: 774-312-4493 E-mail: alper(a)kucukural.com

2 2

Toolshed throwing error when reading tool's XML
by Dooley, Damion 24 Jan '14

24 Jan '14

I'm stumped: Can tools that are destined for the toolshed use <options from_data_table="..." ...> ? In a blast_reporting.xml tool script I have <param name="filter_column" type="select" label="Col"> <options from_data_table="blast_reporting_fields"> <filter type="static_value" value="numeric" column="type" /> <filter type="sort_by" column="name"/> </options> </param> But when this gets submitted as a tar file into a toolshed repository, toolshed comes back with cryptic: Metadata may have been defined for some items in revision '5bbb25b32de2'. Correct the following problems if necessary and reset metadata. blast_reporting.xml - invalid literal for int() with base 10: 'type' I eliminate the <filter type="static_value" value="numeric" column="type" /> line, but similar problem is still reported blast_reporting.xml - invalid literal for int() with base 10: 'name' This hints that maybe the toolshed, when it tests the code, it doesn't know what/where blast_reporting_fields is? Is there some special location that the blast_reporting_fields.loc file should be located within the uploaded tar file? (I tried putting it in various relative paths in the tar file, i.e. root, and under /tool-data/) p.s. the tool runs file with menu functioning in galaxy itself. Thanks for any pro tips you folks can come up with! Damion

2 4

error importing function definition for `module'
by Cantarel, Brandi L. 24 Jan '14

24 Jan '14

Hi all, I am trying to set up my own galaxy server (and I don't know much IT) and thought that I had my sge working properly, but when I try to load data -- via manage data libraries on the web interface, I get the following error: /bin/sh: module: line 1: syntax error: unexpected end of file /bin/sh: error importing function definition for `module' This works if I do not use SGE. Anyone know what this error could mean?? Here are the 2 lines in universal_wsgi.ini that are used for SGE: default_cluster_job_runner = drmaa://-V -v TMPDIR=/scratch/ job_config_file = job_conf.xml Here is my job_conf.xml file: <?xml version="1.0"?> <job_conf> <plugins workers="4">  <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/> </plugins> <handlers default="handlers">  <handler id="main" tags="handlers"/> </handlers> <destinations default="sge_default">  <destination id="sge_default" runner="sge"> <param id="nativeSpecification">-V</param> </destination>  </destinations> </job_conf> Thanks! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Brandi Cantarel, PhD Bioinformatics Research Scientist Baylor Institute for Immunology Research Baylor Health Care System 214-820-9064 (office) ********************************************************************** This e-mail may contain confidential and/or privileged information. This information is intended only for the use of the individual(s) and entity(ies) to whom it is addressed. If you are the intended recipient, further disclosures are prohibited without proper authorization. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden and possibly a violation of federal or state law and regulations. Baylor Health Care System, its subsidiaries, and affiliates hereby claim all applicable privileges related to this information.

1 0

Fetch Taxonomic Representation tool problem
by Carolyn Rogers 24 Jan '14

24 Jan '14

Hello, I am trying to run the Fetch taxonomic representation tool on Galaxy, but with the data set that I have printed below, the system will not let me select column 2 for my GI number. Do you know why this might be and how I could combat it? Thank you very much Carolyn cds.comp100014_c0_seq1|m.104348gi|134287508|ref|YP_001109675.1| 42.071457831 1442313704e-2198.6cds.comp10004_c0_seq1|m.114798 gi|498343962|ref|WP_010658118.1| 62.4012543111214395631e-38148 cds.comp100093_c0_seq1|m.108636gi|544496539|ref|XP_005582687.1| 99.23130105 1345917201e-77258cds.comp100094_c0_seq1|m.121852 gi|495312497|ref|WP_008037245.1| 52.7311049211092093167e-28113 cds.comp100159_c0_seq1|m.120408gi|518255305|ref|WP_019425513.1| 82.881111901 1111262365e-59196cds.comp100404_c0_seq1|m.108184 gi|402903411|ref|XP_003914559.1| 100.001360011361322671e-84264 cds.comp100496_c0_seq1|m.106706gi|498530561|gb|AGL46876.1| 55.13782911781384 5e-1885.5cds.comp100500_c0_seq1|m.116374gi|530409897|ref|XP_005256503.1| 100.00117001117902066e-71230cds.comp100531_c0_seq1|m.40559 gi|426344949|ref|XP_004039166.1| 99.34452301452745250.0944 cds.comp100557_c0_seq1|m.102925gi|497399495|ref|WP_009713708.1| 47.651496245 151321662e-31123cds.comp100567_c0_seq1|m.115616 gi|496179479|ref|WP_008903986.1| 96.64119401119832015e-77241 cds.comp100604_c0_seq1|m.119801gi|512938446|ref|XP_004908634.1| 97.09103306 108361389e-64208cds.comp100642_c0_seq1|m.107918 gi|55956930|ref|NP_001007247.1| 100.00106003113611066e-70217 cds.comp100642_c0_seq2|m.112536gi|55956930|ref|NP_001007247.1| 100.00660031 961667e-39136cds.comp100674_c0_seq1|m.105267gi|2224583|dbj|BAA20779.1|100.00 1440011444686117e-92301cds.comp100694_c0_seq1|m.69083 gi|109084155|ref|XP_001082932.1| 99.59246101125612460.0508 cds.comp100761_c0_seq1|m.101359gi|302565456|ref|NP_001180898.1| 100.00109001 1091032111e-74231cds.comp100781_c0_seq1|m.117426 gi|498807452|ref|WP_010841550.1| 42.53873741741182044e-0961.6

1 0

Unified Genotyper index file error
by Heidi Pagán 24 Jan '14

24 Jan '14

I used BWA on a local galaxy installation (Ubuntu 12.10) to generate an alignment of my FastQ reads. Now I am trying to use GATK Unified Genotyper with my own reference sequence. The first time I ran it, I had forgotten to edit env.sh to show the path to the GATK .jar file. After I fixed that, the next time I tried to run it I got the error shown below. If I understand this correctly, it already tried to create a tmp file from the first run and now is unable to continue until I delete it. However, I cannot locate the file. 1) Am I correct in my understanding of the error? 2) How do I delete the file to clear the error? Thanks everyone! [Thu Jan 23 22:24:59 CST 2014] net.sf.picard.sam.CreateSequenceDictionary REFERENCE=/tmp/tmp-gatk-BG9Hkr/gatk_input.fasta OUTPUT=/tmp/tmp-gatk-BG9Hkr/dict7357081511697052096.tmp TRUNCATE_NAMES_AT_WHITESPACE=true NUM_SEQUENCES=2147483647 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false [Thu Jan 23 22:24:59 CST 2014] Executing as heidi@heidi-HP-Pavilion-dv6-Notebook-PC on Linux 3.5.0-44-generic amd64; OpenJDK 64-Bit Server VM 1.7.0_25-b30; Picard version: null [Thu Jan 23 22:24:59 CST 2014] net.sf.picard.sam.CreateSequenceDictionary done. Elapsed time: 0.00 minutes. Runtime.totalMemory()=173146112 FAQ: http://sourceforge.net/apps/mediawiki/picard/index.php?title=Main_Page ##### ERROR ------------------------------------------------------------------------------------------ ##### ERROR A USER ERROR has occurred (version 2.4-7-g5e89f01): ##### ERROR The invalid arguments or inputs must be corrected before the GATK can proceed ##### ERROR Please do not post this error to the GATK forum ##### ERROR ##### ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments. ##### ERROR Visit our website and forum for extensive documentation and answers to ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk ##### ERROR ##### ERROR MESSAGE: Index file /tmp/tmp-gatk-BG9Hkr/gatk_input.dict does not exist but could not be created because: . /tmp/tmp-gatk-BG9Hkr/dict7357081511697052096.tmp already exists. Delete this file and try again, or specify a different output file. ##### ERROR ------------------------------------------------------------------------------------------

2 1