- galaxy-dev - lists.galaxyproject.org

Decompression Error
by Johnson, Nathan T. (MU-Student) 29 Nov '12

29 Nov '12

Received the Following Error after uploading from FTP. I'm using Main. Error is: Traceback (most recent call last): File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 384, in <module> __main__() File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 373, in __main__ add_file( dataset, registry, json_file, output_path ) File "/galaxy/home/g2main/galaxy_main/tools/data_source/upload.py", line 134, in add_file chunk = gzipped_file.read( CHUNK_SIZE ) File "/usr/lib/python2.7/gzip.py", line 256, in read self._read(readsize) File "/usr/lib/python2.7/gzip.py", line 307, in _read uncompress = self.decompress.decompress(buf) zlib.error: Error -3 while decompressing: invalid stored block lengths Thank you for your assistance. Nathan T. Johnson Department of Veterinary Pathobiology University of Missouri 4011 Discovery Drive Columbia, MO 65201 Lab: N152 RADIL Lab: (573)884-3429 Email: ntjxv7(a)mail.missouri.edu

1 0

Galaxy API call workflow_execute_parameters.py
by Alex Rodriguez 29 Nov '12

29 Nov '12

Dear Galaxy developers and community, I have been testing and using the latest additions to the galaxy api calls which are great and very useful. The script workflow_execute_parameters.py is great because you can modify parameters on the fly when submitting workflows. However, I ran into a problem with this script. This script works nicely if you have to modify only one parameter per tool, however, if I have 2 or more parameters for the same tool that I need to adjust, it will only adjust one parameter, while ignoring the rest of the parameters in my list for that tool. I believe the issue has to do with line 44 in the script which reassigns the parameter name and value for a tool instead of adding to a list. Would a fix for this just be as simple as assigning a list to the dictionary of multiple parameters for the same tool? Thanks, Alex for v in sys.argv[5:]: print("Multiple arguments "); print(v); try: step, src, ds_id = v.split('='); data['ds_map'][step] = {'src':src, 'id':ds_id}; except ValueError: print("VALUE ERROR:"); wtype, wtool, wparam, wvalue = v.split('='); try: Line 44: data['parameters'][wtool] = {'param':wparam, 'value':wvalue} except ValueError: print("TOOL ID ERROR:");

1 0

dynamic column names and R ExpressionSet files
by S. Horsman 28 Nov '12

28 Nov '12

Hi All, I'm trying to dynamically load labels from a BioConductor ExpressionSet file (binary .RData) into a select item type. This works just fine, but with a few caveats. Example (for reference, this is from a modification of https://trac.nbic.nl/galaxytools/browser/trunk/tools/proteomics/NPC/AMC/DE.…) <code file="CL_code.py"/> ... <page> <param name="eset" type="data" label="Expression set" format="rdata" force_select="True" help="An R data file containing an ExpressionSet"/> </page> <page> <param name="label" type="select" dynamic_options="get_label(eset,False,'class')" force_select="True" label="Variable to test" size="20" help="Name of the variable subject to testing" /> ... I would like to remove the deprecated page tags here (also then the tool can be added to a workflow). However if I do that and then select an ExpressionSet file, the select list does not update. It only works when loading the page for the first time, or when I use the page tags. For text files the solution would be easy; just use type="data_column" and data_ref="yourinputfileparamname" and it auto-updates the select field (though not with the actual column names but that is another issue). With a binary file this is not possible, therefore the dynamic_options feature is needed to do this. Does anyone know of a way to get this to work without using the page tags? Kind regards, Bas Horsman Dept. Bioinformatics Erasmus MC

2 3

Problem with Active Directory/LDAP authentication
by Harris Shapiro 28 Nov '12

28 Nov '12

Hello, I'm writing because I've been trying for the past few days to configure Galaxy to use Apache-based LDAP authentication, but have reached a point where I'm basically stuck. The system in a virtual machine running: - CentOS 5.8 - Apache 2.2.3 I'm trying to configure a Galaxy instance at "localhost:8080", with Active Directory authentication. I started with the admin page documentation: http://wiki.galaxyproject.org/Admin/Config/Apache%20Proxy and also worked through the troubleshooting examples found at: a) http://user.list.galaxyproject.org/Galaxy-Apache-External-Authentication-tt… b) http://gmod.827538.n3.nabble.com/galaxy-dev-ldap-integration-td839409.html c) http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-January/001676.html Following the debugging steps suggested by URL (a), I'm able to verify that: 1) The authentication clause I have works, without rewrite rules and without a proxy. 2) Without rewrite rules and a proxy, the Apache environment has the AUTHENTICATE_SAMACCOUNTNAME and REMOTE_USER environment variables set. Unfortunately, when I turn on the proxy (setting "use_remote_user" and "remote_user_maildomain" in universe_wsgi.ini accordingly), I get the following error message when I try accessing the "localhost:8080" URL, the following error message appears: "Access to Galaxy is denied Galaxy is configured to authenticate users via an external method (such as HTTP authentication in Apache), but a username was not provided by the upstream (proxy) server. This is generally due to a misconfiguration in the upstream server. Please contact your local Galaxy administrator." As suggested by URLs (a) and (c), I modified ./lib/galaxy/web/framework/middleware/remoteuser.py to print the environment seen by that script. None of the remote user information seems to be making to the script, either from AUTHENTICATE_SAMACCOUNTNAME or REMOTE_USER, no matter which of the suggested combinations of "RewriteCond", "RewriteRule", and "RequestHeader" options I tried. In addition, the troubleshooting suggestions to force a REMOTE_USER value to be set via a directive like: RequestHeader add REMOTE_USER [user name] also failed to pass REMOTE_USER to the remoteuser.py script. I've included below an extract from the Apache configuration file that I've been trying. Any advice or assistance that people might be able to provide would be greatly appreciated, and please let me know if you'd need any additional information from me. Sincerely, Harris Shapiro Genomic Health, Inc. Extract from Apache configuration file: Note: I've also tried a variant where I replaced REMOTE_USER with AUTHENTICATE_SAMACCOUNTNAME in the various RewriteCond & RequestHeader directives. That variant produced the same error message. <Proxy http://localhost:8080> Order deny,allow Allow from all </Proxy> RewriteEngine on <Location "/"> AuthType Basic AuthName "Please log in with your Windows account" AuthBasicProvider ldap AuthLDAPURL [Verified LDAP connection information] AuthzLDAPAuthoritative off AuthLDAPBindDN [Verified DN] AuthLDAPBindPassword [Verified password] Require valid-user RewriteCond %{IS_SUBREQ} ^false$ RewriteCond %{LA-U:REMOTE_USER} (.+) RewriteRule . - [E=RU:%1] RequestHeader set REMOTE_USER %{RU}e </Location> RewriteRule ^/static/style/(.*) /home/hshapiro/software/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/static/scripts/(.*) /home/hshapiro/software/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/static/(.*) /home/hshapiro/software/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/favicon.ico /home/hshapiro/software/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/robots.txt /home/hshapiro/software/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P]

4 4

Error running SICER peak caller
by pjt2＠alumni.ubc.ca 28 Nov '12

28 Nov '12

Hi, I am trying to run SICER (version 0.0.1) in galaxy to call peaks using the galaxy interface browser. My workflow is as follows. First I uploaded BAM files of my ChIP tag and input tag data, I then convert these to SAM files with the SAMtools tool, and then convert the SAM files to intervals using the SAM to interval tool. Finally I convert the interval files to BED6 format using the convert format tab in the attributes of each file. At every step the tools work successfully. Finally I go to run SICER on the ChIP tag and input BED6 files I've made but after starting successfully I shortly get the following error: "An error occurred running this job: Traceback (most recent call last): File "/galaxy/home/g2main/galaxy_main/tools/peak_calling/sicer_wrapper.py", line 156, in if __name__=="__main__": __main__() File "/galaxy/home/g2main/galaxy_main/tools/peak_calling/sicer_wrapper.py", li" This is all I can see of the error message. I've tried several times now but it still gives the same error. Am I doing something wrong? How do I get SICER to work in galaxy? Thanks, Peter Thompson, PhD Candidate Department of Medical Genetics Life Sciences Centre, UBC Vancouver, BC CANADA V6T 1Z3

1 0

Re: [galaxy-dev] galaxy-dev Digest, Vol 77, Issue 31
by Jim Johnson 28 Nov '12

28 Nov '12

Ilya, Could you send me a failing input file, and I'll try to find out what is failing? Thanks, JJ > ------------------------------ > > Message: 6 > Date: Tue, 27 Nov 2012 07:22:30 +0000 > From: "Sytchev, Ilya" <isytchev(a)hsph.harvard.edu> > To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> > Subject: [galaxy-dev] No output from IGVtools tile > Message-ID: <CCD9D465.30C7%isytchev(a)hsph.harvard.edu> > Content-Type: text/plain; charset="us-ascii" > > Hi, > > I'm running IGVtools > (http://toolshed.g2.bx.psu.edu/repos/jjohnson/igvtools) tile tool in > Galaxy on a WIG file but it's failing with exit code 1 and producing no > output file. On the other hand, IGVtools count and sort run successfully > in Galaxy on their respective inputs. > > Also, IGVtools tile tool runs successfully by itself from the command line > on the same input WIG file from the same user account that runs this > Galaxy instance. > > Standard out and error from running tile in Galaxy are below. I was > wondering if anyone had a similar problem with IGVtools in their Galaxy > instance. > > Thanks, > Ilya > > > stdout: > > Tile. File = input_file.wig > Max zoom = 7 > Probe file = None > Window functions: mean > IGV directory: /n/galaxy/home/galaxy_scc/igv > INFO [2012-11-27 00:14:07,942] [GenomeManager.java:118] [main] Genome > loaded > > Processing chromosome chr19 > .........10.0% > .........20.0% > .........30.0% > .........40.0% > .........50.0% > ..INFO [2012-11-27 00:14:12,778] [Preprocessor.java:179] [main] Ignoring > data from non-existent locus. Probe = null Locus = > chr19:61342442-61342477. chr19 length = 61342430 > Group idx: 1 > 100.0% > > > > stderr: > > cp: cannot stat > `/n/galaxy/www/galaxy_scc_dev/galaxy-central-hbc-20121101220520/database/jo > b_working_directory/000/100/output.tdf': No such file or directory > >

2 3

DRMAA job will now be errored -> Segmentation fault
by Peter Cock 27 Nov '12

27 Nov '12

Hi all, Something has changed in the job handling, and in a bad way. On my development machine submitting jobs to the cluster didn't seem to be working anymore (never sent to SGE). I killed Galaxy and restarted: Starting server in PID 12180. serving on http://127.0.0.1:8081 galaxy.jobs.runners.drmaa ERROR 2012-11-15 09:56:28,192 (320/None) Unable to check job status Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 296, in check_watched_items state = self.ds.jobStatus( job_id ) File "/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/__init__.py", line 522, in jobStatus _h.c(_w.drmaa_job_ps, jobName, _ct.byref(status)) File "/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/helpers.py", line 213, in c return f(*(args + (error_buffer, sizeof(error_buffer)))) File "/mnt/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.6.egg/drmaa/errors.py", line 90, in error_check raise _ERRORS[code-1]("code %s: %s" % (code, error_buffer.value)) InvalidArgumentException: code 4: Job id, "None", is not a valid job id galaxy.jobs.runners.drmaa WARNING 2012-11-15 09:56:28,193 (320/None) job will now be errored ./run.sh: line 86: 12180 Segmentation fault (core dumped) python ./scripts/paster.py serve universe_wsgi.ini $@ I restarted and it happened again, third time lucky. I presume this was one segmentation fault for each orphaned/zombie job (since I'd tried two cluster jobs which got stuck). I was running with revision 340438c62171, https://bitbucket.org/galaxy/galaxy-central/changeset/340438c62171578078323… as merged into my tools branch, https://bitbucket.org/peterjc/galaxy-central/changeset/d49200df0707579f41fc… Any thoughts? Thanks, Peter

2 7

multithreaded tools
by Andreas Kuntzagk 27 Nov '12

27 Nov '12

Hi, I'm wandering how galaxy supports tools that are multithreaded or multi-process. When working with lastz I noticed that it starts 4 parallel processes. Is that always so? Can this be adjusted? What other tools also are multi-process? regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

5 13

Can't get automatic toolshed installation to work....
by Nikhil Joshi 27 Nov '12

27 Nov '12

Hi all, We just upgraded our local galaxy to the latest version and now I am trying to get automatic tool installation to work properly. I changed the universe_wsgi.ini file to enable the toolshed xml file, I created a "shed_tools" directory one above the galaxy install, and restarted galaxy. But when I try to install a repository, I get this error as a popup: "Initializing repository installation failed". I tried this with various repositories, but I get the same error. Then when I go to "Manage installed tool shed repositories", all of the repositories I tried to install are stuck in the "Cloning" state. Finally, if I try to uninstall any of them, I get this error, with a traceback: AttributeError: 'NoneType' object has no attribute 'startswith' Any ideas? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

3 5

lastz on local galaxy failing
by Andreas Kuntzagk 27 Nov '12

27 Nov '12

Hi, when I start a lastz job it submits a job to GridEngine. But this job seems to hang. The process running is python /data/galaxy/galaxy-dist/tools/sr_mapping/lastz_wrapper.py --ref_source= ... This wrapper seems to not start a lastz but lastz is in the path. After 2 days then Galaxy shows "Job did not produce output" Here is the paster.log from the start of the job. galaxy.jobs DEBUG 2012-11-15 09:03:07,539 (254) Working directory for job is: /data/galaxy/galaxy-dist/database/job_working_directory/000/254 galaxy.jobs.handler DEBUG 2012-11-15 09:03:07,540 dispatching job 254 to drmaa runner galaxy.jobs.handler INFO 2012-11-15 09:03:07,688 (254) Job dispatched galaxy.tools DEBUG 2012-11-15 09:03:07,980 Building dependency shell command for dependency 'lastz' galaxy.tools WARNING 2012-11-15 09:03:07,981 Failed to resolve dependency on 'lastz', ignoring galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,669 (254) submitting file /data/galaxy/galaxy-dist/database/pbs/galaxy_254.sh galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,670 (254) command is: python /data/galaxy/galaxy-dist/tools/sr_mapping/lastz_wrapper.py --ref_source=cached --source_select=pre_set --out_format=sam --input2=/data/galaxy/galaxy-dist/database/files/027/dataset_27795.dat --input1="/data/galaxy/galaxy-dist/tool-data/shared/ucsc/hg18/seq/hg18.2bit" --ref_sequences="None" --pre_set_options=yasra98 --identity_min=0 --identity_max=100 --coverage=0 --output=/data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy_dataset_27800.dat --unmask=yes --lastzSeqsFileDir=/data/galaxy/galaxy-dist/tool-data; cd /data/galaxy/galaxy-dist; /data/galaxy/galaxy-dist/set_metadata.sh ./database/files /data/galaxy/galaxy-dist/database/job_working_directory/000/254 . /data/galaxy/galaxy-dist/universe_wsgi.ini /data/galaxy/galaxy-dist/database/tmp/tmpL4MMJV /data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy.json /data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_in_HistoryDatasetAssociation_288_9ynzEN,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_kwds_HistoryDatasetAssociation_288_AEIseO,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_out_HistoryDatasetAssociation_288_RhsufT,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_results_HistoryDatasetAssociation_288_86DYsR,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/galaxy_dataset_27800.dat,/data/galaxy/galaxy-dist/database/job_working_directory/000/254/metadata_override_HistoryDatasetAssociation_288_HMRNEG galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:08,676 run as user ['kuntzagk', '600'] galaxy.jobs DEBUG 2012-11-15 09:03:08,676 (254) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /data/galaxy/galaxy-dist/database/job_working_directory/000/254 kuntzagk 600 galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:09,148 (254) Job script for external submission is: /data/galaxy/galaxy-dist/database/pbs/254.jt_json 141.80.188.178 - - [15/Nov/2012:09:03:10 +0200] "POST /galaxy/root/history_item_updates HTTP/1.1" 200 - "http://bbc.mdc-berlin.de/galaxy/history" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0" galaxy.jobs.runners.drmaa INFO 2012-11-15 09:03:11,553 (254) queued as 1282605 galaxy.jobs.runners.drmaa DEBUG 2012-11-15 09:03:11,767 (254/1282605) state change: job is running -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

2 3