- galaxy-dev - lists.galaxyproject.org

Announcement: Galaxy DevOps Workshop
by Bjoern Gruening 17 Mar '16

17 Mar '16

Hi all, We would like to invite you to our Galaxy DevOps Workshop in Heidelberg from 6-7 of April, organised by de.NBI and ELIXIR. The workshop is intended for Bioinformaticians who want to * learn the internals of the multi-omics Galaxy workbench * learn how to set up and run a Galaxy server * learn about Galaxy tool development * get questions answered from experienced Galaxy admins and community members * are interested in reproducible and accessible research For more information please see https://goo.gl/8dNSdy Ciao, Bjoern

1 0

Using forms to collect user info at registration
by Peter Briggs 17 Mar '16

17 Mar '16

Hello I'm trying to set up a form within Galaxy to collect some user info at registration time. The only documentation I've found on this functionality is: https://wiki.galaxyproject.org/Admin/Config/User%20Information which seems to be precisely what I need. However I can't make this work as described. When I set up a form and then go to the registration page, instead of the additional form options I get a single field called "User type" with a drop-down menu widget with a single option ("User registration form" i.e. the name of the form that I created). I've looked to see if there is a config option to turn something on, or if I need to create a role, but so far nothing I've tried seems to make a difference. Does anyone have any suggestions on how I might get this feature to operate as advertised? (Btw I'm seeing this in both 15.10 and 16.01.) Thanks for your help, Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

Galaxy drops my tenth xml tool parameter
by rbrown1422＠comcast.net 16 Mar '16

16 Mar '16

Good afternoon, It appears my Galaxy updated overnight and has caused issues that I am fairly certain I didn't introduce. Did something change or what did I do? 1 the Parameter $GALAXY_ROOT_DIR/tools/Heatmaps/' is not showing up see bottom where I dump parameters. 2. The repeat seems to always run once even when I do not click and fill it. This was working a couple days ago. I was playing with it - but very minimally-- yeah i understand but have worked with Galaxy for awhile. Thanks Bob 3. <?xml version="1.0" encoding="UTF-8" ?><tool id="heatmap" name="Heatmap" version="3.1.1"> <description>Create HeatMap </description><command>$GALAXY_ROOT_DIR/tools/MDA_Heatmaps/heatmap.sh '$hmname' '$hmdesc' '$inputmatrix' $rowOrderMethod $rowDistanceMeasure $rowAgglomerationMethod $columnOrderMethod $columnDistanceMeasure $columnAgglomerationMethod '$summarymethod' '$GALAXY_ROOT_DIR/tools/Heatmaps/' #for $op in $operations ${op.class_name} ${op.repeatinput.file_name} ${op.cat} #end for '$output' </command> <stdio> <exit_code range="1:" level="fatal" /> </stdio> <inputs> <param name="hmname" size="20" type="text" value="HM_name" label="User Defined HeatMap Name"/> <param name="hmdesc" size="100" optional="true" type="text" value="_" label="Optional HeatMap Description"/> <param name="inputmatrix" type="data" label="Input Matrix" /> <param name="summarymethod" type="select" label="Data Summarization Method"> <option value="average">average</option> <option value="sample">sample</option> <option value="mode">mode</option> </param> <param name="rowOrderMethod" type="select" label="Row ordering method" help="Choices -- Hierarchical Clustering, Original Order, Random"> <option value="HierarchicalClustering">Hierarchical Clustering</option> <option value="OriginalOrder">Original Order</option> <option value="Random">Random</option> </param> <param name="rowDistanceMeasure" type="select" label="Row Distance Metric" help="euclidean, binary, manhattan, maximum, canberra, minkowski, or correlation"> <option value="euclidean">Euclidean</option> <option value="binary">Binary</option> <option value="manhattan">Manhattan</option> </param> <param name="rowAgglomerationMethod" type="select" label="Row Clustering Method" help="Choices: 'average' for Average Linkage, 'complete' for Complete Linkage, 'single' for Single Linkage, 'ward', 'mcquitty', 'median', or 'centroid'."> <option value="average">Average Linkage</option> <option value="complete">Complete Linkage</option> <option value="single">Single Linkage</option> </param> <param name="columnOrderMethod" type="select" label="Column ordering method" help="Choices -- Hierarchical Clustering, Original Order, Random"> <option value="HierarchicalClustering">Hierarchical Clustering</option> <option value="OriginalOrder">Original Order</option> <option value="Random">Random</option> </param> <param name="columnDistanceMeasure" type="select" label="Column Distance Metric" help="euclidean, binary, manhattan, maximum, canberra, minkowski, or correlation"> <option value="euclidean">Euclidean</option> <option value="binary">Binary</option> <option value="manhattan">Manhattan</option> <option value="correlation">Correlation</option> </param> <param name="columnAgglomerationMethod" type="select" label="Column Clustering Method" help="Choices: 'average' for Average Linkage, 'complete' for Complete Linkage, 'single' for Single Linkage, 'ward', 'mcquitty', 'median', or 'centroid'."> <option value="average">Average Linkage</option> <option value="complete">Complete Linkage</option> <option value="single">Single Linkage</option> <option value="ward">Ward</option> <option value="mcquitty">Mcquitty</option> <option value="median">Median</option> <option value="centroid">Centroid</option> </param> <repeat name="operations" title="Covariate Bars"> <param name="class_name" size="20" type="text" value="" label="Axis Covariate Name"/> <param name="repeatinput" type="data" format="text" label="Axis Covariate File"/> <param name="cat" type="select" label="Axis Covariate Type"> <option value="row_categorical" >row categorical</option> <option value="row_continuous" >row continuous</option> <option value="column_categorical" >column categorical</option> <option value="column_continuous" >column continuous</option> </param> </repeat> </inputs> <outputs> <data name="output" label="${inputmatrix.name}_Heatmap" format="zip"/> </outputs></tool> 4. dump of part of heatmap.sh commands echo $1 $2 $3 $4 $5 $6 $7 $8 $9 $10echo 11-14 belowecho $11 $12 $13 $14 5. shows in Stdout the 10th parameter '$GALAXY_ROOT_DIR/tools/Heatmaps/' is missing The first 10 params are: namexyz test /home/rbrown/HM_galaxy/database/files/000/dataset_500.dat HierarchicalClustering euclidean average HierarchicalClustering euclidean average namexyz0 11-14 below namexyz1 namexyz2 namexyz3 namexyz47. and where do the namexyzN come from? I didn't click the repeat -- yes one of them should by file Output

1 0

Setting job_working_directory on per-tool basis? (cuffdiff/cummerbund SQLite db error)
by Peter Briggs 16 Mar '16

16 Mar '16

Hello I've encountered an issue with running the cuffdiff tool on our local production Galaxy instance. Specifically when a cummerbund SQLite database is requested as output then the tool fails with an error: Creating database ./cummeRbund.sqlite Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Error in sqliteSendQuery(con, statement, bind.data) : error in statement: disk I/O error Calls: readCufflinks ... .local -> sqliteGetQuery -> sqliteSendQuery -> .Call Execution halted Our Galaxy is configured to use a LustreFS scratch directory for the 'job_working_directory', which I understand is where each job will have its own working directory created, and I'm wondering if the SQLite error above is a manifestation of a general problem with SQLite on LustreFS (see e.g. the answer to this BioStars question https://www.biostars.org/p/115452/) The obvious workaround is to change 'job_working_directory', preferably just for the cuffdiff tool. Does anyone know if it's possible to do this i.e. set 'job_working_directory' on a per-tool basis? (Also, has anyone else seen this specific problem with cuffdiff/cummerbund/SQLite on their local Galaxy instances?) Thanks for your help - any suggestions greatly appreciated, Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

Please help fix toolshed repository: trtr
by Lance Parsons 15 Mar '16

15 Mar '16

I was helping someone with their toolshed repository and we ran into an issue, there is an old folder "myTools" in the repository that we cannot delete. It has a second copy of the tool (inadvertently). However, when we upload a new version with Planemo, the directory remains. Attempts to delete it via the toolshed web interface result in internal server errors. Can an admin take a look? https://toolshed.g2.bx.psu.edu/view/mrvollger/trtr/13cad5d36301 -- Lance Parsons - Scientific Programmer Carl C. Icahn Laboratory - Room 136 Lewis-Sigler Institute for Integrative Genomics Princeton University

2 2

Help with dynamic job destination
by Lance Parsons 15 Mar '16

15 Mar '16

I'm trying to setup a dynamic job destination for bowtie2 so that it submits to a different destination if the `-a` option is set or the `-k` option is set to a high value. I've got things working, but the function I've written seems rather complicated and ugly. I was wondering if anyone had any advice on a better way to query job parameters. See the function here: https://gist.github.com/lparsons/a4724b51afd9027e3c0e Thanks in advance for any suggestions or comments. -- Lance Parsons - Scientific Programmer Carl C. Icahn Laboratory - Room 136 Lewis-Sigler Institute for Integrative Genomics Princeton University

1 0

Job stuck in running state.
by Zuzanna K. Filutowska 15 Mar '16

15 Mar '16

Dear All, I am running newest version of Galaxy with PBS and I am observing strange behaviour: some jobs that were finished on the queue system are still shown as running on Galaxy Web interface. Communication with PBS server seems to work fine all the time. Nothing interesting seems to be in logs despite debug being on: sqlalchemy.pool.QueuePool ERROR 2016-03-13 11:36:02,544 Error closing cursor Any ideas, what can be a problem? -- Pozdrawiam, -- Zuzanna K. Filutowska, HPC Systems Administrator Poznan Supercomputing and Networking Center Institute of Bioorganic Chemistry Polish Academy of Sciences Seize the day boys! Make your lifes extraordinary! --John Keating

2 1

Re: [galaxy-dev] Trackster in 16.01 broken
by Marius van den Beek 15 Mar '16

15 Mar '16

Hi Lubos, I opened a Pull Request on https://github.com/galaxyproject/galaxy/pull/1930, which should hopefully fix the problem. If you want to give it a try, you can do `git fetch origin pull/1930/head:more_bytestring_for_pysam` and `git checkout more_bytestring_for_pysam`. When the problem has been solved, you can switch back to release_16.01 by doing `git checkout release_16.01`. Thanks for reporting the error, Marius On 15 March 2016 at 19:25, Marius van den Beek <m.vandenbeek(a)gmail.com> wrote: > Yes, `git checkout release_16.01` should do it, but wait a second, there > might actually be a problem, > since ` File "/usr/local/galaxy-prod/lib/galaxy/visualization/data_providers/genome.py", > line 892, in get_iterator`, > while the PR addresses 354-356! > > > On 15 March 2016 at 19:23, Lubos Klucar <klucar(a)embnet.sk> wrote: > >> >> >> On 15. 3. 2016 19:15, Marius van den Beek wrote: >> >>> Hi Lubos, >>> >>> hmm, I'm surprised about this. >>> Just to make sure, you are on the release_16.01 branch? >>> (`git branch` should highlight release_16.01) >>> This PR hasn't been merged into dev yet. >>> >>> >> Hi Marius, >> >> it seems I need more help with git, not the Galaxy!. ;-) My git branch >> gives me: >> >> dev >> * master >> >> How should I 'switch' to 16.01??.. >> >> regards >> >> Lubos >> >> Cheers, >>> Marius >>> >>> On 15 March 2016 at 19:08, Lubos Klucar <klucar(a)embnet.sk >>> <mailto:klucar@embnet.sk>> wrote: >>> >>> Dear Marius, >>> >>> thanks for help, but it looks our galaxy is up to date: >>> >>> remote: Counting objects: 111, done. >>> remote: Compressing objects: 100% (23/23), done. >>> remote: Total 111 (delta 82), reused 75 (delta 75), pack-reused 13 >>> Receiving objects: 100% (111/111), 39.85 KiB, done. >>> Resolving deltas: 100% (82/82), completed with 29 local objects. >>> >From https://github.com/galaxyproject/galaxy >>> 8423fe7..23c3300 dev -> origin/dev >>> 5a50a46..65a5b08 release_16.01 -> origin/release_16.01 >>> Already up-to-date. >>> >>> After Galaxy restart the problem still persist... >>> >>> all the best >>> >>> Lubos >>> >>> On 15. 3. 2016 18:59, Marius van den Beek wrote: >>> >>> Hello Lubos, >>> >>> this should be fixed with >>> https://github.com/galaxyproject/galaxy/pull/1897. >>> If you pull in the latest changes (git pull) on the >>> release_16.01 branch >>> things should work again. >>> >>> Cheers, >>> Marius >>> >>> On 15 March 2016 at 18:54, Lubos Klucar <klucar(a)embnet.sk >>> <mailto:klucar@embnet.sk> >>> <mailto:klucar@embnet.sk <mailto:klucar@embnet.sk>>> wrote: >>> >>> Hi, >>> >>> after upgrading from 15.10 to 16.01, the trackster >>> visualisation >>> tool is partly defective. It can successfully show e.g. GFF >>> tracks, >>> but after adding e.g. BAM track nothing is shown. The only >>> suspicious debug output I could get when loading >>> non-working-track is: >>> >>> galaxy.webapps.galaxy.api.datasets ERROR 2016-03-15 >>> 11:44:38,853 >>> Error in dataset API at listing contents: Expected bytes, >>> got >>> unicode: Expected bytes, got unicode >>> Traceback (most recent call last): >>> File >>> >>> >>> "/usr/local/galaxy-prod/lib/galaxy/webapps/galaxy/api/datasets.py", >>> line 66, in show >>> rval = self._data( trans, dataset, **kwd ) >>> File >>> >>> >>> "/usr/local/galaxy-prod/lib/galaxy/webapps/galaxy/api/datasets.py", >>> line 235, in _data >>> ref_seq=region, mean_depth=mean_depth, **kwargs ) >>> File >>> >>> >>> "/usr/local/galaxy-prod/lib/galaxy/visualization/data_providers/genome.py", >>> line 192, in get_data >>> iterator = self.get_iterator( data_file, chrom, start, >>> end, >>> **kwargs ) >>> File >>> >>> >>> "/usr/local/galaxy-prod/lib/galaxy/visualization/data_providers/genome.py", >>> line 892, in get_iterator >>> data = data_file.fetch( start=start, end=end, >>> reference=chrom ) >>> File "pysam/calignmentfile.pyx", line 868, in >>> pysam.calignmentfile.AlignmentFile.fetch >>> (pysam/calignmentfile.c:10170) >>> File "pysam/calignmentfile.pyx", line 787, in >>> pysam.calignmentfile.AlignmentFile.parse_region >>> (pysam/calignmentfile.c:9605) >>> File "pysam/calignmentfile.pyx", line 1576, in >>> pysam.calignmentfile.AlignmentFile.gettid >>> (pysam/calignmentfile.c:16665) >>> File "pysam/calignmentfile.pyx", line 640, in >>> pysam.calignmentfile.AlignmentFile.get_tid >>> (pysam/calignmentfile.c:8208) >>> File "pysam/cutils.pyx", line 106, in >>> pysam.cutils.force_bytes >>> (pysam/cutils.c:2170) >>> TypeError: Expected bytes, got unicode >>> >>> Any help to solve this problem would be highly appreciated! >>> >>> many thanks >>> -- >>> >>> Lubos Klucar >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >>> >>> >>> >>> -- >>> >>> Lubos Klucar >>> >>> Institute of Molecular Biology >>> Slovak Academy of Science >>> Dubravska cesta 21 >>> 845 51 Bratislava 45 >>> Slovakia >>> >>> Tel.: +421 2 5930 7413 >>> <tel:%2B421%202%205930%207413> >>> Fax.: +421 2 5930 7416 >>> <tel:%2B421%202%205930%207416> >>> >>> >>> >> -- >> >> Lubos Klucar >> >> Institute of Molecular Biology >> Slovak Academy of Science >> Dubravska cesta 21 >> 845 51 Bratislava 45 >> Slovakia >> >> Tel.: +421 2 5930 7413 >> Fax.: +421 2 5930 7416 >> > >

2 1

Trackster in 16.01 broken
by Lubos Klucar 15 Mar '16

15 Mar '16

Hi, after upgrading from 15.10 to 16.01, the trackster visualisation tool is partly defective. It can successfully show e.g. GFF tracks, but after adding e.g. BAM track nothing is shown. The only suspicious debug output I could get when loading non-working-track is: galaxy.webapps.galaxy.api.datasets ERROR 2016-03-15 11:44:38,853 Error in dataset API at listing contents: Expected bytes, got unicode: Expected bytes, got unicode Traceback (most recent call last): File "/usr/local/galaxy-prod/lib/galaxy/webapps/galaxy/api/datasets.py", line 66, in show rval = self._data( trans, dataset, **kwd ) File "/usr/local/galaxy-prod/lib/galaxy/webapps/galaxy/api/datasets.py", line 235, in _data ref_seq=region, mean_depth=mean_depth, **kwargs ) File "/usr/local/galaxy-prod/lib/galaxy/visualization/data_providers/genome.py", line 192, in get_data iterator = self.get_iterator( data_file, chrom, start, end, **kwargs ) File "/usr/local/galaxy-prod/lib/galaxy/visualization/data_providers/genome.py", line 892, in get_iterator data = data_file.fetch( start=start, end=end, reference=chrom ) File "pysam/calignmentfile.pyx", line 868, in pysam.calignmentfile.AlignmentFile.fetch (pysam/calignmentfile.c:10170) File "pysam/calignmentfile.pyx", line 787, in pysam.calignmentfile.AlignmentFile.parse_region (pysam/calignmentfile.c:9605) File "pysam/calignmentfile.pyx", line 1576, in pysam.calignmentfile.AlignmentFile.gettid (pysam/calignmentfile.c:16665) File "pysam/calignmentfile.pyx", line 640, in pysam.calignmentfile.AlignmentFile.get_tid (pysam/calignmentfile.c:8208) File "pysam/cutils.pyx", line 106, in pysam.cutils.force_bytes (pysam/cutils.c:2170) TypeError: Expected bytes, got unicode Any help to solve this problem would be highly appreciated! many thanks -- Lubos Klucar

2 3

Problems uploading files to galaxy
by Lemley, Rob 15 Mar '16

15 Mar '16

Hi, I'm hoping you can help out with a problem we are experiencing uploading files to our own Galaxy instance. When a user tries to upload a file to Galaxy through their web browser, the job errors out: Traceback (most recent call last): File "/gpfs/fs1/sfw/cibem_galaxy/app/galaxy/tools/data_source/upload.py", line 430, in <module> __main__() File "/gpfs/fs1/sfw/cibem_galaxy/app/galaxy/tools/data_source/upload.py", line 407, in __main__ for line in open( sys.argv[3], 'r' ): IOError: [Errno 13] Permission denied: '/scratch/XXXXXX/XXXXXX/galaxy/tmp/tmpSlG2pC' The permissions on the referenced file: -rw-------+ 1 rlemley rlemley 317 Mar 7 13:42 tmpSlG2pC As the Galaxy daemons run as XXXXXX_galaxy, it does not have permission to read the file. I tried modifying external_chown_script.py so that XXXXXX_galaxy has permission to read that file, but that caused problems elsewhere so I backed out the changes. Any suggestions? Many thanks, -Rob Rob Lemley Systems Administrator - CIRC University of Rochester

3 5