July 2015 - galaxy-dev - lists.galaxyproject.org

Possible bug on "Create a collection of paired datasets"
by Ata Roudgar 20 Jul '15

20 Jul '15

Hi, I am a computer cluster admin and I am trying to install galaxy on our cluster. I used galaxy-release_15.05 and because we need to serve Galaxy at a sub directory, e.g. irida-galaxy I modified the options prefix and cookie_path in the galaxy configuration file as follow: prefix = /irida-galaxy filter-with = proxy-prefix cookie_path = /irida-galaxy We use apache as an interface with the Following configuration: <Proxy http://localhost:9010> Order deny,allow Allow from all </Proxy> RewriteRule ^/irida-galaxy$ /irida-galaxy/ [R] RewriteRule ^/irida-galaxy/static/style/(.*) /home/irida/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/irida-galaxy/static/scripts/(.*) /home/irida/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/irida-galaxy/static/(.*) /home/irida/galaxy-dist/static/$1 [L] #RewriteRule ^/irida-galaxy/favicon.ico /home/irida/galaxy-dist/static/favicon.ico [L] RewriteRule ^/irida-galaxy/robots.txt /home/irida/galaxy-dist/static/robots.txt [L] RewriteRule ^/irida-galaxy(.*) http://localhost:9010$1 [P] Everything works fairly good. Now I want to create a Dataset Pairs. first I uploads some fastaq files, e.g., a_1.fastaq a_2.fastaq b_1.fastaq b_2.fastaq c_1.fastaq c_2.fastaq then I choose: "Build List of Dataset Pairs" (See Fig-1 attached) There is a window appears on my screen with the title: Create a collection of paired datasets. Now I am able to choose a name and save the paired collection. However, once I click on "Create list" buttons I receive a error message on top of the windows: "An error occurred While creating this collection: Not found See Fig-2 attached. To be able to search the cause of error I turn on the "Browser Consul" on my web browser and notice that the error is related to the following web form post: https://newrobson.westgrid.ca:8444/api/histories/4936629215c25989/contents/… Which makes sense because such address do not exist. The address should be https://newrobson.westgrid.ca:8444/irida-galaxy/api/histories/4936629215c25… So the prefix (irida-galaxy) is missing before /api/histories...../dataset_collection Is it a bug or am I missing something? Please note that such error does not appear when I do not use prefix in my galaxy configration. Thanks for your time, Cheers, Ata ---- Ata Roudgar Research Computing WestGrid Site IT Services Simon Fraser University Burnaby, British Columbia Canada V5A 1S6 phone: 778 782-8860 fax: 778 782-4242

2 2

Troubleshooting
by Lionel Mavoungou 20 Jul '15

20 Jul '15

Hello, I have a problem when I try to align en FATSQ file for a ChIP-seq. I have done a FASTQ grooming because it has been performed on Illumina. No my file is a .FASTSANGER. However my file is not recognized. I see this : No fastqsanger, fastqillumina or fastqsolexa dataset collection available. How could I fix it. It seems that my fastq file as well as my fastqsanger are not recognized. Thanks a lot, Lionel

2 2

Tophat run stops
by McCully, Dwayne (NIH/NIAMS) [C] 20 Jul '15

20 Jul '15

When running a Tophat job for over a hour, I get the following message and the job stops. Would anyone know why and is there a way to stop Galaxy from cleaning up the job working directory so that I can check the error log files? Dwayne galaxy.jobs.runners.local DEBUG 2015-07-14 16:54:10,478 execution finished: /data/galaxy/galaxy_prod/database/job_working_directory/000/26/galaxy_26.sh galaxy.jobs.output_checker INFO 2015-07-14 16:54:10,595 Job 26: Log: tool progress galaxy.jobs.output_checker INFO 2015-07-14 16:54:10,595 Job 26: Log: tool progress galaxy.datatypes.metadata DEBUG 2015-07-14 16:54:10,675 setting metadata externally failed for HistoryDatasetAssociation 32: [Errno 2] No such file or directory galaxy.jobs.runners.local ERROR 2015-07-14 16:54:10,754 Job wrapper finish method failed Traceback (most recent call last): File "/data/galaxy/galaxy_prod/lib/galaxy/jobs/runners/local.py", line 129, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/data/galaxy/galaxy_prod/lib/galaxy/jobs/__init__.py", line 1126, in finish dataset.datatype.set_meta( dataset, overwrite=False ) # call datatype.set_meta directly for the initial set_meta call during dataset creation File "/data/galaxy/galaxy_prod/lib/galaxy/datatypes/binary.py", line 268, in set_meta proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory

1 0

Re: [galaxy-dev] Error sorting or uploading Bam File
by julie dubois 20 Jul '15

20 Jul '15

Hi, Thanks for your answers. I understand that permit to load or create "unsorted bam" (or sorted by name) can create bug with lot of tools and it's difficult to restrict this manipulation just for few tools. But I don't understand one thing. Why can I download from the toolshed a tool which permit me to sort bam file by name (from devteam) while it's not supported by galaxy ?! Other question. Anyone can help me to write a new entry for an ‘unsorted bam’ (or namesorted) file type in the file which describe the format file uploaded ? Or do you think that I can run this sorting during the processing of the tool which require this type of sorting : so run the tool of interest and internally to the process run the sorting by name before treating the bam file. In this way do you think that Galaxy will not create an index during the process ? Thanks. Julie 2015-07-17 21:51 GMT+02:00 Frederik Coppens <frcop(a)psb.vib-ugent.be>: > We also encountered this issue. > As I understood it: if Galaxy gets a BAM file (either through upload or > generated by a tool), it will generate metadata and indexing is part of > that. This only works if the BAM file is coordinate sorted, otherwise it > fails (although there is nothing wrong with the BAM file). > > As a lot of tools (and visualisation) require sorted BAM files, this avoids > a lot of problems for users (but does create this issue). > > An ‘unsorted bam’ (or namesorted) file type could be a solution although > this would cause the illogic situation where a ‘bam' is hierarchically not > above ‘unsorted bam’ as is the case for e.g. the types of fastq. I guess > changing what is now ‘bam’ to ‘sorted bam’ will cause problems for a lot of > users... > > Frederik > > On 17 Jul 2015, at 21:29, Alexander Vowinkel <vowinkel.alexander(a)gmail.com> > wrote: > > Indeed, this would be the right approach. > But so far the current strategy is quite imputed also in tools. > E.g. BWA from devteam automatically sorts the bam afterwards - wanted or > not. > > 2015-07-17 11:59 GMT-05:00 Ryan G <ngsbioinformatics(a)gmail.com>: >> >> I would think the correct approach would be to allow any bam file and if a >> tool requires a coordinate sorted bam and one is not provided it would make >> sense for the tool to fail with an error. >> >> Does galaxy itself require a coordinate sorted bam or this this so the >> majority of tools can function properly? >> >> Sent from my iPhone >> >> > On Jul 17, 2015, at 11:08 AM, John Chilton <jmchilton(a)gmail.com> wrote: >> > >> > I'm not the right person to respond to this but since no one else has >> > I will explain my limited (probably incorrect) understanding of the >> > problem and what needs to be done. I believe Galaxy assumes all bam >> > files are coordinated sorted by default - so Galaxy's bam datatype >> > would be better thought of as a coordinate-sorted bam. Certain people >> > do not want to allow non-coordinate sorted bam files because these >> > would be allowed by Galaxy as valid as inputs to tools that expect >> > sorted bams - and things would break. For the same reason I imagine >> > these people would argue the bai file needs to be generated because >> > certain tools will depend on its existence. >> > >> > I believe the proposed solution is to add an unsorted bam datatype to >> > Galaxy. I don't know if there is any work in progress on this but it >> > seems to be needed by many researchers. >> > >> > Hope this clarifies things somewhat, >> > -John >> > >> > >> >> On Mon, Jun 29, 2015 at 11:06 AM, julie dubois <dubjulie(a)gmail.com> >> >> wrote: >> >> Hi all, >> >> >> >> My galaxy is the last update and I work with samtools 1.2. >> >> I have a bug with the sorting and the upload of bam file. >> >> This bug appears only when I try to upload bam sorted by name or when >> >> I try to sort by name. >> >> When I work with bam sorted by coordinate, there is no problem. >> >> >> >> So, when I try to sort a bam file by name, this error appears under >> >> the "bug icon" : >> >> >> >> Tool execution generated the following error message: >> >> [bam_sort_core] merging from 20 files... >> >> *** Error in `python': double free or corruption (!prev): >> >> 0x0000000002897990 *** >> >> Aborted (core dumped) >> >> >> >> The tool produced the following additional output: >> >> >> >> [E::hts_idx_push] chromosome blocks not continuous >> >> Galaxy attempted to build the BAM index with samtools 1.0+ but failed: >> >> [Errno 2] No such file or directory: >> >> >> >> '/data/galaxy-dist/database/job_working_directory/024/24109/__dataset_48619_metadata_temp_file_MqhjBN.bai' >> >> >> >> And the same type of error appears when I try to upload bam sorted by >> >> name. >> >> >> >> >> >> What I understand is that the bam can be sorted properly (The command >> >> line to sort is ok in a terminal with no bug and I can follow this >> >> step in the job_working_directory during the run) but Galaxy seems to >> >> fail to create a bai. >> >> >> >> For our analysis pipeline, I think the bai is not crucial, is it >> >> possible to avoid this creation of bai during uploading and sorting of >> >> bam file by name ? >> >> >> >> Is there another problem than this creation of bai ? >> >> >> >> Thanks. >> >> >> >> Julie >> >> ___________________________________________________________ >> >> Please keep all replies on the list by using "reply all" >> >> in your mail client. To manage your subscriptions to this >> >> and other Galaxy lists, please use the interface at: >> >> https://lists.galaxyproject.org/ >> >> >> >> To search Galaxy mailing lists use the unified search at: >> >> http://galaxyproject.org/search/mailinglists/ >> > ___________________________________________________________ >> > Please keep all replies on the list by using "reply all" >> > in your mail client. To manage your subscriptions to this >> > and other Galaxy lists, please use the interface at: >> > https://lists.galaxyproject.org/ >> > >> > To search Galaxy mailing lists use the unified search at: >> > http://galaxyproject.org/search/mailinglists/ >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > > > ================================================================== > Frederik Coppens, Ph.D. > Project Leader Applied Bioinformatics & Biostatistics > IT manager > > Tel: 32 (0)9 3313692 Fax:32 (0)9 3313809 > VIB DEPARTMENT OF PLANT SYSTEMS BIOLOGY, UGent > Technologiepark 927, 9052 Gent, BELGIUM > > frcop(a)psb.ugent.be http://www.psb.ugent.be/abb > ================================================================== >

1 0

./run.sh segfault
by Martin Vickers 20 Jul '15

20 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I have a weird issue that's just cropped up. After a new install of galaxy (checked out on Monday from github) on a ubuntu vm, using postgres rather than sqlite as well as a few other production recommendations, I started playing around with the Data Libraries functionality. I linked a bunch of fastq.gz files into galaxy (around 150 in total) and everything was working fine. I went home and the next day, it was down. I tried to start it up as usual (using an init.d script), it worked for less than a minute and then disappeared again. So I tried running it as the galaxy user using ./run.sh and I get a seg fault; Starting server in PID 23173. serving on http://144.124.110.39:8080 Segmentation fault Tried again with strace Starting server in PID 23552. serving on http://144.124.110.39:8080 [{WIFSIGNALED(s) && WTERMSIG(s) == SIGKILL}], 0, NULL) = 23552 - --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=23552, si_status=SIGKILL, si_utime=1590, si_stime=1930} --- rt_sigreturn() = 23552 write(2, "Killed\n", 7Killed ) = 7 read(10, "", 8192) = 0 exit_group(137) = ? +++ exited with 137 +++ I can't see anything odd in the log file and I've turned debugging on in galaxy.ini. I'm at a bit of a loss. Does anyone know what might be causing it? Cheers, - -- Dr. Martin Vickers Data Manager/HPC Systems Administrator Institute of Biological, Environmental and Rural Sciences IBERS New Building Aberystwyth University w: http://www.martin-vickers.co.uk/ e: mjv08(a)aber.ac.uk t: 01970 62 2807 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (GNU/Linux) iQEcBAEBAgAGBQJVp80YAAoJEHa0a8GkKQgIGlIH/1VfAPbs/5ApDBdyoOV5qf1y oCOv93IojARyfI0ksSjF8NRNzw5fNp1R8AzZzomaR3SOUkBuZutre600sy0azTZw E6gjxtMuvaMyEsOTXtToVarVJT0wTG8+5DJRIYLxtYZm7kvbZK0WuzrN2zDT6663 Rnm7zI/zBpTAyp6uXwgmz0x5gpH6KFwRcEHEbU3JWy6nj1zithJShwYPlBuhT5IB OaPwOKflcZpZ8NBTEGsh038JrkU+eE50a9aEjQ2m/DpfM/TN9ujgEFm1dyy/iQS7 ewwQUpWJDkA/u0ZX602dsNdV2LvGuKVVMEHiQ25zaUQZ/iGTwKBQsFM2LlDybgA= =jzYG -----END PGP SIGNATURE-----

3 4

yaml import problem on startup
by Poole, Richard 20 Jul '15

20 Jul '15

Hi all, With the most recent update of Galaxy and also OS X 10.10.4 I am seeing this error message when first starting up the server: /Users/galaxy/galaxy-dist/lib/galaxy/__init__.py:63: UserWarning: Module yaml was already imported from /Library/Python/2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/__init__.pyc, but /Users/galaxy/galaxy-dist/eggs/PyYAML-3.10-py2.7-macosx-10.6-intel-ucs2.egg is being added to sys.path self.check_version_conflict() This then is preventing certain tools from running correctly (such as samtools_filter (jjohnson) and they give the same error message when they fail: /Users/galaxy/galaxy-dist/lib/galaxy/__init__.py:63: UserWarning: Module yaml was already imported from /Library/Python/2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/__init__.pyc, but /Users/galaxy/galaxy-dist/eggs/PyYAML-3.10-py2.7-macosx-10.6-intel-ucs2.egg is being added to sys.path self.check_version_conflict() Any thoughts? Thanks, Richard Richard J Poole PhD Wellcome Trust Fellow Department of Cell and Developmental Biology University College London 21 University Street, London WC1E 6DE Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133)

2 4

Password-Reset
by Matthias Enders 20 Jul '15

20 Jul '15

Dear Galaxy Community, as we tried to establish the mail functionalities for our Galaxy instance, we managed to send email upon the "I forgot my password" user request. In the Email a Link is provided. Here two errors arise: 1. In the File "galaxy/lib/galaxy/webapps/galaxy/controllers/user.pyc" HTML Tags for this Email are generated, but the Email is send as plain/text, therefore the HTML Tags, as well as the (doubled) Link are visible for the user. 2. In our case, the provided link takes us to the normal user login page, where the user has no option of resetting his/her password. Can you provide me some assistance to tackle the problems? Kind Regards, Matthias Enders

1 0

Error sorting or uploading Bam File
by julie dubois 18 Jul '15

18 Jul '15

Hi all, My galaxy is the last update and I work with samtools 1.2. I have a bug with the sorting and the upload of bam file. This bug appears only when I try to upload bam sorted by name or when I try to sort by name. When I work with bam sorted by coordinate, there is no problem. So, when I try to sort a bam file by name, this error appears under the "bug icon" : Tool execution generated the following error message: [bam_sort_core] merging from 20 files... *** Error in `python': double free or corruption (!prev): 0x0000000002897990 *** Aborted (core dumped) The tool produced the following additional output: [E::hts_idx_push] chromosome blocks not continuous Galaxy attempted to build the BAM index with samtools 1.0+ but failed: [Errno 2] No such file or directory: '/data/galaxy-dist/database/job_working_directory/024/24109/__dataset_48619_metadata_temp_file_MqhjBN.bai' And the same type of error appears when I try to upload bam sorted by name. What I understand is that the bam can be sorted properly (The command line to sort is ok in a terminal with no bug and I can follow this step in the job_working_directory during the run) but Galaxy seems to fail to create a bai. For our analysis pipeline, I think the bai is not crucial, is it possible to avoid this creation of bai during uploading and sorting of bam file by name ? Is there another problem than this creation of bai ? Thanks. Julie

5 4

unsubscribe
by Ricardo Perez 17 Jul '15

17 Jul '15

On Jul 17, 2015 11:00 AM, <galaxy-dev-request(a)lists.galaxyproject.org> wrote: > Send galaxy-dev mailing list submissions to > galaxy-dev(a)lists.galaxyproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.galaxyproject.org/listinfo/galaxy-dev > or, via email, send a message with subject or body 'help' to > galaxy-dev-request(a)lists.galaxyproject.org > > You can reach the person managing the list at > galaxy-dev-owner(a)lists.galaxyproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of galaxy-dev digest..." > > > HEY! This is important! If you reply to a thread in a digest, please > 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to > the original subject for the thread. > 2. Strip out everything else in the digest that is not part of the thread > you are responding to. > > Why? > 1. This will keep the subject meaningful. People will have some idea from > the subject line if they should read it or not. > 2. Not doing this greatly increases the number of emails that match search > queries, but that aren't actually informative. > > Today's Topics: > > 1. Re: yaml import problem on startup (Marius van den Beek) > 2. Re: yaml import problem on startup (Marius van den Beek) > 3. Re: yaml import problem on startup (Poole, Richard) > 4. Re: Data Libraries: Unable to Link to Files When Paths > Contain Spaces (John Chilton) > 5. Re: Error sorting or uploading Bam File (John Chilton) > 6. Re: CloudMan auto-scaling - Use spot instance for worker > node? (Alexander Vowinkel) > 7. Re: Download data by data manager with non generic ftp-proxy > (John Chilton) > 8. Re: CloudMan auto-scaling - Use spot instance for worker > node? (Enis Afgan) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 17 Jul 2015 12:23:06 +0200 > From: Marius van den Beek <m.vandenbeek(a)gmail.com> > To: "Poole, Richard" <r.poole(a)ucl.ac.uk> > Cc: "galaxy-dev(a)lists.galaxyproject.org" > <galaxy-dev(a)lists.galaxyproject.org> > Subject: Re: [galaxy-dev] yaml import problem on startup > Message-ID: > < > CACJqqiQo3uti7-doVVg_1L7-rzkwSxh9ccz2NSfJFUrg+kGsOQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Richard, > > Well, I do have the same problem now (running bamtools), after applying the > patch that should fix > metadata issue we discussed yesterday. > I'll dig into this. > > Marius > > On 17 July 2015 at 11:37, Poole, Richard <r.poole(a)ucl.ac.uk> wrote: > > > Hi all, > > > > With the most recent update of Galaxy and also OS X 10.10.4 I am seeing > > this error message when first starting up the server: > > > > /Users/galaxy/galaxy-dist/lib/galaxy/__init__.py:63: UserWarning: > > Module yaml was already imported from > > > /Library/Python/2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/__init__.pyc, > > but > > > /Users/galaxy/galaxy-dist/eggs/PyYAML-3.10-py2.7-macosx-10.6-intel-ucs2.egg > > is being added to sys.path > > self.check_version_conflict() > > > > This then is preventing certain tools from running correctly (such as > > samtools_filter (jjohnson) and they give the same error message when they > > fail: > > > > /Users/galaxy/galaxy-dist/lib/galaxy/__init__.py:63: UserWarning: > Module yaml was already imported from > /Library/Python/2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/__init__.pyc, > but > /Users/galaxy/galaxy-dist/eggs/PyYAML-3.10-py2.7-macosx-10.6-intel-ucs2.egg > is being added to sys.path > > self.check_version_conflict() > > > > Any thoughts? > > > > Thanks, > > Richard > > > > *Richard J Poole PhD* > > Wellcome Trust Fellow > > Department of Cell and Developmental Biology > > University College London > > 21 University Street, London WC1E 6DE > > Office (518 Rockefeller): +44 20 7679 6577 (int. 46577) > > Lab (529 Rockefeller): +44 20 7679 6133 (int. 46133) > > > > > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > >

1 0

Re: [galaxy-dev] CloudMan auto-scaling - Use spot instance for worker node?
by Alexander Vowinkel 17 Jul '15

17 Jul '15

Hi, I am referring to this: http://dev.list.galaxyproject.org/CloudMan-auto-scaling-Use-spot-instance-f… I would give an upvote for spot instances for auto-scaling. Have there been more ideas about this since? Best, Alexander

2 1