Local Instance: BWA, Sam-to-Bam, file upload, filter Sam
by Moritz Juchler
Hello Ladies and Gentlemen,
I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I
have to setup Galaxy to find SNP's in genomes from hcc patients. I have a
64-bit openSuse 11.3 server on which I installed Galaxy locally, since we
have a) very large files (>30GB per patient) and b) the data is protection
sensitive.
I have to run this pipeline:
> http://www.nature.com/ng/journal/v44/n6/extref/ng.2256-S1.pdf
1. Which version of BWA should I use? The current gives me the EoF error
and the 0.5.9. gives me "missing argument -I"
2. Originally I had this workflow to start with
https://main.g2.bx.psu.edu/u/mj--/w/ngs
, but I at the sam-to-bam conversion I get the "sequences are not currently
available for specified build" error when using "locally cached" and I
can't figure out how to use the reference file hg19.fa without actually
uploading it to Galaxy, because I do not have enough space on the
filesystem where the Galaxy distribution is placed ( /home). The genedata
are all on /genedata. So my question here is: How to use the hg19.fa file
placed on another filesystem then the galaxydist?
3. Thus I searched the web and found this workaround:
https://main.g2.bx.psu.edu/u/mj--/w/sample-workflow-whole-exome-sequencing
which runs fine ONLINE at usegalaxy but produces this error in my local
instance
Dataset generation errors
*Dataset 18: Filter SAM on data 7*
Tool execution generated the following error message:
Traceback (most recent call last):
File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py",
line 148, in <module>
if __name__ == "__main__": main()
File "/home/trr/galaxy-dist/tools/samtools/sam_bitwise_flag_filter.py",
line 137, in main
flags = int( fields[flag_col] )
IndexError: list index out of range
So thats my 3rd question: How to solve this error? I didnt find anything
online.
Best
Moritz
9 years, 6 months
GATK fasta file missing ERROR
by Sridhar Srinivasan
Dear Developers,
I am GATK (beta ) Unified Genotyper module to get the indels and SNP.
I placed the jar file in (/galaxy-dist/tool-data/shared/jars/gatk) and i
link the reference genome in .loc file (gatk_sorted_picard_index.loc). I
could see the Name i mentioned in the .loc file is appearing in the galaxy
page while choosing the Reference genome.
Now while running GATK in galaxy , The command runs for a while and gives
ERROR as i am getting ERROR as *(The fasta file you specified
(/tmp/tmp-gatk-Jjg9Mi) does not exist.)*
Could anybody suggest on the same..
Thanks
Sridhar
9 years, 6 months
Bowtie2 error: Creates a 62 kb file.
by Perez, Ricardo
Dear all,
Today we ran some problems when running bowtie2. When we run bowtie2, the run manages to complete. However, bowtie2 creates a 62kb file.
The error log that we get is the following:
galaxy.jobs DEBUG 2013-07-23 17:09:42,378 (1483) Persisting job destination (destination id: local:///)
galaxy.jobs.handler INFO 2013-07-23 17:09:42,392 (1483) Job dispatched
galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell command for dependency 'bowtie2'
galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency on 'bowtie2', ignoring
galaxy.tools DEBUG 2013-07-23 17:09:42,556 Building dependency shell command for dependency 'samtools'
galaxy.tools WARNING 2013-07-23 17:09:42,556 Failed to resolve dependency on 'samtools', ignoring
galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,626 (1483) executing: bowtie2 --version &> /usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py --num-threads="4" --output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.dat --indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.Chr07.+ChrM.+rDNA_v12" --single-paired=paired --input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.dat --input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.dat -I 0 -X 250 --settings=preSet
galaxy.jobs DEBUG 2013-07-23 17:09:42,696 (1483) Persisting job destination (destination id: local:///)
galaxy.jobs.runners.local DEBUG 2013-07-23 17:09:42,766 execution finished: bowtie2 --version &> /usr/local/galaxy/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_1483; python /usr/local/galaxy/galaxy-dist/tools/sr_mapping/bowtie2_wrapper.py --num-threads="4" --output=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1673.dat --indexes-path="/data/db/fungidb/Ncrassa_OR74A/bowtie2/Ncrassa_wt_Chr01.Chr07.+ChrM.+rDNA_v12" --single-paired=paired --input1=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1668.dat --input2=/usr/local/galaxy/galaxy-dist/database/files/001/dataset_1669.dat -I 0 -X 250 --settings=preSet
galaxy.jobs.runners DEBUG 2013-07-23 17:09:42,850 executing external set_meta script for job 1483: /usr/local/galaxy/galaxy-dist/set_metadata.sh ./database/files /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483 . /usr/local/galaxy/galaxy-dist/universe_wsgi.ini /usr/local/galaxy/galaxy-dist/database/tmp/tmpUiqpPD /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/galaxy.json /usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_in_HistoryDatasetAssociation_1767_wqhEmr,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_kwds_HistoryDatasetAssociation_1767_qVZjpG,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_out_HistoryDatasetAssociation_1767_vIfpRO,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_results_HistoryDatasetAssociation_1767_UNl6OR,,/usr/local/galaxy/galaxy-dist/database/job_working_directory/001/1483/metadata_override_HistoryDatasetAssociation_1767_MZilrL
165.91.11.135 - - [23/Jul/2013:17:09:43 -0500] "GET /api/histories/ca8efa3c47500414 HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.22 (KHTML, like Gecko) Ubuntu Chromium/25.0.1364.160 Chrome/25.0.1364.160 Safari/537.22"
165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /history HTTP/1.1" 200 - "http://galaxy.tamu.edu/tool_runner/index" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36"
165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /api/histories/1ab12f1e464820dd HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36"
165.91.11.231 - - [23/Jul/2013:17:09:43 -0500] "GET /history/get_display_application_links HTTP/1.1" 200 - "http://galaxy.tamu.edu/history" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.71 Safari/537.36"
galaxy.jobs.runners DEBUG 2013-07-23 17:09:44,457 execution of external set_meta for job 1483 finished
galaxy.datatypes.metadata DEBUG 2013-07-23 17:09:44,545 loading metadata from file for: HistoryDatasetAssociation 1767
We currently have the most recent version of bowtie2 and samtools. Anyone know how to fix this?
Thank you for your time,
--Ricardo Perez
9 years, 6 months
workflow: default parameters
by Keilwagen, Jens
Hi guys,
when building workflows we can choose for each parameter whether to 'set at runtime' or to 'set in advance' (when creating or editing the workflow). We are looking for a possiblity to set a default value that is used (only) for the workflow, i.e., we like to specify a value that might be used by most users, but might be altered by expert users. This value might be different from the default value of the corresponding tool. So far, we did not find any possiblity.
Thanks for any help.
Best regards, Jens
9 years, 6 months
Cloudman master cannot connect to volume at start-up
by Ravpreet Setia
* This is what has been reported by the cluster status log:
*
* 20:01:14 - Master starting
* 20:01:16 - Trouble getting volume reference for volume vol-91cfd1cb: EC2ResponseError: 400 Bad RequestInvalidVolume.NotFoundThe volume 'vol-91cfd1cb' does not exist.9179663c-c5ec-4e90-88ff-ecf606b5e9a6
* 20:01:16 - Attempting to connect to a non-existent volume vol-91cfd1cb
* 20:01:16 - Error processing filesystems in existing cluster configuration: 'NoneType' object has no attribute '__getitem__'
* 20:01:16 - ***** Manager failed to start *****
Any suggestions?
9 years, 6 months
RFC: Cheetah template in tool_dependencies.xml
by Björn Grüning
Hi,
I was thinking how to tackle the build failures like the atlas library
on OS-X or the ones from xlib and ease the writing of the shell_command
a little bit.
Dave B did some fancy shell scripting in 'package_emboss_5_0_0' and I
was also sometimes forced to use some bash tricks.
What about to make that more easy and powerful, and as a plus more
consistent with the tool-command tag, and use cheetah for that kind of
work.
Something like that could work easily:
#if ${os.environ}.getVar('X11_LIB_DIR', None):
./configure ... --x-includes=....
#else:
./configure
#end if
Also we could define some reserved variables like $__is64__ and cheetah
evaluate that. $INSTALL_DIR and $REPOSITORY_DIR can also be a cheetah
variable and we do not need to replace it later manually.
What do you think any comments?
If no one has any objections I can try to implement an <action
type="cheetah_command">.
Thanks,
Björn
9 years, 6 months
HTML output with external Javascript
by Sinthong, Phanwadee
Hi galaxy-dev,
I have a tool that generates a HTML output file which uses some javascript library. I cannot view the output file with the eye button in Galaxy, but it still works fine from the HTML file opened out of the file directory.
I changed the sanitize_all_html = False and restarted the galaxy server as some earlier post suggested but still couldn't view the output file. What else can I do to execute Javascript in the Galaxy browser?
Thanks,
Gift
9 years, 6 months
Two questions
by Gromobir
Hello list,
I have two questions regarding galaxy:
1.) Is it possible to configure the amount of lines listed in the tool
standard output (stdout)? On my system
only the last five lines of tool output are shown there.
2.) Currently I get the message "This dataset is large and only the
first megabyte is shown below" every time
I try to display a file, which is larger than one megabyte. Is it
possible to change this size, such that I won't get this
message anymore for larger files?
Thank you very much in advance!
Cheers,
Gromobir
9 years, 6 months
Re: [galaxy-dev] Problem in uploading file
by Himanshu Gupta
Hello,
It seems you are right. It has nothing to do with the SQLAlchemy. I checked
the Galaxy log and didn't find any other error log. After upgrading the
database to MySQL when I uploaded the first input file, job dispatched to
local runner and everything was fine. But when I tried to upload another
file Galaxy didn't even dispatched the Job. I am pasting below the snapshot
of the Galaxy log for your reference.
galaxy.jobs DEBUG 2013-07-23 09:36:15,579 job 1 ended
galaxy.datatypes.metadata DEBUG 2013-07-23 09:36:15,579 Cleaning up
external metadata files
10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET /api/users/current
HTTP/1.1" 200 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:36:15 +0600] "GET
/api/histories/f2db41e1fa331b3e/contents?ids=ebfb8f50c6abde6d HTTP/1.1" 200
- "http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:36:33 +0600] "GET
/tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:36:38 +0600] "POST
/tool_runner/upload_async_create HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:36:38 +0600] "GET
/tool_runner/upload_async_message HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:01 +0600] "GET
/tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:08 +0600] "POST
/tool_runner/upload_async_create HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:08 +0600] "GET
/tool_runner/upload_async_message HTTP/1.1" 200 - "http://10.180.8.54:6080/"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET /history HTTP/1.1" 200 -
"http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4
(KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:53 +0600] "GET
/history/get_display_application_links HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET / HTTP/1.1" 200 - "-"
"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4 (KHTML, like Gecko)
Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /root/welcome HTTP/1.1"
302 - "http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET /history HTTP/1.1" 200 -
"http://10.180.8.54:6080/" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.4
(KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET
/history/get_display_application_links HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:37:57 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:02 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:06 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:10 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:14 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:18 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:22 +0600] "GET
/api/histories/f2db41e1fa331b3e HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
10.180.4.152 - - [23/Jul/2013:09:38:24 +0600] "POST
/datasets/33b43b4e7093c91f/delete_async HTTP/1.1" 200 - "
http://10.180.8.54:6080/history" "Mozilla/5.0 (X11; Linux i686)
AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4"
Thanks
On Mon, Jul 22, 2013 at 1:05 PM, James Taylor <james(a)jamestaylor.org> wrote:
> The SQLAlchemy warning shouldn't have anything to do with uploads.
> Does Galaxy still start correctly and do you see any other errors on
> the log? What is logged when you try to do the upload?
>
> --
> James Taylor, Assistant Professor, Biology/CS, Emory University
>
>
> On Fri, Jul 19, 2013 at 9:19 AM, Himanshu Gupta <himanshucse10(a)gmail.com>
> wrote:
> > Hello everyone,
> >
> > I had configured galaxy with MySQL database and for some time it was
> working
> > fine. But suddenly it started showing following warning whenever I start
> > galaxy.
> >
> ".../galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs2.egg/sqlalchemy/sql/expression.py:1927:
> > SAWarning: The IN-predicate on "request_type.id" was invoked with an
> empty
> > sequence. This results in a contradiction, which nonetheless can be
> > expensive to evaluate. Consider alternative strategies for improved
> > performance."
> >
> > And now I am not able to upload any file. Please help me to resolve it.
> >
> > Thanks
> >
> > ___________________________________________________________
> > Please keep all replies on the list by using "reply all"
> > in your mail client. To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> > http://lists.bx.psu.edu/
> >
> > To search Galaxy mailing lists use the unified search at:
> > http://galaxyproject.org/search/mailinglists/
>
9 years, 6 months
