- galaxy-user - lists.galaxyproject.org

Re: [galaxy-user] downloading huge data sets from history using wget
by Florent Angly 31 May '10

31 May '10

Hi Peter, Please use 'reply all' so that everyone on the mailing list can participe in the discussion. I did not publish my history, so that's probably not what causes problems for you. If you can click on the 'save' icon and it starts the download successfully, then you ought to be able to copy the download link and use it in wget and have it working. What happens when you click on 'save'? Does it start the download? Florent On 31/05/10 09:57, pis(a)duke.edu wrote: > Hi Florent, > > Do you think that I need to publish it first as a history and then try > it again? > I suspect that may be the reason for the strange behavior. > > I will let you know when I get it to work > > Thank you very much for your help > > Have a nice day > Peter > > > > Zitat von Florent Angly <florent.angly(a)gmail.com>: > >> Hi Peter, >> >> See an example below: >>> $ wget >>> http://main.g2.bx.psu.edu/datasets/59a2a6ec00c47fc4/display?to_ext=fasta >>> >>> --2010-05-31 09:45:24-- >>> http://main.g2.bx.psu.edu/datasets/59a2a6ec00c47fc4/display?to_ext=fasta >>> >>> Resolving main.g2.bx.psu.edu... 128.118.201.93 >>> Connecting to main.g2.bx.psu.edu|128.118.201.93|:80... connected. >>> HTTP request sent, awaiting response... 200 OK >>> Length: 1177056689 (1.1G) [text/plain] >>> Saving to: `display?to_ext=fasta' >>> 0% [ ] 224,112 243K/s >> The download link was copied from the "save" icon. >> >> When I try with your link, I get: >>> $ wget >>> http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan… >>> >>> --2010-05-31 09:49:10-- >>> http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan… >>> >>> Resolving main.g2.bx.psu.edu... 128.118.201.93 >>> Connecting to main.g2.bx.psu.edu|128.118.201.93|:80... connected. >>> HTTP request sent, awaiting response... 416 Request Range Not >>> Satisfiable >>> >>> The file is already fully retrieved; nothing to do. >> The "request range not satisfiable" makes me think that your download >> link is not valid for some reason. >> >> Florent >> >> >> On 30/05/10 23:43, pis(a)duke.edu wrote: >>> Dear Florent Angly, >>> >>> Thank you very much for your response. I have actually tried to do >>> that but it >>> still does not work. When I choose "copy link location" in firefox >>> (in my >>> version no save link location appears" I get an URL with a strange >>> data file >>> name such as >>> http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan…. >>> This will not work with wget ("wget: No match"). I does not work >>> either when I >>> replace the data file name with the name of the file that apper when >>> I want to >>> download the file using the disl ikon. I would be happy for further >>> advice on >>> that since it has apparently worked for you. >>> >>> Thank you very much for your help >>> Peter >>> >>> Zitat von Florent Angly <florent.angly(a)gmail.com>: >>> >>>> Hi Peter, >>>> It's pretty easy. In the Galaxy interface, use the "save" icon >>>> represented as a floppy disk. Instead of clicking on the icon, get >>>> the URL for it (in Firefox: right click > save link location). Then >>>> simply copy this URL in your terminal for wget to use. >>>> Florent >>>> >>>> On 30/05/10 07:12, pis(a)duke.edu wrote: >>>>> Dear Galaxy team, >>>>> >>>>> I am doing some standard clipping and trimming using galaxy and >>>>> would be happy >>>>> to download the generated files using a unix terminal and wget. Is >>>>> it a way to >>>>> figure it out the exact link of the data file to wget it easily? I >>>>> am talking >>>>> about file sizes over 5 Gb that are really time consuming and >>>>> error prone to >>>>> download using a web browser. >>>>> >>>>> Thank you very much for developing this excellent tool >>>>> Peter >>>>> >>>>> _______________________________________________ >>>>> galaxy-user mailing list >>>>> galaxy-user(a)lists.bx.psu.edu >>>>> http://lists.bx.psu.edu/listinfo/galaxy-user >>>>> >>>> >>>> _______________________________________________ >>>> galaxy-user mailing list >>>> galaxy-user(a)lists.bx.psu.edu >>>> http://lists.bx.psu.edu/listinfo/galaxy-user >>>> >>> >>> >>> >> >> > > >

1 0

Re: [galaxy-user] downloading huge data sets from history using wget
by Florent Angly 31 May '10

31 May '10

Hi Peter, See an example below: > $ wget > http://main.g2.bx.psu.edu/datasets/59a2a6ec00c47fc4/display?to_ext=fasta > --2010-05-31 09:45:24-- > http://main.g2.bx.psu.edu/datasets/59a2a6ec00c47fc4/display?to_ext=fasta > Resolving main.g2.bx.psu.edu... 128.118.201.93 > Connecting to main.g2.bx.psu.edu|128.118.201.93|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: 1177056689 (1.1G) [text/plain] > Saving to: `display?to_ext=fasta' > 0% [ ] 224,112 243K/s The download link was copied from the "save" icon. When I try with your link, I get: > $ wget > http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan… > --2010-05-31 09:49:10-- > http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan… > Resolving main.g2.bx.psu.edu... 128.118.201.93 > Connecting to main.g2.bx.psu.edu|128.118.201.93|:80... connected. > HTTP request sent, awaiting response... 416 Request Range Not Satisfiable > > The file is already fully retrieved; nothing to do. The "request range not satisfiable" makes me think that your download link is not valid for some reason. Florent On 30/05/10 23:43, pis(a)duke.edu wrote: > Dear Florent Angly, > > Thank you very much for your response. I have actually tried to do > that but it > still does not work. When I choose "copy link location" in firefox > (in my > version no save link location appears" I get an URL with a strange > data file > name such as > http://main.g2.bx.psu.edu/datasets/c3a8db0a339f7a43/display?to_ext=fastqsan…. > > This will not work with wget ("wget: No match"). I does not work > either when I > replace the data file name with the name of the file that apper when I > want to > download the file using the disl ikon. I would be happy for further > advice on > that since it has apparently worked for you. > > Thank you very much for your help > Peter > > Zitat von Florent Angly <florent.angly(a)gmail.com>: > >> Hi Peter, >> It's pretty easy. In the Galaxy interface, use the "save" icon >> represented as a floppy disk. Instead of clicking on the icon, get >> the URL for it (in Firefox: right click > save link location). Then >> simply copy this URL in your terminal for wget to use. >> Florent >> >> On 30/05/10 07:12, pis(a)duke.edu wrote: >>> Dear Galaxy team, >>> >>> I am doing some standard clipping and trimming using galaxy and >>> would be happy >>> to download the generated files using a unix terminal and wget. Is >>> it a way to >>> figure it out the exact link of the data file to wget it easily? I >>> am talking >>> about file sizes over 5 Gb that are really time consuming and error >>> prone to >>> download using a web browser. >>> >>> Thank you very much for developing this excellent tool >>> Peter >>> >>> _______________________________________________ >>> galaxy-user mailing list >>> galaxy-user(a)lists.bx.psu.edu >>> http://lists.bx.psu.edu/listinfo/galaxy-user >>> >> >> _______________________________________________ >> galaxy-user mailing list >> galaxy-user(a)lists.bx.psu.edu >> http://lists.bx.psu.edu/listinfo/galaxy-user >> > > >

1 0

downloading huge data sets from history using wget
by pis＠duke.edu 30 May '10

30 May '10

Dear Galaxy team, I am doing some standard clipping and trimming using galaxy and would be happy to download the generated files using a unix terminal and wget. Is it a way to figure it out the exact link of the data file to wget it easily? I am talking about file sizes over 5 Gb that are really time consuming and error prone to download using a web browser. Thank you very much for developing this excellent tool Peter

2 1

nmslss Lee Sze Sing is out of the office.
by Lee.Sze.Sing＠nccs.com.sg 28 May '10

28 May '10

I will be out of the office starting 28/05/2010 and will not return until 14/06/2010. I will respond to your message when I return.

1 0

Problems with Galaxy kindly reply urgently
by Amit Pande 28 May '10

28 May '10

Dear Galaxy, We have installed Galaxy on one of our Institute's server http://totoro:8080/ but the problem is when ever some file is uploaded for example in the bed format to retrieve sequences from fetch sequences tool the message that gets displayed is :"Sequences not available for the specific build". Kindly help. warm regards, Amit.

1 0

Problems with Galaxy kindly reply urgently
by Amit Pande 28 May '10

28 May '10

Dear Galaxy, We have installed Galaxy on one of our Institute's server http://totoro:8080/ but the problem is when ever some file is uploaded for example in the bed format to retrieve sequences from fetch sequences tool the message that gets displayed is :"Sequences not available for the specific build". Kindly help. warm regards, Amit.

1 0

installing galaxy
by pande 25 May '10

25 May '10

Dear Galaxy, If I download galaxy on the server of my Institute , will I be in a position to extract the sequences of my interest ? and use the tools as I would do them on your server....I have the ENCODE data and it is difficult to upload these files from the internet because of their enormous size. Kindly help. regards, Amit.

1 0

Issue with saving 'manipulate fastq' in workflow; and request for advice dealing with barcoded 454 data
by Pip Griffin 25 May '10

25 May '10

Hi, I'm a new user, learning how to use Galaxy while I wait for my 454 results. So I'm not actually playing with any data yet but I'm trying to set up a draft workflow as practice. Two issues: Issue 1. I am having trouble with the 'manipulate fastq' command. Without this, my workflow saves quickly and seems fine, but when I include even a (seemingly simple) 'manipulate fastq' step, it tries to save for many minutes, unsuccessfully, until I get sick of it and close the window. Issue 2. Well this isn't really an issue, just a request for advice! My dataset will be a barcoded amplicon library, containing 8 different gene regions (which I can recognise from the amplicon-specific primer sequences) amplified in 64 different individuals (which I can recognise by an individual-specific barcode sequence). I thought I'd set up a workflow with the following steps: 1) convert to FASTQ format. 2) grooming, filtering to remove short reads etc. 3) 'manipulate FASTQ' to match all sequences containing one of the eight reverse primer sequences, and reverse-complement them. 4) FASTQ--tabular format conversion. 5) eight separate 'select' steps to select sequences with a match to either the forward primer or the reverse-complemented reverse primer of the desired gene region. My question is: does this seem sensible? Is there a more efficient way to do this that I haven't discovered yet? I was thinking I'd then set up another workflow to label barcoded individuals, for I could use each of the eight gene 'output files' in turn as input. Thanks so much for this service! The screencasts are especially great. Pip Griffin University of Melbourne, Australia

1 0

Fwd: To get the gene name from BED file
by Anton Nekrutenko 24 May '10

24 May '10

Begin forwarded message: > From: Shibu John <shibujohn82(a)gmail.com> > Date: May 24, 2010 8:25:35 AM EDT > To: Anton Nekrutenko <anton(a)bx.psu.edu> > Subject: To get the gene name from BED file > > Hi, > How can I get the gene names for the whole exome locations ( hg18).. > > ******************** > chr1 58932 59892 > chr1 357510 358470 > chr1 610947 611907 > chr1 851160 851280 > chr1 855395 855608 > chr1 856246 856366 > chr1 860956 861196 > ******************** > > I used the "Profile Annotations for a set of genomic intervals".. > But I am not getting proper output... > > I just want the at least gene name in the fourth column for each > location..like this > ************************** > chr1 58932 59892 OR4F5:ccds|CCDS30547.1 > chr1 357510 358470 OR4F16:ccds|CCDS41220.1 > chr1 610947 611907 OR4F16:ccds|CCDS41221.1 > chr1 851160 851280 SAMD11:ccds|CCDS2.2 > chr1 855395 855608 SAMD11:ccds|CCDS2.2 > chr1 856246 856366 SAMD11:ccds|CCDS2.2 > chr1 860956 861196 SAMD11:ccds|CCDS2.2 > ****************** > > Can you help me to get this information from galaxy? > > Thanks, > Shibu > > On Thu, May 20, 2010 at 2:52 PM, Anton Nekrutenko <anton(a)bx.psu.edu> > wrote: > Shibu: > > We experienced disk problems overnight. Galaxy is back to normal. > > Tx, > > anton > galaxy team > > Anton Nekrutenko http://nekrut.bx.psu.edu http://usegalaxy.org

2 3

To get the gene name from BED file
by Shibu John 24 May '10

24 May '10

Hi, How can I get the gene names for the whole exome locations ( hg18).. ******************** chr1 58932 59892 chr1 357510 358470 chr1 610947 611907 chr1 851160 851280 chr1 855395 855608 chr1 856246 856366 chr1 860956 861196 ******************** I used the "Profile Annotations<http://main.g2.bx.psu.edu/tool_runner?tool_id=Annotation_Profiler_0>for a set of genomic intervals".. But I am not getting proper output... I just want the at least gene name in the fourth column for each location..like this ************************** chr1 58932 59892 OR4F5:ccds|CCDS30547.1 chr1 357510 358470 OR4F16:ccds|CCDS41220.1 chr1 610947 611907 OR4F16:ccds|CCDS41221.1 chr1 851160 851280 SAMD11:ccds|CCDS2.2 chr1 855395 855608 SAMD11:ccds|CCDS2.2 chr1 856246 856366 SAMD11:ccds|CCDS2.2 chr1 860956 861196 SAMD11:ccds|CCDS2.2 ****************** Can you help me to get this information from galaxy? Thanks, Shibu On Thu, May 20, 2010 at 2:52 PM, Anton Nekrutenko <anton(a)bx.psu.edu> wrote: > Shibu: > > We experienced disk problems overnight. Galaxy is back to normal. > > Tx, > > anton > galaxy team > > >

1 0