4:54 a.m.
Hi galaxy
I am a new user of galaxy. i met a problem and didnot find similar
question in FAQ. I wanted to upload the data from DDBJ DRA dataset to
galaxy through UTL method. The file is around 800M. However after
uploading, the FASTQ file was just around 2M. So I wanted know whether
it is possible to upload a large file to galaxy through URL method? or
I should download the file to my pc and then uploading to galaxy
through FTP method.
Thanks
xiangmimg
9:48 a.m.
You can download DRA files directly by FTP to Galaxy . .
Just paste the FTP address directly in the file box when using upload from
my computer
Best
Simon
On 7 November 2011 04:54, Xiangming Ding <dingxm(a)ucla.edu> wrote:
Hi galaxy
I am a new user of galaxy. i met a problem and didnot find similar
question in FAQ. I wanted to upload the data from DDBJ DRA dataset to
galaxy through UTL method. The file is around 800M. However after
uploading, the FASTQ file was just around 2M. So I wanted know whether it
is possible to upload a large file to galaxy through URL method? or I
should download the file to my pc and then uploading to galaxy through FTP
method.
Thanks
xiangmimg
______________________________**_____________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/**listinfo/galaxy-dev<http://lists.bx.psu.edu/...
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
2:20 p.m.
Hello Xiangmimg,
Data files can be loaded using a URL on the "Get Data => Upload" form.
FTP and HTTP connections are supported. This is briefly described on
that form.
If you are still having issues, there may be a problem with file
compression or the connection. Downloading locally then using Galaxy's
FTP upload function is certainly an option.
http://wiki.g2.bx.psu.edu/Learn/Upload%20via%20FTP
Best,
Jen
Galaxy team
On 11/6/11 8:54 PM, Xiangming Ding wrote:
Hi galaxy
I am a new user of galaxy. i met a problem and didnot find similar
question in FAQ. I wanted to upload the data from DDBJ DRA dataset to
galaxy through UTL method. The file is around 800M. However after
uploading, the FASTQ file was just around 2M. So I wanted know whether
it is possible to upload a large file to galaxy through URL method? or I
should download the file to my pc and then uploading to galaxy through
FTP method.
Thanks
xiangmimg
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
3:50 p.m.
New subject: downloading (and renaming) output files from Galaxy public server
Hi,
I am using the galaxy public server.
Is there a way to access output files (via ftp, perhaps) so I can bulk download them to
my computer? I am over my quota and want to get data off of Galaxy but prefer not to do
this all one at a time.
Similarly, is there a way to access a directory (via unix, ftp, etc) to rename files
quickly while they are on Galaxy, since renaming each output file (i.e. the multiple ones
output from cuffdiff) within galaxy is very inefficient and time consuming.
Thanks.
Rich
10:49 p.m.
New subject: downloading (and renaming) output files from Galaxy public server
Hi Rich,
This is a good question! Maybe people have been asking about this. To
download data with a unix line command method, please try wget or curl,
for example:
unix% wget 'url_for_the_dataset'
or
unix% wget 'url_for_the_history'
To capture the url for a dataset, right click on the disk icon for a
dataset and select "copy link location".
To capture the url for an entire history, select "Options -> Export to
File". The middle panel will display a link.
A downloaded history can be loaded into a local Galaxy instance where
the datasets can be managed (copy/rename) or the histories archived.
Hopefully this helps you and others that are managing larger datasets &
histories,
Best,
Jen
Galaxy team
On 11/7/11 7:50 AM, Richard Mark White wrote:
Hi,
I am using the galaxy public server.
Is there a way to access output files (via ftp, perhaps) so I can bulk
download them to my computer? I am over my quota and want to get data
off of Galaxy but prefer not to do this all one at a time.
Similarly, is there a way to access a directory (via unix, ftp, etc) to
rename files quickly while they are on Galaxy, since renaming each
output file (i.e. the multiple ones output from cuffdiff) within galaxy
is very inefficient and time consuming.
Thanks.
Rich
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
5:49 p.m.
New subject: disk quota not updating
Hi,
I was nearing my disk quota (at 97%), so I deleted a large number of datasets using
"delete permanently". But my usage did not go down at all. Is there a delay in
this happening, or is there some way to purge the files?
richard
6:09 p.m.
New subject: disk quota not updating
Hi Richard,
Yes, it takes a short time for the UI counts to update. If you deleted
permanently, then the result should be what you expected. Should the
quota count remain high by tomorrow, that would point to an issue with
lingering data counting in the quota.
Places to search for unexplained disk use:
1 - Older pre-quota "deleted" datasets that were not permanently
deleted. You can check for these in the View Histories -> advanced ->
deleted set. The far right column "Status" will note deleted vs
permanently deleted.
2 - Shared histories can count towards a quota. So, if not needed or
only portions are, copy out of these what you want to use and ask the
user that shared the data to "unshared" you, so you don't get stuck with
the entire history in your quota. Shared histories/data and quotas are
somewhat tricky to tune, and better solutions may be developed as the
details are worked out, but this is the current implementation. A good
feature to know about is that an imported dataset from a public Data
Library never counts towards your quota (if left unmodified).
You have probably seen this, but for others who may be reading the
thread, this wiki has many details and tips for managing data:
http://galaxyproject.org/wiki/Learn/Managing%20Datasets
One last comment - it would be very helpful for us if questions were
sent with the mailing list as a "to" recipient, so that our ticket
tracker picks it up.
Hopefully this helps! And please feel free to ask if you need more help
or the disk size is not what you expect after the counts refresh.
Best,
Jen
Galaxy team
On 11/30/11 9:49 AM, Richard Mark White wrote:
Hi,
I was nearing my disk quota (at 97%), so I deleted a large number of
datasets using "delete permanently". But my usage did not go down at
all. Is there a delay in this happening, or is there some way to purge
the files?
richard
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
8:07 p.m.
New subject: disk quota not updating
Hi,
Thanks for the info and I did what you suggested. But still no luck. I deleted
everything, and when I add up the data totals in my active histories (I have nothing
shared) it adds up to 376gb, but i am showing 100%. any ideas?
rich
________________________________
From: Jennifer Jackson <jen(a)bx.psu.edu>
To: Richard Mark White <whiter3(a)yahoo.com>; "galaxy-user(a)bx.psu.edu"
<galaxy-user(a)bx.psu.edu>
Cc: closeticket(a)galaxyproject.org
Sent: Wednesday, November 30, 2011 1:09 PM
Subject: disk quota not updating
Hi Richard,
Yes, it takes a short time for the UI counts to update. If you deleted permanently, then
the result should be what you expected. Should the quota count remain high by tomorrow,
that would point to an issue with lingering data counting in the quota.
Places to search for unexplained disk use:
1 - Older pre-quota "deleted" datasets that were not permanently deleted. You
can check for these in the View Histories -> advanced -> deleted set. The far right
column "Status" will note deleted vs permanently deleted.
2 - Shared histories can count towards a quota. So, if not needed or only portions are,
copy out of these what you want to use and ask the user that shared the data to
"unshared" you, so you don't get stuck with the entire history in your
quota. Shared histories/data and quotas are somewhat tricky to tune, and better solutions
may be developed as the details are worked out, but this is the current implementation. A
good feature to know about is that an imported dataset from a public Data Library never
counts towards your quota (if left unmodified).
You have probably seen this, but for others who may be reading the thread, this wiki has
many details and tips for managing data:
http://galaxyproject.org/wiki/Learn/Managing%20Datasets
One last comment - it would be very helpful for us if questions were sent with the mailing
list as a "to" recipient, so that our ticket tracker picks it up.
Hopefully this helps! And please feel free to ask if you need more help or the disk size
is not what you expect after the counts refresh.
Best,
Jen
Galaxy team
On 11/30/11 9:49 AM, Richard Mark White wrote:
Hi,
I was nearing my disk quota (at 97%), so I deleted a large number of
datasets using "delete permanently". But my usage did not go down at
all. Is there a delay in this happening, or is there some way to purge
the files?
richard
-- Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
10:26 p.m.
New subject: disk quota not updating
Hi Rich,
This sounds correct. The quota is set on Main to be 250G. So anything at
or over that amount will be 100% of use.
http://galaxyproject.org/wiki/Main#User_data_and_job_quotas
If I misunderstood your question, please provide more details.
Thanks!
Jen
Galaxy team
On 11/30/11 12:07 PM, Richard Mark White wrote:
Hi,
Thanks for the info and I did what you suggested. But still no luck. I
deleted everything, and when I add up the data totals in my active
histories (I have nothing shared) it adds up to 376gb, but i am showing
100%. any ideas?
rich
------------------------------------------------------------------------
*From:* Jennifer Jackson <jen(a)bx.psu.edu>
*To:* Richard Mark White <whiter3(a)yahoo.com>; "galaxy-user(a)bx.psu.edu"
<galaxy-user(a)bx.psu.edu>
*Cc:* closeticket(a)galaxyproject.org
*Sent:* Wednesday, November 30, 2011 1:09 PM
*Subject:* disk quota not updating
Hi Richard,
Yes, it takes a short time for the UI counts to update. If you deleted
permanently, then the result should be what you expected. Should the
quota count remain high by tomorrow, that would point to an issue with
lingering data counting in the quota.
Places to search for unexplained disk use:
1 - Older pre-quota "deleted" datasets that were not permanently
deleted. You can check for these in the View Histories -> advanced ->
deleted set. The far right column "Status" will note deleted vs
permanently deleted.
2 - Shared histories can count towards a quota. So, if not needed or
only portions are, copy out of these what you want to use and ask the
user that shared the data to "unshared" you, so you don't get stuck with
the entire history in your quota. Shared histories/data and quotas are
somewhat tricky to tune, and better solutions may be developed as the
details are worked out, but this is the current implementation. A good
feature to know about is that an imported dataset from a public Data
Library never counts towards your quota (if left unmodified).
You have probably seen this, but for others who may be reading the
thread, this wiki has many details and tips for managing data:
http://galaxyproject.org/wiki/Learn/Managing%20Datasets
One last comment - it would be very helpful for us if questions were
sent with the mailing list as a "to" recipient, so that our ticket
tracker picks it up.
Hopefully this helps! And please feel free to ask if you need more help
or the disk size is not what you expect after the counts refresh.
Best,
Jen
Galaxy team
On 11/30/11 9:49 AM, Richard Mark White wrote:
> Hi,
> I was nearing my disk quota (at 97%), so I deleted a large number of
> datasets using "delete permanently". But my usage did not go down at
> all. Is there a delay in this happening, or is there some way to purge
> the files?
>
> richard
>
-- Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
8:39 p.m.
New subject: disk quota not updating
On Nov 30, 2011, at 12:49 PM, Richard Mark White wrote:
Hi,
I was nearing my disk quota (at 97%), so I deleted a large number of datasets using
"delete permanently". But my usage did not go down at all. Is there a delay in
this happening, or is there some way to purge the files?
Hi Richard,
It can take a bit if you delete a large amount of data at once. Did your usage eventually
decrease?
--nate
richard
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
2:45 a.m.
New subject: disk quota not updating
yup...took a while, but eventually resolved. thanks.
rich
________________________________
From: Nate Coraor <nate(a)bx.psu.edu>
To: Richard Mark White <whiter3(a)yahoo.com>
Cc: Jennifer Jackson <jen(a)bx.psu.edu>; "galaxy-user(a)bx.psu.edu"
<galaxy-user(a)bx.psu.edu>
Sent: Monday, December 5, 2011 3:39 PM
Subject: Re: [galaxy-user] disk quota not updating
On Nov 30, 2011, at 12:49 PM, Richard Mark White wrote:
Hi,
I was nearing my disk quota (at 97%), so I deleted a large number of datasets using
"delete permanently". But my usage did not go down at all. Is there a delay in
this happening, or is there some way to purge the files?
Hi Richard,
It can take a bit if you delete a large amount of data at once. Did your usage eventually
decrease?
--nate
richard
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
4:30 p.m.
New subject: tar.gz files
Hi,
My seq core returns FASTQ files to me in *.txt.tar.gz format. When I upload this to
galaxy, it unzips, but it is apparently still TAR'd and cannot be read.
Is it possible to upload this format, or do I need to untar and unzip it first (which is
less than ideal)?
Rich
1:34 p.m.
New subject: is public galaxy down?
I am unable to access for past several hours. Are others having the same issue?
rich
3:05 p.m.
New subject: is public galaxy down?
On Dec 17, 2011, at 8:34 AM, Richard Mark White wrote:
I am unable to access for past several hours. Are others having the
same issue?
Hi Rich,
Our core router has crashed, we're working on the problem and hope to have it fixed
within the next few hours. Sorry for the inconvenience.
--nate
rich
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
3:09 p.m.
New subject: downloading (and renaming) output files from Galaxy public server
Hi,
I have generated a transcript file using cufflinks for the human (hg19) or
zebrafish(zv9) assemblies. When I try to display the cufflinks "assembled
transcripts" in UCSC I get this error in the UCSC browser and it wont display the
transcripts.
human: GFF/GTF group NM_005638 on chrX+, this line is on chrY+, all group members must be
on same seq and strand
zebrafish: "GFF/GTF group vapb on chr6+, this line is on chr7-, all group members
must be on same seq and strand"
Any ideas?
Rich
6:22 p.m.
New subject: Cufflinks display in ucsc
Hi,
I have generated a transcript file using cufflinks for the human (hg19) or
zebrafish(zv9) assemblies. When I try to display the cufflinks "assembled
transcripts" in UCSC I get this error in the UCSC browser and it wont display the
transcripts.
human: GFF/GTF group NM_005638 on chrX+, this line is on chrY+, all group members must be
on same seq and strand
zebrafish: "GFF/GTF group vapb on chr6+, this line is on chr7-, all group members
must be on same seq and strand"
Any ideas?
Rich
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
12:13 p.m.
New subject: ftp
Hi,
Is anyone else having trouble connecting to main.g2.bx.psu.edu for FTP uploads? I cannot
seem to connect since yesterday.
Rich
1:39 p.m.
New subject: ftp
On Mar 20, 2012, at 8:13 AM, Richard Mark White wrote:
Hi,
Is anyone else having trouble connecting to main.g2.bx.psu.edu for FTP uploads? I cannot
seem to connect since yesterday.
Hi Rich,
It's back up now.
--nate
Rich
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org. Please keep all replies on the list by
using "reply all" in your mail client. For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists,
please use the interface at:
http://lists.bx.psu.edu/
6:35 a.m.
Hi
the file name is spr097786.fastq.bz2.After upload it showed
spr097786.fastq. It showed it only contain around 5000 sequence reads.
I also tried to upload through FTP. so i download the file to my
computer and then upload to FTP in galaxy. the totlal 800M file was
uploaded to the FTP successfully. But when i transfered the file to
the history i met the same problem. only 5000 sequence reads was moved
to history. I donnot whether it is because of the bz2 file extension.
or i should try other compressed file extension.
xaingmimg
Quoting Jennifer Jackson <jen(a)bx.psu.edu>:
Hello Xiangmimg,
Data files can be loaded using a URL on the "Get Data => Upload"
form. FTP and HTTP connections are supported. This is briefly
described on that form.
If you are still having issues, there may be a problem with file
compression or the connection. Downloading locally then using
Galaxy's FTP upload function is certainly an option.
http://wiki.g2.bx.psu.edu/Learn/Upload%20via%20FTP
Best,
Jen
Galaxy team
On 11/6/11 8:54 PM, Xiangming Ding wrote:
> Hi galaxy
>
> I am a new user of galaxy. i met a problem and didnot find similar
> question in FAQ. I wanted to upload the data from DDBJ DRA dataset to
> galaxy through UTL method. The file is around 800M. However after
> uploading, the FASTQ file was just around 2M. So I wanted know whether
> it is possible to upload a large file to galaxy through URL method? or I
> should download the file to my pc and then uploading to galaxy through
> FTP method.
>
> Thanks
>
> xiangmimg
> ___________________________________________________________
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org. Please keep all replies on the list by
> using "reply all" in your mail client. For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
>
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
>
> http://lists.bx.psu.edu/
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
1:49 p.m.
Hello Xaingmimg,
When you uncompress the archive locally, does it contain a single file
with more than 5000 reads? The consistent results and even number of
reads (5000) may mean that the archive contains more than one file.
Currently, Galaxy will only load the first file in an archive.
Hopefully this helps or you have already found the solution,
Take care,
Jen
Galaxy team
On 11/7/11 10:35 PM, Xiangming Ding wrote:
Hi
the file name is spr097786.fastq.bz2.After upload it showed
spr097786.fastq. It showed it only contain around 5000 sequence reads.
I also tried to upload through FTP. so i download the file to my
computer and then upload to FTP in galaxy. the totlal 800M file was
uploaded to the FTP successfully. But when i transfered the file to the
history i met the same problem. only 5000 sequence reads was moved to
history. I donnot whether it is because of the bz2 file extension. or i
should try other compressed file extension.
xaingmimg
Quoting Jennifer Jackson <jen(a)bx.psu.edu>:
> Hello Xiangmimg,
>
> Data files can be loaded using a URL on the "Get Data => Upload" form.
> FTP and HTTP connections are supported. This is briefly described on
> that form.
>
> If you are still having issues, there may be a problem with file
> compression or the connection. Downloading locally then using Galaxy's
> FTP upload function is certainly an option.
> http://wiki.g2.bx.psu.edu/Learn/Upload%20via%20FTP
>
> Best,
>
> Jen
> Galaxy team
>
> On 11/6/11 8:54 PM, Xiangming Ding wrote:
>> Hi galaxy
>>
>> I am a new user of galaxy. i met a problem and didnot find similar
>> question in FAQ. I wanted to upload the data from DDBJ DRA dataset to
>> galaxy through UTL method. The file is around 800M. However after
>> uploading, the FASTQ file was just around 2M. So I wanted know whether
>> it is possible to upload a large file to galaxy through URL method? or I
>> should download the file to my pc and then uploading to galaxy through
>> FTP method.
>>
>> Thanks
>>
>> xiangmimg
>> ___________________________________________________________
>> The Galaxy User list should be used for the discussion of
>> Galaxy analysis and other features on the public server
>> at usegalaxy.org. Please keep all replies on the list by
>> using "reply all" in your mail client. For discussion of
>> local Galaxy instances and the Galaxy source code, please
>> use the Galaxy Development list:
>>
>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>>
>> To manage your subscriptions to this and other Galaxy lists,
>> please use the interface at:
>>
>> http://lists.bx.psu.edu/
>
> --
> Jennifer Jackson
> http://usegalaxy.org
> http://galaxyproject.org/wiki/Support
>
--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
3477
days inactive
3611
days old
galaxy-user@lists.galaxyproject.org
19 comments
6 participants
participants (6)
-
Jennifer Jackson -
Nate Coraor -
Richard Mark White -
Richard White
-
Simao Lee -
Xiangming Ding