Hi Martin,

Thanks. I'll read the manual.

Best,

Rui

On Fri, Feb 28, 2014 at 8:47 AM, Martin Čech <marten@bx.psu.edu> wrote:

Hello Rui,

the process of migrating tools from Galaxy itself to the ToolShed (Galaxy's appstore) is ongoing and is well described here: https://wiki.galaxyproject.org/MigratingToolsFromGalaxyDistribution

Please check the page and let us know if it helps you with your instance. 

Afterwards we can continue to troubleshoot the upload problems.

goodluck

Martin

On Fri, Feb 28, 2014 at 12:20 AM, ruiwang.sz <ruiwang.sz@gmail.com> wrote:

Hi Sam,

I used Get Data and Upload file from your computer...and this just started this Feb, actually a couple days ago it was fine. It happened always, no matter we load from localhost or remote host, and other means like paste the path etc didn't work either.

Actually, I found that there are many released change on the galaxy release and then I ran a 'hg pull -u'. However I had some error message saying many of the tools installed are eliminated in the new distribution and I could install them back from toolshed. I'm quite confused. 

Here is the command I ran:

1. hg pull -u

2. ./run.sh --daemon    (many error msg, follow the suggestion,then )

3. sh manage_db.sh upgrade      (update db to 118 version, start again)

4. ./run.sh --daemon    (many error msg, I'll paste it below, but next I did)

5. sh ./scripts/migrate_tools/0009_tools.sh

now after 5, I run it again, it started, but when I log in, seems this page is hanging for few hours:

so the error message in step 4 above was:

The list of files at the end of this message refers to tools that are configured to load into the tool panel for

this Galaxy instance, but have been removed from the Galaxy distribution.  These tools and their dependencies can be

automatically installed from the Galaxy tool shed at http://toolshed.g2.bx.psu.edu.

To skip this process, attempt to start your Galaxy server again (e.g., sh run.sh or whatever you use).  If you do this,

be aware that these tools will no longer be available in your Galaxy tool panel, and entries for each of them should

be removed from your file named ./tool_conf.xml

...migrate 7->8

...migrate 8->9

from the main Galaxy tool shed at http://toolshed.g2.bx.psu.edu

and will be installed into your local Galaxy instance at the

location discussed above by running the following command.

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

sh ./scripts/migrate_tools/0009_tools.sh

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tool dependencies listed above will be installed along with the repositories if you add the 'install_dependencies'

option to the above command like this:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

sh ./scripts/migrate_tools/0009_tools.sh install_dependencies

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tool dependencies can be installed after the repositories have been installed as well.

After the installation process finishes, you can start your Galaxy server.  As part of this installation process,

entries for each of the following tool config files will be added to the file named ./migrated_tool_conf.xml, so these

tools will continue to be loaded into your tool panel.  Because of this, existing entries for these tools have been

removed from your file named ./tool_conf.xml.

fastq_to_fasta.xml

execute_dwt_var_perFeature.xml

GMAJ.xml

rgWebLogo3.xml

rgClustalw.xml

gi2taxonomy.xml

t2t_report.xml

t2ps_wrapper.xml

find_diag_hits.xml

lca.xml

poisson2test.xml

rgFastQC.xml

fastq_groomer.xml

fastq_paired_end_splitter.xml

fastq_paired_end_joiner.xml

fastq_stats.xml

fastq_combiner.xml

fastq_filter.xml

fastq_trimmer.xml

fastq_trimmer_by_quality.xml

fastq_masker_by_quality.xml

fastq_paired_end_interlacer.xml

fastq_paired_end_deinterlacer.xml

fastq_manipulation.xml

fastq_to_tabular.xml

tabular_to_fastq.xml

bowtie2_wrapper.xml

tophat_wrapper.xml

tophat2_wrapper.xml

express_wrapper.xml

filter_transcripts_via_tracking.xml

macs_wrapper.xml

sicer_wrapper.xml

ccat_wrapper.xml

freebayes.xml

I ran the command

sh ./scripts/migrate_tools/0009_tools.sh

and after that, the start up still shows some errors like:

galaxy.tools ERROR 2014-02-27 18:08:16,514 Error reading tool from path: data_source/CR.xml

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set

    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool

    tree = load_tool( config_file )

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool

    tree = parse_xml(path)

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml

    tree = ElementTree.parse(fname)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse

    tree.parse(source, parser)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse

    source = open(source, "rb")

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'

galaxy.tools ERROR 2014-02-27 18:08:16,539 Error reading tool from path: data_source/genomespace_file_browser_prod.xml

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 502, in load_tool_tag_set

    tool = self.load_tool( os.path.join( tool_path, path ), guid=guid, repository_id=repository_id )

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 615, in load_tool

    tree = load_tool( config_file )

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/loader.py", line 13, in load_tool

    tree = parse_xml(path)

  File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/__init__.py", line 151, in parse_xml

    tree = ElementTree.parse(fname)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse

    tree.parse(source, parser)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse

    source = open(source, "rb")

IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'

galaxy.tools WARNING 2014-02-27 18:08:16,548 Tool peakcalling_macs: a when tag has been defined for 'genome_size_cond (genome_size) --> 90300000', but does not appear to be selectable.

now if I do a 'grep IOError paster.log', I got the following:

IOError: [Errno 2] No such file or directory: './tools/data_source/CR.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/genomespace_file_browser_prod.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/bx_browser.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/flymine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/flymine_test.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/modmine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/mousemine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/ratmine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/yeastmine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/metabolicmine.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/encode_db.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import.xml'

IOError: [Errno 2] No such file or directory: './tools/data_source/epigraph_import_test.xml'

IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph.xml'

IOError: [Errno 2] No such file or directory: './tools/data_destination/epigraph_test.xml'

IOError: [Errno 2] No such file or directory: './tools/encode/gencode_partition.xml'

IOError: [Errno 2] No such file or directory: './tools/encode/random_intervals.xml'

IOError: [Errno 2] No such file or directory: './tools/new_operations/column_join.xml'

IOError: [Errno 2] No such file or directory: './tools/extract/phastOdds/phastOdds_tool.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/t_test_two_samples.xml'

IOError: [Errno 2] No such file or directory: './tools/plotting/xy_plot.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/getIndels_3way.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/delete_overlapping_indels.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motifs_frequency.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/compute_motif_frequencies_for_all_motifs.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/categorize_elements_satisfying_criteria.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/draw_stacked_barplots.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/multispecies_MicrosatDataGenerator_interrupted_GALAXY.xml'

IOError: [Errno 2] No such file or directory: './tools/regVariation/microsatellite_birthdeath.xml'

IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_branch_lengths_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_nj_tree_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/hyphy/hyphy_dnds_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/evolution/mutate_snp_codon.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_formatter.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fasta_nucleotide_changer.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_converter.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_quality_statistics.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_boxplot.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_nucleotides_distribution.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastq_quality_filter.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_artifacts_filter.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_barcode_splitter.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_clipper.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_collapser.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_renamer.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_reverse_complement.xml'

IOError: [Errno 2] No such file or directory: './tools/fastx_toolkit/fastx_trimmer.xml'

IOError: [Errno 2] No such file or directory: './tools/indels/sam_indel_filter.xml'

IOError: [Errno 2] No such file or directory: './tools/indels/indel_sam2interval.xml'

IOError: [Errno 2] No such file or directory: './tools/indels/indel_table.xml'

IOError: [Errno 2] No such file or directory: './tools/indels/indel_analysis.xml'

IOError: [Errno 2] No such file or directory: './tools/ngs_rna/tophat_color_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cufflinks_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffcompare_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffmerge_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/ngs_rna/cuffdiff_wrapper.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/sam_bitwise_flag_filter.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/sam2interval.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/sam_to_bam.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/bam_to_sam.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/sam_merge.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_mpileup.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/sam_pileup.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_parser.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/pileup_interval.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_flagstat.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_rmdup.xml'

IOError: [Errno 2] No such file or directory: './tools/samtools/samtools_slice_bam.xml'

IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_indexer.xml'

IOError: [Errno 2] No such file or directory: './tools/genetrack/genetrack_peak_prediction.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgClean.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgPedSub.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgLDIndep.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePhe.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgfakePed.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgQC.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgEigPCA.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgHaploView.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGRR.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgCaCo.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgTDT.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgGLM.xml'

IOError: [Errno 2] No such file or directory: './tools/rgenetics/rgManQQ.xml'

IOError: [Errno 2] No such file or directory: './tools/phenotype_association/ctd.xml'

IOError: [Errno 2] No such file or directory: './tools/phenotype_association/funDo.xml'

IOError: [Errno 2] No such file or directory: './tools/phenotype_association/hilbertvis.xml'

IOError: [Errno 2] No such file or directory: './tools/phenotype_association/dividePgSnpAlleles.xml'

IOError: [Errno 2] No such file or directory: './tools/vcf_tools/intersect.xml'

I'm not sure if I successfully updated galaxy source...since right now the screen is always hanging. I think I miss some config files or so.

Could you please advise?

Thanks,

Rui

On Thu, Feb 27, 2014 at 7:21 PM, sam guerler <aysam.guerler@gmail.com> wrote:

Hi Rui,

Are you using the upload tool available in the tool panel under 'Get Data' > 'Upload file from your computer'? Or the new upload dialog by clicking on 'Load Data'? Could you also please let me know if this error occurs always or just sometimes?

Thanks Sam

On Thu, Feb 27, 2014 at 8:50 PM, ruiwang.sz <ruiwang.sz@gmail.com> wrote:

Hi Guys, 

Our galaxy instance has been in use for a couple years but since Feb 2014 it started to show us some weird behavior. Recently the upload function suddenly stopped working properly. We are getting messages like the following. I'm not sure if this only happens to us. If someone has seen this before, please give me some hints.

P.S., I searched online and many says that this is because the browser does not wait till all the data from the server is received and closes the socket. However it happens both in firefox and chrome. I'm wondering if it is because galaxy has any change recently but when I do 'hg update stable' it reported that it is up to date.

Thanks!

Rui

Error messages:

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

Exception happened during processing of request from (, 52259'127.0.0.1', 52260)

Traceback (most recent call last):

)

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

----------------------------------------

Exception happened during processing of request from (----------------------------------------

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from'127.0.0.1' , ('127.0.0.1'52264, 52262----------------------------------------

)

Exception happened during processing of request from ('127.0.0.1'----------------------------------------

, )

Traceback (most recent call last):

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Exception happened during processing of request from 52261('127.0.0.1', ----------------------------------------

)Exception happened during processing of request from

Traceback (most recent call last):

52263('127.0.0.1')

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

, 52265)

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

Traceback (most recent call last):

  File "/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py", line 1068, in process_request_in_thread

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

    self.finish_request(request, client_address)

  File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request

127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET /tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0"

    self.RequestHandlerClass(request, client_address, self)

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

    self.RequestHandlerClass(request, client_address, self)

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.RequestHandlerClass(request, client_address, self)

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

  File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__

    self.finish()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

    self.wfile.flush()

    self.finish()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

    self.finish()

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/SocketServer.py", line 693, in finish

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self.wfile.flush()

  File "/usr/lib/python2.7/socket.py", line 303, in flush

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

error: [Errno 32] Broken pipe

----------------------------------------

    self._sock.sendall(view[write_offset:write_offset+buffer_size])

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

----------------------------------------

error: [Errno 32] Broken pipe

----------------------------------------

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/