Team,

I'm up and running fine, thanks.

Thanks Dannon for the info on 'migrated_tools_conf.xml.sample', that allowed a normal update.
Thanks Ross for the info on coverage datatype definition from datatypes_conf.xml.
Thanks Enis for the tip that a new clustername guarantees a clean new Galaxy.

I've found something that may be of interest. A few days ago AWS EC2 added an m1.small and m1.medium model for 64-bit AMI's, and Galaxy starts ok with these. But it has a timing problem that I mentioned before. On m1.medium the jobs sit in the History queue in the gray waiting state for 1 to 30 minutes before they run. They run ok after that. On m1.small they sit there forever and don't run. Or they certainly don't run within 5 to 6 hours. Things work ok (or used to anyway) on t1.micro so I'm guessing there's a 'case' statement on instance type somewhere that is falling through for m1.medium and m1.small. (A wild guess but you know how it goes, it's usually simple  :=)

One last gremlin. My new tools are added at the top of tool_conf.xml and have always shown up at the top of the Tools list, in a local Galaxy and previous Cloud Galaxies. Now they show up near the bottom of the tools list. There's no obvious ordering happening like alphabetical tool name or file-date or name of tools/tooldir. Any ideas here appreciated.

Thanks !

Greg E


On Sat, Mar 10, 2012 at 3:36 PM, Enis Afgan <afgane@gmail.com> wrote:

Also Greg
Starting a new cloudman instance with a different cluster name than previously used will always give you a clean environment and setup so you can continue to always use the us-east region.

Cheers,
Enis

On 10/03/2012 2:08 PM, "Ross" <ross.lazarus@gmail.com> wrote:
Since Dannon is likely asleep, maybe this will help until he can respond:
http://dev.list.galaxyproject.org/error-on-fastq-file-while-grooming-or-fastqc-report-or-fastqc-converter-td4396664.html

ie: try removing the second coverage datatype definition from
datatypes_conf.xml and restarting the Galaxy process.

On Sat, Mar 10, 2012 at 2:00 PM, Greg Edwards <gedwards2@gmail.com> wrote:
> Dannon,
>
> Thanks for that. Adding 'migrated_tools_conf.xml.sample' fixed up most
> problems. I'm left with a couple though.
>
> 1. The tools (apart from file upload with Get Data) don't initially execute.
> This includes my new tools and existing ones like Text Manipulation / Paste
> Files side-by-side. They stay in the grey "waiting to run" mode for about
> 15-30 minutes, then run.
>
> 2. When they finally run, they compute and report fine but finish in the Red
> error colour, and stderr has reported as below.This is both my tools and the
> system standard ones.
>
>
> WARNING:galaxy.datatypes.registry:Overriding conflicting datatype with
> extension 'coverage', using datatype from /mnt/galaxyData/tmp/tmp6o5hnt.
>
>
> There's an oddity I should mention. When I first fire
> up galaxy-cloudman-2011-03-22, a remnant of my add-on tools is already
> showing at the top of the Tools menu. "PTM Tools" is there, and in
> tools_conf.xml. But the tools are not in tools/ptmtools, so the menu is
> inactive. It's like I've corrupted the file systems associated
> with galaxy-cloudman-2011-03-22.
>
> I'm thinking of trying it all on the Singapore AWS node where I can't have
> touched the AMI's or file sets. This has all been on US East. In fact it's
> been on US East 1d, I could try on 1a, I never go there normally, but they
> probably share data. I could try on US West. I'll let you know, if I don't
> hear before.
>
> Greg.
>
>
>
>
> On Sat, Mar 10, 2012 at 12:14 AM, Dannon Baker <dannonbaker@me.com> wrote:
>>
>> Greg,
>>
>> The problem here is that the galaxy update failed to merge a change to
>> run.sh because of minor customizations it has.  We'll have a long term fix
>> out for this soon, but for now what you can do is ssh in to your instance
>> and update run.sh yourself prior to restarting galaxy.  All you need to do
>> is add 'migrated_tools_conf.xml.sample' to the SAMPLES in
>> /mnt/galaxyTools/galaxy-central/run.sh, execute `sh run.sh --run-daemon` (or
>> restart galaxy again from the admin page) and you should be good to go.
>>
>> That new AMI you're seeing is not owned by the Galaxy Team, and we don't
>> actually know who made it.  Keep using the same galaxy-cloudman-2011-03-22
>> for now (and we'll always have the most up-to-date AMI listed at
>> usegalaxy.org/cloud).  Because of Cloudman's modular volume design almost
>> nothing resides on the AMI itself, so we can (and do) update the tools and
>> index volumes without having to touch it.  So while the AMI reflects a date
>> of almost a year old, the galaxy tools volume (and thus the actual Galaxy
>> instance you're running) has been updated much more recently.
>>
>> One last note- if you're updating and copying your tools in every time,
>> you may want to try using the 'Persist changes' functionality available in
>> the Cloudman admin panel.  Once you've set your instance up how you want, if
>> you click 'Persist changes to galaxyTools', it'll create a custom snapshot
>> of your tools volume that will be used from that point forward with this
>> instance.
>>
>> Let me know if you have any more issues,
>>
>> -Dannon
>>
>> On Mar 9, 2012, at 2:53 AM, Greg Edwards wrote:
>>
>> > Hi,
>> >
>> > I'm trying to restart my Galaxy Cloudman service, using the same
>> > approach that has been successful over the last couple of months ..
>> >
>> >  - launch AMI 861460482541/galaxy-cloudman-2011-03-22 as m1.large
>> >  - update from Cloudman console
>> >  - copy in my tools etc etc
>> >  - restart
>> >  - away we go, all works
>> >
>> > However today the update fails, the log says ...
>> >
>> >
>> > /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1:
>> > RuntimeWarning: __builtin__.file size changed, may indicate binary
>> > incompatibility
>> >   from csamtools import *
>> > python path is:
>> > /mnt/galaxyTools/galaxy-central/eggs/numpy-1.6.0-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/boto-2.2.2-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Whoosh-0.3.18-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/pycrypto-2.0.1-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/python_lzo-1.08_2.03_static-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/amqplib-0.6.1-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/pexpect-2.4-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Babel-0.9.4-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Mako-0.4.1-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/WebHelpers-0.2-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/wchartype-0.1-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/docutils-0.7-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/WebOb-0.8.5-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Routes-1.12.3-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/eggs/Paste-1.6-py2.6.egg,
>> > /mnt/galaxyTools/galaxy-central/lib, /usr/lib/python2.6/,
>> > /usr/lib/python2.6/plat-linux2, /usr/lib/python2.6/lib-tk,
>> > /usr/lib/python2.6/lib-old, /usr/lib/python2.6/lib-dynload
>> > Traceback (most recent call last):
>> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/web/buildapp.py",
>> > line 82, in app_factory
>> >     app = UniverseApplication( global_conf = global_conf, **kwargs )
>> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/app.py", line 24, in
>> > __init__
>> >     self.config.check()
>> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/config.py", line 243,
>> > in check
>> >     tree = parse_xml( config_filename )
>> >   File "/mnt/galaxyTools/galaxy-central/lib/galaxy/util/__init__.py",
>> > line 105, in parse_xml
>> >     tree = ElementTree.parse(fname)
>> >   File
>> > "/mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
>> > line 859, in parse
>> >     tree.parse(source, parser)
>> >   File
>> > "/mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py",
>> > line 576, in parse
>> >     source = open(source, "rb")
>> > IOError: [Errno 2] No such file or directory:
>> > './migrated_tools_conf.xml'
>> > Removing PID file paster.pid
>> >
>> >
>> > While I'm here, I see a new Galaxy Cloudman AMI
>> >  072133624695/galaxy-cloudman-2012-02-26.  I can't manage to start that, I
>> > get an error as below, with all types of instance,
>> > (tiny/small/medium/large). Is that a recommended AMI now ?  It would be good
>> > to have a new updated AMI.
>> >
>> > <snap1.png>
>> >
>> > Thanks !
>> >
>> > --
>> > Greg Edwards,
>> > Port Jackson Bioinformatics
>> > gedwards2@gmail.com
>> >
>> > ___________________________________________________________
>> > The Galaxy User list should be used for the discussion of
>> > Galaxy analysis and other features on the public server
>> > at usegalaxy.org.  Please keep all replies on the list by
>> > using "reply all" in your mail client.  For discussion of
>> > local Galaxy instances and the Galaxy source code, please
>> > use the Galaxy Development list:
>> >
>> >  http://lists.bx.psu.edu/listinfo/galaxy-dev
>> >
>> > To manage your subscriptions to this and other Galaxy lists,
>> > please use the interface at:
>> >
>> >  http://lists.bx.psu.edu/
>>
>
>
>
> --
> Greg Edwards,
> Port Jackson Bioinformatics
> gedwards2@gmail.com
>
>
> ___________________________________________________________
> The Galaxy User list should be used for the discussion of
> Galaxy analysis and other features on the public server
> at usegalaxy.org.  Please keep all replies on the list by
> using "reply all" in your mail client.  For discussion of
> local Galaxy instances and the Galaxy source code, please
> use the Galaxy Development list:
>
>  http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> To manage your subscriptions to this and other Galaxy lists,
> please use the interface at:
>
>  http://lists.bx.psu.edu/



--
Ross Lazarus MBBS MPH;
Associate Professor, Harvard Medical School;
Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

 http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

 http://lists.bx.psu.edu/



--
Greg Edwards,
Port Jackson Bioinformatics
gedwards2@gmail.com