- galaxy-dev - lists.galaxyproject.org

Barcode splitter configuration
by UMD Bioinformatics 16 Oct '13

16 Oct '13

Hello Devteam, I've read through the fastx_readme regarding the barcode splitter. I know that i'm supposed to modify the "publicurl" and "basepath" variables in order to produce the necessary table. I've set the base path to a local folder but I don't know what to use for the public url. I've tried file://path/to/folder but that failed. I've also tried using a dropbox folder and it's public url but that also failed. error An error occurred with this dataset:sed: illegal option -- r sed: : sed script [-Ealn] [-i extension] [file ...] sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file …] The files are generated in both cases so as long as I know where to look I can add them manually but this is hardly a work around. I know that usegalaxy.org places the barcode split files in https://usegalaxy.org/dataset/path/to/file but how do I set that up locally? Thanks Ian

1 0

Re: [galaxy-dev] launching CloudMan 2.0 on AWS
by Enis Afgan 15 Oct '13

15 Oct '13

Hi Vipin, It seems there are too many volumes (or too much volume storage) in your AWS account and AWS is preventing any more from being created. You can either delete some volumes if you don't need them or send a request to AWS to increase your volume limit: https://aws.amazon.com/contact-us/ebs_volume_limit_request/ Cheers, Enis > > On Mon, Oct 14, 2013 at 10:06 PM, Vipin TS <vipin.ts(a)gmail.com> wrote: > >> Hi dev-team, >> >> I am trying to launch cloudman on AWS us-east-1a and I am getting the >> message >> *All cluster services started; the cluster is ready for use. >> (2013-10-14 19:45:52)* >> >> I am seeing some error message in the log. >> * 19:05:33 - Master starting* >> * 19:05:35 - Could not find service class matching userData service >> entry: PSS* >> * 19:05:35 - Completed the initial cluster startup process. This is a >> new cluster; waiting to configure the type.* >> * 19:06:11 - Migration service prerequisites OK; starting the service* >> * 19:06:11 - SGE service prerequisites OK; starting the service* >> * 19:06:26 - Setting up SGE...* >> * 19:06:40 - HTCondor service prerequisites OK; starting the service* >> * 19:06:48 - Hadoop service prerequisites OK; starting the service* >> * 19:07:04 - Done adding Hadoop service; service running.* >> * 19:08:13 - Initializing 'Galaxy' cluster type. Please wait...* >> * 19:08:14 - Error creating volume: EC2ResponseError: 400 Bad Request >> VolumeLimitExceededMaximum number of active volumes bytes, 20, >> exceeded.66659429-b0dd-4f88-b763-36fd9232d3c4* >> * 19:08:17 - Adding volume vol-f9c4728e (FS object for galaxy)...* >> * 19:08:42 - Successfully grew file system FS object for galaxy* >> * 19:08:42 - Successfully mounted file system /mnt/galaxy from >> /dev/xvdf* >> >> It seems that the galaxyIndices filesytem is not mounted to the instance. >> Also I am not able to start the postgres or Galaxy service through cloudman >> interface, both says that Cannot find the service. >> >> Please find the attached image. I think I am missing the galaxyIndices >> volume from the image,. any idea about the failure to launch the instance. >> >> >> Thanks, >> Vipin >> http://galaxy.cbio.mskcc.org >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ >> > >

2 1

Fastx toolkit broken pipe
by UMD Bioinformatics 15 Oct '13

15 Oct '13

Hello, I've got a local install of galaxy with fastx tools installed. I'm getting a broken pipe error but I'm not sure why. I install fastx using the compiler outside galaxy and it was working fine inside galaxy previously. Fastx works fine via the command line. I've checked the xml files and I don't see how galaxy cannot find the commands. I am running this off and external volume, but I don't think that matters. Cheers Ian

5 12

Occasional job failure with /scripts/set_metadata.py ElementTree.parse(fname)
by Peter Cock 15 Oct '13

15 Oct '13

Hi all, Running galaxy-central default branch, I've seen a couple of cases where what otherwise appears to be a successful job has failed with: Traceback (most recent call last): File "./scripts/set_metadata.py", line 122, in <module> __main__() File "./scripts/set_metadata.py", line 72, in __main__ datatypes_registry.load_datatypes( root_dir=config_root, config=datatypes_config ) File "/mnt/galaxy/galaxy-central/lib/galaxy/datatypes/registry.py", line 67, in load_datatypes tree = galaxy.util.parse_xml( config ) File "/mnt/galaxy/galaxy-central/lib/galaxy/util/__init__.py", line 135, in parse_xml tree = ElementTree.parse(fname) File "/mnt/galaxy/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 859, in parse File "/mnt/galaxy/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg/elementtree/ElementTree.py", line 576, in parse IOError: [Errno 2] No such file or directory: '/mnt/galaxy/galaxy-central/database/tmp/tmpWJZ1Aw' I'm not quite sure what was in the temporary XML file which no longer exists, perhaps there is a race condition with the removal of the temporary files? Has anyone else observed this? Thanks, Peter One minor change which might help give a clearer error message here (and in related issues like the functional tests where not all the tools listed in tool_conf.xml.sample always exists on disk) is: $ hg diff /mnt/galaxy/galaxy-central/lib/galaxy/util/__init__.py diff -r 01b94ad2c377 lib/galaxy/util/__init__.py --- a/lib/galaxy/util/__init__.py Tue Oct 15 00:12:08 2013 -0400 +++ b/lib/galaxy/util/__init__.py Tue Oct 15 14:20:01 2013 +0100 @@ -132,6 +132,8 @@ def parse_xml(fname): """Returns a parsed xml tree""" + if not os.path.isfile(fname): + raise IOError("Asked to parse non-existant XML file %r" % fname) tree = ElementTree.parse(fname) root = tree.getroot() ElementInclude.include(root)

1 0

/dev/xvda1 filling up when downloading large files via ftp
by Stanislas Werfel 15 Oct '13

15 Oct '13

Dear all, this problem has been mentioned before by McDowell(a)vims.ed, but there was no solution so far posted. I'm using the current cloudman on Amazon AWS. When I download a fastq.gz file via an EBI-SRA ftp link using the "Upload File" function, although the file itself is loading to the /mnt/galaxy drive, at the same time /dev/xvda1 is filling up and at some point the download stops and says "[Errno 28] No space left on device". I've checked in the universe_wsgi.ini and the ftp upload path is set correctly to /mnt/galaxy/tmp/ftp. Here's the df output: Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvda1 10321208 9797012 0 100% / udev 8736124 8 8736116 1% /dev tmpfs 3497960 232 3497728 1% /run none 5120 0 5120 0% /run/lock none 8744896 0 8744896 0% /run/shm cgroup 8744896 0 8744896 0% /sys/fs/cgroup /dev/xvdb 423135208 207440 401433728 1% /mnt /dev/xvdf 104847360 22055572 82791788 22% /mnt/galaxy /dev/xvdg 733900800 685188288 48712512 94% /mnt/galaxyIndices I'm also having another issue: when I delete files in the history in galaxy, they don't acually get deleted in the files/000/ folder. And galaxy is still showing them under "Using ..." in the total space used. So if I download a 3GB fastq file, then delete everything in the history, Using is still showing 3GB and the file is still there in the folder. Could the two issues be connected? Are there maybe some rights missing for the galaxy instance to delete files? Thanks in advance for any help Best Stanislas

1 0

about the sqlalchemy-migrate
by ngsflow＠hygenomics.com 15 Oct '13

15 Oct '13

hi,all: 1. when I started the galaxy daemon. In the paster.log has a warning.who can help me. url= http://code.google.com/p/sqlalchemy-migrate/ 2. missing file. 3.What should I do about it? 4.What should I do about it? ngsflow(a)hygenomics.com

2 1

How to specify the availability zone when using cloudman?
by Carlos Borroto 15 Oct '13

15 Oct '13

Hi, I'm getting this error when trying to restart a previously used cluster: "InvalidVolume.ZoneMismatchThe volume '' is not in the same availability zone as instance" I did a search and I could only find this previous email in galaxy-user: http://lists.bx.psu.edu/pipermail/galaxy-user/2012-January/003946.html Which while confirming I should make sure to match the new instance zone with previous volumes zone, it does not say how to do it. Thanks, Carlos

1 0

launching CloudMan 2.0 on AWS
by Vipin TS 14 Oct '13

14 Oct '13

Hi dev-team, I am trying to launch cloudman on AWS us-east-1a and I am getting the message *All cluster services started; the cluster is ready for use. (2013-10-14 19:45:52)* I am seeing some error message in the log. * 19:05:33 - Master starting* * 19:05:35 - Could not find service class matching userData service entry: PSS* * 19:05:35 - Completed the initial cluster startup process. This is a new cluster; waiting to configure the type.* * 19:06:11 - Migration service prerequisites OK; starting the service* * 19:06:11 - SGE service prerequisites OK; starting the service* * 19:06:26 - Setting up SGE...* * 19:06:40 - HTCondor service prerequisites OK; starting the service* * 19:06:48 - Hadoop service prerequisites OK; starting the service* * 19:07:04 - Done adding Hadoop service; service running.* * 19:08:13 - Initializing 'Galaxy' cluster type. Please wait...* * 19:08:14 - Error creating volume: EC2ResponseError: 400 Bad Request VolumeLimitExceededMaximum number of active volumes bytes, 20, exceeded.66659429-b0dd-4f88-b763-36fd9232d3c4* * 19:08:17 - Adding volume vol-f9c4728e (FS object for galaxy)...* * 19:08:42 - Successfully grew file system FS object for galaxy* * 19:08:42 - Successfully mounted file system /mnt/galaxy from /dev/xvdf* It seems that the galaxyIndices filesytem is not mounted to the instance. Also I am not able to start the postgres or Galaxy service through cloudman interface, both says that Cannot find the service. Please find the attached image. I think I am missing the galaxyIndices volume from the image,. any idea about the failure to launch the instance. Thanks, Vipin http://galaxy.cbio.mskcc.org

1 0

Getting bowtie2 and tophat2 to work on galaxy on amazon aws cloud
by Stanislas Werfel 14 Oct '13

14 Oct '13

Dear all, I'm trying to run Galaxy on an Amazon AWS cloud using the normal Cloudman distribution (ami-118bfc78). The instatiated Galaxy however does not contain Bowtie2 and Tophat2, even after updating through the Cloudman Admin section. Although the wrappers are there in the tools directory, when I access it through ssh. I've also installed Bowtie2 manually on the server and the command bowtie2 does get recognized. How cat I get these two to run on the Galaxy instance? Thanks for your help. Best Stanislas

2 4

AttributeError: type object 'InvalidJobException' has no attribute 'name'
by Adhemar 14 Oct '13

14 Oct '13

Hi, After the last update I'm getting the following error. The job is submitted to SGE e executed, but galaxy doesn't get the result and keeps showing the job is executing (yellow box). Any clues? Thanks, Adhemar galaxy.jobs.runners ERROR 2013-10-08 13:01:18,488 Unhandled exception checking active jobs Traceback (most recent call last): File "/opt/bioinformatics/share/galaxy20130410/lib/galaxy/jobs/runners/__init__.py", line 362, in monitor self.check_watched_items() File "/opt/bioinformatics/share/galaxy20130410/lib/galaxy/jobs/runners/drmaa.py", line 217, in check_watched_items log.warning( "(%s/%s) job check resulted in %s: %s", galaxy_id_tag, external_job_id, e.__class__.name, e ) AttributeError: type object 'InvalidJobException' has no attribute 'name'

3 6