April 2012 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Cloud instance connection refused
by Dan Tenenbaum 24 Apr '12

24 Apr '12

Hi Julia, On Tue, Apr 24, 2012 at 7:54 PM, Hui Zhao <zhaoh(a)cbio.mskcc.org> wrote: > Hi, > I tried to recreate a security group and still have the same error. Attached is the screen shot I have for the security group. I was able to launch the galaxy instance last year without any problem. Not sure what had changed, Can you tell if there is anything wrong in the security group configuration? Did you click Apply Rule Changes when you finished adding rules? Other than that, I'm not sure what the problem is, but I have not used cloudman before. I followed the instructions and it works for me. Dan > Thanks, > Julia > > > > On Apr 24, 2012, at 9:01 PM, Dan Tenenbaum wrote: > >> On Tue, Apr 24, 2012 at 9:16 AM, Hui Zhao <zhaoh(a)cbio.mskcc.org> wrote: >>> Hi, >>> This is Julia from Memorial SLoan-Kettering Cancer Center. I've been using >>> the galaxy Cloudman instance last year and it worked fine. But today when I >>> tried to launch instance galaxy-cloudman-2011-03-22(ami-da58aab3). After the >>> instance was running, it gave me "tcp_error: A communication error occurred: >>> "Connection refused". Any ideas? >>> >> >> Sounds like maybe a security group problem. Did you set up your >> security group as described here: >> http://wiki.g2.bx.psu.edu/Admin/Cloud >> >> Search for "Add Inbound Rules". >> Dan >> >>> Thanks a lot, >>> Julia >>> >>> >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ > >

1 0

Re: [galaxy-dev] Cloud instance connection refused
by Dan Tenenbaum 24 Apr '12

24 Apr '12

On Tue, Apr 24, 2012 at 9:16 AM, Hui Zhao <zhaoh(a)cbio.mskcc.org> wrote: > Hi, > This is Julia from Memorial SLoan-Kettering Cancer Center. I've been using > the galaxy Cloudman instance last year and it worked fine. But today when I > tried to launch instance galaxy-cloudman-2011-03-22(ami-da58aab3). After the > instance was running, it gave me "tcp_error: A communication error occurred: > "Connection refused". Any ideas? > Sounds like maybe a security group problem. Did you set up your security group as described here: http://wiki.g2.bx.psu.edu/Admin/Cloud Search for "Add Inbound Rules". Dan > Thanks a lot, > Julia > > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/

1 0

Delete multiple datasets
by SHAUN WEBB 24 Apr '12

24 Apr '12

Hi, is there any plan to add capability to delete multiple datasets at once, similar to the copy datasets interface in the history options menu. If I am running workflows and want to delete all the created datasets (during testing or if I've used the wrong parameters etc) it is quite tedious to delete each individual dataset. It would be useful to see and delete any hidden datasets in a similar fashion. Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1

Cloudman- problem with galaxy update
by Dave Lin 24 Apr '12

24 Apr '12

Dear Cloudman Team, I created a fresh galaxy instance using AMI: galaxy-cloudman-2011-03-22 (ami-da58aab3) In case it matters, instance type = High-Memory Double Extra Large I was trying to use the Cloudman Admin Console to update Galaxy. ( http://bitbucket.org/galaxy/galaxy-central) but galaxy ran into an issue during the update process. Log message copied below: Any suggestions? Thanks in advance, Dave /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility from csamtools import * python path is: /mnt/galaxyTools/galaxy-central/eggs/numpy-1.6.0-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/boto-2.2.2-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/mercurial-2.1.2-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/Whoosh-0.3.18-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/pycrypto-2.0.1-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/python_lzo-1.08_2.03_static-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/bx_python-0.7.1_7b95ff194725-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/amqplib-0.6.1-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/pexpect-2.4-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/Babel-0.9.4-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/Mako-0.4.1-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/WebHelpers-0.2-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/wchartype-0.1-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/docutils-0.7-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/WebOb-0.8.5-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/Routes-1.12.3-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/Cheetah-2.2.2-py2.6-linux-x86_64-ucs4.egg, /mnt/galaxyTools/galaxy-central/eggs/PasteDeploy-1.3.3-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/PasteScript-1.7.3-py2.6.egg, /mnt/galaxyTools/galaxy-central/eggs/Paste-1.6-py2.6.egg, /mnt/galaxyTools/galaxy-central/lib, /usr/lib/python2.6/, /usr/lib/python2.6/plat-linux2, /usr/lib/python2.6/lib-tk, /usr/lib/python2.6/lib-old, /usr/lib/python2.6/lib-dynload Traceback (most recent call last): File "/mnt/galaxyTools/galaxy-central/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/mnt/galaxyTools/galaxy-central/lib/galaxy/app.py", line 25, in __init__ self.config.check() File "/mnt/galaxyTools/galaxy-central/lib/galaxy/config.py", line 290, in check raise ConfigurationError("File not found: %s" % path ) ConfigurationError: File not found: ./migrated_tools_conf.xml

2 2

Galaxy pbs scripts and job_working_directory files
by Josh Nielsen 24 Apr '12

24 Apr '12

Hello again, As I mentioned in a recent post I often find need to debug jobs running from our local Galaxy mirror and I often have the need to look at the script & data files that the job is trying to use in order to figure out what is causing a problem. The directories containing those file are in '[galaxy_dist]/database/pbs/' and '[galaxy_dist]/database/job_working_directory/' for me. Each job that is run gets a corresponding .sh file in the pbs/ directory (like 344.sh) which will have the entire sequence of bash commands to execute the job with and also a call to a wrapper script somewhere in the middle normally. That script information is very useful, but the problem is that when a job fails (often within the first 30 seconds of running it) the script is deleted and there is no trace of it left in the directory. The same with the output or job data files in job_working_directory/. I have had to suffice with using the technique of coordinating with the user when to (re)run their failed job and then quickly within the 30 second window do a "cp -R script_I_care_about.sh copy_of_script.sh" command, so that when the script is deleted I have a copy that I can examine. The same goes with copying the job_working_directory/ files. I know that it would get very cluttered in those directories if they were not automatically cleaned/deleted but I find those files essential for debugging. Is there a way to force Galaxy to retain those files (optionally) for debugging purposes? Maybe make a new option in the universe.ini file for that purpose that can be set for people who want it? Thanks, Josh Nielsen

2 2

Cloud instance connection refused
by Hui Zhao 24 Apr '12

24 Apr '12

Hi, This is Julia from Memorial SLoan-Kettering Cancer Center. I've been using the galaxy Cloudman instance last year and it worked fine. But today when I tried to launch instance galaxy-cloudman-2011-03-22(ami-da58aab3). After the instance was running, it gave me "tcp_error: A communication error occurred: "Connection refused". Any ideas? Thanks a lot, Julia

1 0

JobManager object has no attribute dispatcher
by Peter Cock 24 Apr '12

24 Apr '12

Hi all, Does anyone know what might have introduced this problem running galaxy-dist when using the task splitting functionality? I'm using the latest code from the default branch, changeset: 7027:f6e790d94282 galaxy.jobs.manager DEBUG 2012-04-17 13:55:03,610 (4) Job assigned to handler 'main' 127.0.0.1 - - [17/Apr/2012:13:55:06 +0100] "POST /root/history_item_updates HTTP/1.1" 200 - "http://127.0.0.1:8081/history" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110622 CentOS/3.6-1.el5.centos Firefox/3.6.18" galaxy.jobs DEBUG 2012-04-17 13:55:08,710 (4) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/4 galaxy.jobs.handler DEBUG 2012-04-17 13:55:08,711 dispatching job 4 to tasks runner galaxy.jobs.handler INFO 2012-04-17 13:55:08,845 (4) Job dispatched galaxy.datatypes.sequence DEBUG 2012-04-17 13:55:09,470 Split /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat into 4 parts... galaxy.datatypes.sequence DEBUG 2012-04-17 13:55:09,470 Attemping to split FASTA file /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat into chunks of 1 sequences galaxy.datatypes.sequence DEBUG 2012-04-17 13:55:09,471 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/4/task_0/dataset_2.dat galaxy.datatypes.sequence DEBUG 2012-04-17 13:55:09,472 Writing /mnt/galaxy/galaxy-central/database/files/000/dataset_2.dat part to /mnt/galaxy/galaxy-central/database/job_working_directory/000/4/task_1/dataset_2.dat galaxy.jobs.splitters.multi DEBUG 2012-04-17 13:55:09,472 do_split created 2 parts galaxy.jobs DEBUG 2012-04-17 13:55:09,506 (4) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/4 galaxy.jobs.runners.tasks ERROR 2012-04-17 13:55:09,784 failure running job 4 Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/tasks.py", line 86, in run_job self.app.job_manager.dispatcher.put(tw) AttributeError: 'JobManager' object has no attribute 'dispatcher' Thanks, Peter

2 5

Re: [galaxy-dev] Sample tracking
by Greg Von Kuster 24 Apr '12

24 Apr '12

Hello Leandro, I've forwarded your request to the galaxy-dev mail list as this is where issues like this are discussed. I want to make sure I'm clear on this issue. Can you provide some clarification? On Apr 11, 2012, at 5:05 AM, Leandro Hermida wrote: > Based on our initial use of the sample tracking system I haven't found > any additional bugs, but we did realize one big functionality that is > missing which makes the system somewhat hard to use. > > The current mechanism to link datasets to their corresponding samples > is very cumbersome and takes a long time as it has to be done one > sample at a time with a lot of UI clicking and there is a big > potential for human error. I would say without an easier way to do > this it would detract people from using the sample tracking system. Do > you have any ideas to change/enhance the way this is done? The current UI is something that we have plans to improve - the underlying framework is flexible and allows for improvement in several areas. We will certainly consider any recommendations from the community, including yours below. > My initial > suggestion would be that when you upload the samples using a CSV file > that you can have a field like "DatasetsName" after FolderName where > you can have a colon (:) separated list of file paths from the > configured external service? That would solve it I think and make > things much easier. > I've pasted an example screenshot below for reference. At the point that you are importing samples from a CSV file, the sequence run is not yet started (generally speaking), so there are no sequence run datasets (file names) yet produced. So in order to create the csv file with the correct sequence run file names, the user will have to know beforehand the resulting files produced by the sequencer. Will this always be possible with your "Illumina external service"? If so, what are the names of the files, and how are they distinguished between runs? Do they just go in different directories in the sequencer that are also know beforehand? On the other hand, are you saying that your lab will perform the sequence run and wait until the run is complete and the datasets are produced and then create the csv file, entering the known dataset file names to produce the sample line items for the sequencing request? The weakness of this process is that your lab's customers will not be able to use Galaxy sample tracking to view the status of their requests throughout their lifecycle since the request's sample line items will not be created until the run is finished. I'll work on implementing this enhancement, but I want to understand how you're lab will use it. Any additional information you can provide will be helpful. Thanks! Greg Add samples to sequencing request "one" Name State Data Library Folder History Workflow (required) For each sample, select the data library and folder in which you would like the run datasets deposited. To automatically run a workflow on run datastets, select a history first and then the desired workflow. Layout Grid1 Sample form layout 1 Copy samples from sample Select the sample from which the new sample should be copied or leave selection as None to add a new "generic" sample. Click the Add sample button for each new sample and click the Save button when you have finished adding samples. Import samples from csv file The csv file must be in the following format. The [:FieldValue] is optional, the named form field will contain the value after the ':' if included. SampleName,DataLibraryName,FolderName,HistoryName,WorkflowName,Field1Name:Field1Value,Field2Name:Field2Value...

1 0

Deleted datasets
by SHAUN WEBB 24 Apr '12

24 Apr '12

Hi, looking at Galaxy reports I have 505 datasets which have been deleted >60 days ago but have not been purged. My purging scripts seem to be working correctly in cron. Why do these datasets not get deleted? Is it likely that these are in shared or accessible histories and only deleted by the owner? Any tips would be good, trying to free up some space on our server. Thanks Shaun Webb -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

About an additional tool
by Ciara Ledero 24 Apr '12

24 Apr '12

Hi there, I know Galaxy already has a SAM-to-BAM converter, but part of my exercise/task is to incorporate a script that uses samtools' view command. I get this error: [samopen] SAM header is present: 66338 sequences. according to Galaxy. But this might not be an error at all. Is there any way that I could tell Galaxy to ignore this and just continue with the script? Thanks in advance! Any help would be greatly appreciated. CL

2 1