March 2012 - galaxy-dev - lists.galaxyproject.org

GCC2012 Early Registration is Now Open
by Dave Clements 20 Mar '12

20 Mar '12

Hello all, We are pleased to announce that early registration<http://wiki.g2.bx.psu.edu/Events/GCC2012/Register>for the 2012 Galaxy Community Conference (GCC2012)<http://wiki.g2.bx.psu.edu/Events/GCC2012>is now open. GCC2012 will be held July 25-27, at the UIC Forum <http://www.uic.edu/depts/uicforum/>, in Chicago, Illinois. The conference will feature two full days<http://wiki.g2.bx.psu.edu/Events/GCC2012/Program>of presentations, discussions, lightning talks, and breakouts. We have also added a new full day of training<http://wiki.g2.bx.psu.edu/Events/GCC2012/TrainingDay>this year, featuring 3 parallel tracks with four workshops each, covering seven different topics. The Galaxy Community Conference is for: - Sequencing core facility staff - Bioinformatics core staff - Bioinformatics tool and workflow developers - Bioinformatics focused principal investigators and researchers - Data producers - Power bioinformatics users This event will help you with integrating, analyzing, and sharing the diverse and very large datasets that are now typical in biomedical research. GCC2012 is an opportunity to share best practices with, and learn from, a large community of researchers and support staff who are facing the challenges of data-intensive biology. Registration <http://wiki.g2.bx.psu.edu/Events/GCC2012/Register> is very affordable this year, especially for post-docs and students. You can can save an additional 40% by registering on or before June 11. Conference lodging<http://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#Lodging>can also be booked. Low-cost rooms have been reserved on the UIC campus<http://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#James_Stuckel_Towers>. You can also stay at the official conference hotel<http://wiki.g2.bx.psu.edu/Events/GCC2012/Logistics#Crowne_Plaza_Chicago_Met…>, at a substantial discount arranged for conference attendees. There are a limited rooms available in both, and you are encouraged to register early<http://wiki.g2.bx.psu.edu/Events/GCC2012/Register>, if you would like to stay in either. Finally, abstract submission<http://wiki.g2.bx.psu.edu/Events/GCC2012/Abstracts>is also open, from now through April 16. Abstracts <http://wiki.g2.bx.psu.edu/Events/GCC2012/Abstracts> on any topic of interest to the Galaxy community are strongly encouraged. See the Abstracts <http://wiki.g2.bx.psu.edu/Events/GCC2012/Abstracts> page for details. Looking forward to seeing you in Chicago! Dave Clements <http://wiki.g2.bx.psu.edu/DaveClements>, on behalf of the GCC2012 Organizing Committee<http://wiki.g2.bx.psu.edu/Events/GCC2012/Organizing%20Committee> PS: Please help get the word out! A flyer and graphics are at http://wiki.g2.bx.psu.edu/Events/GCC2012/Promotion -- http://galaxyproject.org/GCC2012 http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

1 0

Re: [galaxy-dev] [galaxy-user] Tophat error
by Jennifer Jackson 20 Mar '12

20 Mar '12

Hi David, You question has posted to the list now and we will be getting back to you. It didn't post immediately due to some mail mailman server issues here. This looks like a problem that came up on a local instance. Because of that, I am going to send this over to the galaxy-dev(a)bx.psu.edu mailing list. At first glance, this appears to be a problem with the NGS genome indexes used for the target genome. These are the instructions you followed? http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup (Bowtie indexes are used for TopHat) We will be looking at this more later today, but I wanted to get back to you, so you that you know that this doesn't need to be posted again. Thanks! Jen Galaxy team On 3/14/12 6:48 AM, David Matthews wrote: > Hi, > > JUst running a TopHat job which returned the following error: > > Executing: /gpfs/cluster/isys/galaxy/Software/bin/bowtie-inspect /local/tmp5Ywx45/dataset_942> ./tophat_out/tmp/dataset_942.fa > [Tue Mar 13 12:45:08 2012] Checking for Bowtie > Bowtie version: 0.12.7.0 > [Tue Mar 13 12:45:08 2012] Checking for Samtools > Samtools Version: 0.1.18 > [Tue Mar 13 12:45:08 2012] Generating SAM header for /local/tmp5Ywx45/dataset_942 > format: fastq > quality scale: phred33 (default) > [Tue Mar 13 12:45:21 2012] Preparing reads > left reads: min. length=56, count=29523921 > right reads: min. length=56, count=29543412 > [Tue Mar 13 13:07:54 2012] Mapping left_kept_reads against dataset_942 with Bowtie > [Tue Mar 13 13:45:26 2012] Processing bowtie hits > [Tue Mar 13 14:11:28 2012] Mapping left_kept_reads_seg1 against dataset_942 with Bowtie (1/2) > [Tue Mar 13 14:43:27 2012] Mapping left_kept_reads_seg2 against dataset_942 with Bowtie (2/2) > [Tue Mar 13 14:57:50 2012] Mapping right_kept_reads against dataset_942 with Bowtie > [Tue Mar 13 15:37:46 2012] Processing bowtie hits > [Tue Mar 13 16:04:28 2012] Mapping right_kept_reads_seg1 against dataset_942 with Bowtie (1/2) > [Tue Mar 13 16:37:18 2012] Mapping right_kept_reads_seg2 against dataset_942 with Bowtie (2/2) > [Tue Mar 13 16:50:40 2012] Searching for junctions via segment mapping > Traceback (most recent call last): > File "/gpfs/cluster/isys/galaxy/Software/bin/tophat", line 3063, in<module> > sys.exit(main()) > File "/gpfs/cluster/isys/galaxy/Software/bin/tophat", line 3029, in main > user_supplied_deletions) > File "/gpfs/cluster/isys/galaxy/Software/bin/tophat", line 2681, in spliced_alignment > [maps[initial_reads[left_reads]].unspliced_bwt, maps[initial_reads[left_reads]].seg_maps[-1]], > TypeError: list indices must be integers, not str > > Does anyone know what this kind of error is? > > Best Wishes, > David. > > > > __________________________________ > Dr David A. Matthews > > Senior Lecturer in Virology > Room E49 > Department of Cellular and Molecular Medicine, > School of Medical Sciences > University Walk, > University of Bristol > Bristol. > BS8 1TD > U.K. > > Tel. +44 117 3312058 > Fax. +44 117 3312091 > > D.A.Matthews(a)bristol.ac.uk <mailto:D.A.Matthews@bristol.ac.uk> > > > > > > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

2 2

Dear Galaxy Developer, where is the "history/workflow" file stored in the local machine?
by yangchen＠sibs.ac.cn 20 Mar '12

20 Mar '12

Dear Galaxy Developer, where is the "history/workflow" file stored in the local machine? I mean the real file store in the disk instead of browser access. Thanks and Best regards -- Yang Chen Room 635, No.41 Building,320 Yueyang Road Institute of Biochemistry and Cell Biology Shanghai Institutes for Biological Sciences Chinese Academy of Sciences Shanghai,P.R.China Post Code:200032 Tel: 86-21-54921375

2 1

Help writing a tool
by Mark Johnson 20 Mar '12

20 Mar '12

3 3

Reading user id before the execution of the tool
by Matloob Khushi 20 Mar '12

20 Mar '12

Dear Galaxy Developers, This post went unanswered could I re-submit in a hope if there is any guy out there who could stop me pulling my few left hairs. My tool meant to load up (in a dropdown) some custom data from a non-galaxy database for the user to select and submit a job. I am using dynamic_options for loading such dynamic values, however to restrict data for a logged-in user I need to know the user id. So far I have tried many things including setting dynamic_options="load_dynamic_values($__user_id__) in the xml wrapper but it did not work. I know $__user_id__ can be passed to the tool as a command line arguments, however, my issue is to read user id even before the execution of main tool. Thanks for your help guys. I have also tried following code in a hope to grab the webapp session. ini_file = '/galaxy-dist/universe_wsgi.ini' conf_parser = ConfigParser.ConfigParser( {'here':os.getcwd()} ) conf_parser.read( ini_file ) configuration = {} for key, value in conf_parser.items( "app:main" ): configuration[key] = value global_conf = { '__file__' : ini_file } app = galaxy.app.UniverseApplication(**configuration ) But this generates following error. UnboundExecutionError: Parent instance <History at 0x581cdd0> is not bound to a Session; lazy load operation of attribute 'active_datasets' cannot proceed Regards Matloob Khushi

1 0

Import error when running galaxy on Debian Lenny 64bit
by Jemma Wu 20 Mar '12

20 Mar '12

Dear galaxy, I'm trying to install galaxy on a 64bit Debian (5.0) Lenny server with python2.5.2. When running %sh run.sh, the fetch_eggs completed successfully. However, I have problem with running galaxy instance. I've posted the error messages below. Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.5.egg/paste/script/ command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.5.egg/paste/script/ command.py", line 123, in invoke exit_code = runner.run(args) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.5.egg/paste/script/ command.py", line 218, in run result = self.command() File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.5.egg/paste/script/ serve.py", line 276, in command relative_to=base, global_conf=vars) File "/home/galaxy/galaxy-dist/eggs/PasteScript-1.7.3-py2.5.egg/paste/script/ serve.py", line 313, in loadapp **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.5.egg/paste/deploy/ loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.5.egg/paste/deploy/ loadwsgi.py", line 225, in loadobj return context.create() File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.5.egg/paste/deploy/ loadwsgi.py", line 625, in create return self.object_type.invoke(self) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.5.egg/paste/deploy/ loadwsgi.py", line 110, in invoke return fix_call(context.object, context.global_conf, **context.local_conf) File "/home/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.5.egg/paste/deploy/ util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/home/galaxy/galaxy-dist/lib/galaxy/web/buildapp.py", line 90, in app_factory add_ui_controllers( webapp, app ) File "/home/galaxy/galaxy-dist/lib/galaxy/web/buildapp.py", line 39, in add_ui_controllers module = __import__( module_name ) File "/home/galaxy/galaxy-dist/lib/galaxy/web/controllers/cloud.py", line 9, in <module> import boto ImportError: No module named boto We have another Debian (6.0) Squeeze 32 bit server with python 2.6.6, and I did the same installation of galaxy. Everything went smoothly and galaxy server instance can run successfully on it. Do you know why this importError happens on our 64bit Debian 5.0 server and how I could fix it? Many thanks. Jemma Wu Australian Proteome Analysis Facility (APAF) Level 4, Building F7B, Research Park Drive Macquarie University Sydney NSW 2109 Australia

1 0

Re: [galaxy-dev] Reducing costs in Cloud Galaxy
by Enis Afgan 20 Mar '12

20 Mar '12

Greg, Regarding the performance of different types of instances, I came across this and thought you might potentially find it useful: http://cloudharmony.com/benchmarks Enis On Mon, Mar 19, 2012 at 7:49 PM, Greg Edwards <gedwards2(a)gmail.com> wrote: > Enis, > > Thanks. Will try that re the storage. > > Greg E > > > On Mon, Mar 19, 2012 at 4:49 PM, Enis Afgan <eafgan(a)emory.edu> wrote: > >> Hi Greg, >> >> On Mon, Mar 19, 2012 at 11:01 AM, Greg Edwards <gedwards2(a)gmail.com>wrote: >> >>> Hi, >>> >>> I've got an implementation of some proteomics tools going well in Galaxy >>> on AWS EC2 under Cloudman. Thanks for the help along the way. >>> >>> I need to drive the costs down a bit. I'm using an m1.large AMI and it's >>> costing about $180 - $200 / month. This is about 55% storage and 45% >>> instance costs. That's peanuts in some senses but for now we need to get it >>> down so that it comes out of petty cash for the department, while the case >>> is proven for it's use. >>> >>> I have a few questions and would appreciate ny insights .. >>> >>> >>> 1. AWS has just released an m1.medium and m1.small instance type, which >>> are 1/2 and 1/4 the cost of m1.large. >>> >>> http://aws.amazon.com/ec2/instance-types/ >>> http://aws.amazon.com/ec2/pricing/ >>> >>> I tried the m1.small and m1.medium with the latest Cloudman AMI * *galaxy-cloudman-2011-03-22 >>> (ami-da58aab3) >>> All seemed to install ok, but the Tools took up tp 30 minutes to start >>> execution on m1.medium, and never started on m1.small. >>> >>> m1.medium only added about 15% to run times compared with m1.large, >>> can't say for m1.small. t1.micro does run (and for free in my Free Tier >>> first year) but blows execution times out by a factor of about 3 which is >>> too much. >>> >>> Has anyone tried these new Instance Types ? (m1.small/medium) >>> >> I have no real experience with these instance types yet either so maybe >> someone else can chime in on this? >> >>> >>> >>> 2. The vast majority of the storage costs are fro the Gemome databases >>> in the 700GB /mnt/galaxyIndices, which I don't need. Can this be reduced to >>> the bare essentials ? >>> >> >> You can do this manually: >> 1. Start a new Galaxy cluster (ie, one you can easily delete later) >> 2. ssh into the master instance and delete whatever genomes you don't >> need/want (these are all located under /mnt/galaxyIndices) >> 3. Create a new EBS volume of size that'll fit whatever's left on the >> original volume, attach it and mount it >> 4. Copy over the data from the original volume to the new one while >> keeping the directory structure the same (rsync is probably the best tool >> for this) >> 5. Unmount & detach the new volume; create a snapshot from it >> 6. For the cluster you want to keep around (while it is terminated), edit >> persistent_data.yaml in it's bucket on S3 and replace the existing snap ID >> for the galaxyIndices with the snapshot ID you got in the previous step >> 7. Start that cluster and you should have a file system from the new >> snapshot mounted. >> 8. Terminate & delete the cluster you created in step 1 >> >> If you don't want to have to do this the first time around on your custom >> cluster, you can first try it with another temporary cluster and make sure >> it all works as expected and then move on to the real cluster. >> >> Best, >> Enis >> >>> >>> Using m1.small/medium and getting rid of the 700GB would being my costs >>> down to say $50 / month which is ok. >>> >>> >>> Thanks ! >>> Greg E >>> >>> >>> -- >>> Greg Edwards, >>> Port Jackson Bioinformatics >>> gedwards2(a)gmail.com >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >>> >> >> > > > -- > Greg Edwards, > Port Jackson Bioinformatics > gedwards2(a)gmail.com > >

3 2

Trackster cannot display INTERVAL or GATK-INTERVAL files???
by Anthonius deBoer 19 Mar '12

19 Mar '12

It seems that trackster does not know how to display INTERVAL files? Is that true? Surely there is an easy way to support those kinds of simple files without having to convert them? Am I missing something? I also don't seem to be able to convert an interval file into a BED file? Thanks, Thon

2 2

Hard-links and sshfs
by Daniel Sobral 19 Mar '12

19 Mar '12

Hello, My instance of galaxy is running on an sshfs mount. The problem comes when I set the new_file_path in the universe_wsgi.ini to be a folder within that sshfs. When I upload something, the hard link in the mkstemp_ln function fails because of apparent limitations of sshfs with hard links. It works if I replace that hard link by a file copy, but I would appreciate an alternative that would avoid me messing around the internals of the code. Thanks, Daniel

1 0

Re: [galaxy-dev] sharplabtool in Galaxy Tool Shed
by Greg Von Kuster 17 Mar '12

17 Mar '12

Hello Xuebing, Your updates that will be available for an installed repository are restricted to the version that is installed where the version is defined by the change set revision number. This enables different versions of tools in the repository to always be accessible and installable. The following explanation is taken from the tool shed wiki. http://wiki.g2.bx.psu.edu/Tool%20Shed#Getting_updates_for_tool_shed_reposit… A very important point to convey here is that updates retrieved from the relevant Galaxy tool shed will be restricted to the latest change set that includes those versions of tools that are currently in your installed tool shed repository. Remember that the tool shed repository revision values are a number followed by a : and an alpha-numeric string (e.g., 6:98d05121d41e). Let's assume that at some point you installed revision 0:sdj45ger5fr4 of a tool shed repository into your local Galaxy instance. Then at some later point the related repository in the Galaxy tool shed was updated with revision 1:si88rhjk8hfh. Then even later the same repository in the Galaxy tool shed was updated to a new revision number, say 2:srjls89ojf8e. Let's assume that this latest version resulted in a Revision select list for the repository in the Galaxy tool shed because the version of one or more tools within the repository changed. If you updated your locally installed tool shed repository after these changes to the repository within the Galaxy tool shed were made, your local repository would be updated to revision 1:si88rhjk8hfh, but would not be updated to include the change in revision 2:srjls89ojf8e. Since revision 2:srjls89ojf8e of the repository within the Galaxy tool shed includes tools that have different versions, you have to install that revision into your local Galaxy instance as a separate tool shed repository installation if you want to use the new versions of the tools. On Mar 16, 2012, at 3:52 PM, Xuebing Wu wrote: > Hi Dave, > > I'm not sure if this is the right way to ask questions about tool > shed: I'm having trouble updating my tools installed in a local > instance. I deleted old xml files and uploaded a new one. I can see > revision number changes in toolshed website, but when I tried to > manually update tools by clicking "Get Updates" item in "Repository > Actions", I got message saying that there is no updates available. > > best, > Xuebing > > On Tue, Mar 13, 2012 at 7:01 PM, Dave Clements > <clements(a)galaxyproject.org> wrote: >> Hi Xuebing, >> >> if you are planning to make these available in small pieces, then I won't >> Tweet sharplabtool (unless you tell me to!). >> >> However, I will Tweet the smaller repositories, if and when they show up. >> >> Thanks for sharing this with the community, >> >> Dave C >> >> >> >> On Tue, Mar 13, 2012 at 3:34 PM, Xuebing Wu <wuxbl(a)mit.edu> wrote: >>> >>> I see. It's a set of tools we have on our local galaxy server. So it's >>> pretty heterogeneous, including interval manipulation, meta-gene >>> analysis, sequence and motif analysis, and some others. I don't have a >>> good description for them. I'm planning to separately them anyway. >>> >>> best, >>> Xuebing >>> >>> On Tue, Mar 13, 2012 at 6:26 PM, Greg Von Kuster <greg(a)bx.psu.edu> wrote: >>>> Xuebing, it's fine to have many tools in the same repository, so please >>>> don't feel you have to separate them into multiple repositories unless it is >>>> something you were planning anyway. Dave is just wondering if you have a >>>> good description for the set of tools you have made available. >>>> >>>> Thanks! >>>> >>>> On Mar 13, 2012, at 5:58 PM, Xuebing Wu wrote: >>>> >>>>> Sorry about this. I'm just testing. May be I should try the test >>>>> website first. I'll remove them and then create separate repositories. >>>>> >>>>> best, >>>>> Xuebing >>>>> >>>>> On Tue, Mar 13, 2012 at 5:22 PM, Dave Clements >>>>> <clements(a)galaxyproject.org> wrote: >>>>>> Dear Xuebing, >>>>>> >>>>>> Several days ago you created the new "sharplabtool" in the Galaxy Tool >>>>>> Shed. Normally, I tweet newly deposited tools, but I am not sure how >>>>>> to >>>>>> summarize this repository. It is enormous and includes many tools >>>>>> that >>>>>> would be generally useful at many Galaxy installations. Is there a >>>>>> more >>>>>> informative summary than "sharplabtool" that I could use in the Tweet? >>>>>> I >>>>>> can also make something up. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Dave C >>>>>> >>>>>> -- >>>>>> http://galaxyproject.org/GCC2012 >>>>>> http://galaxyproject.org/ >>>>>> http://getgalaxy.org/ >>>>>> http://usegalaxy.org/ >>>>>> http://galaxyproject.org/wiki/ >>>>>> >>>>> >>>> >> >> >> >> >> -- >> http://galaxyproject.org/GCC2012 >> http://galaxyproject.org/ >> http://getgalaxy.org/ >> http://usegalaxy.org/ >> http://galaxyproject.org/wiki/ >> >

1 0