December 2011 - galaxy-dev - lists.galaxyproject.org

SICER bug in Workflow
by Susan Newman 07 Dec '11

07 Dec '11

I'm working on a SOLiD ChIP-Seq workflow in which I want to use SICER for peak detection. In attempting to add SICER to the workflow, I've run into some difficulty: 1) My input is an interval file which I've created using Convert SAM. In order to connect to the SICER module, I had to use a trick by setting up the connection with the Convert SAM set to BED. The SICER algorithm fails to run when tested with the bed file. If I then change the connected Convert SAM to interval, SICER will run. 2) I need to change the default run parameters but they are not saved in the workflow. There is always reversion to the default values. I get a Server error notice. This workflow also contains Bowtie and MACS. Parameter changes for these algorithms are saved correctly. 3) I would like to run SICER twice in the workflow with different parameters. Is it allowable to load SICER a second time? May I run from the same interval conversion file? I tried this and again could not get the input file recognized as a valid connection. Thanks in advance, Susan Susan S. Newman, Laboratory Manager Genomics Core Facility, L2012 Pennington Biomedical Research Center 6400 Perkins Road Baton Rouge, LA 70808 225-763-0255 (o) 225-763-0257 (lab) http://gcf.pbrc.edu

2 1

Configuring Toolsheds
by Zachary Charlop-Powers 07 Dec '11

07 Dec '11

Hello Galaxy, I have been having trouble getting Toolsheds to work on a local install of Galaxy. Based on the error messages I think that I have not correctly modified the nginx server port forwarding and/or the community_wsgi.ini file. If anyone has successfully setup a local toolshed repository I would love to hear from you. thanks zach cp

2 1

Tool creation: how to restrict input dataset selector to appropriate file types?
by Louise-Amélie Schmitt 07 Dec '11

07 Dec '11

Hello, I'm running in a strange issue which I cannot seem to find a solution for. In every tool I created from scratch, the input dataset selector does not only include the datasets corresponding to the file types I set in the format="" attribute in the <param> tag in my XML tool file, but instead it includes the whole history contents. I searched in the wiki's tool syntax section to no avail. I believe I stupidly missed something huge there but I'm really confused. What should I do to have the selector's contents appropriate? Here is one of the tools for which I have this issue: http://pastebin.com/XqvSwtMH Thanks, L-A

2 2

About database indexes
by Louise-Amélie Schmitt 06 Dec '11

06 Dec '11

Hello, Since our local Galaxy is a little sluggish, we were wondering if it could be related to broken or missing database indexes (we're using PostgreSQL), so we would like to know how they are managed in Galaxy, and if there is a way to restore them automatically. Thanks, L-A

2 1

Re: [galaxy-dev] [Internal - Galaxy-dev #2159] (New) 2 questions about Galaxy's functional testing support
by Greg Von Kuster 06 Dec '11

06 Dec '11

Hello Eric, > Submitted by epaniagu(a)cshl.edu > Hi all, > > I've read the Wiki page on Writing Functional Tests (http://wiki.g2.bx.psu.edu/Admin/Tools/Writing%20Tests) and I've been looking through test/base and test/functional and I am left with two questions: > > * Is it possible to write a test to validate metadata directly on an (optionally composite) output dataset? I'm sure this is possible, but it would require enhancements to the current functional test framework. > Everything described on the above page is file oriented. I see that there is TwillTestCase.check_metadata_for_string, but as far as I can tell this is a bit nonspecific since it appears to just do a text search on the Edit page. This is correct. > I don't yet fully understand the context in which tests run, but is there some way to access a "live" dataset's metadata directly, either as a dictionary or just as attributes? Or even to get the actual dataset object? Not with the current functional test framework. Doing this would require enhancements to the framework. > > * Does the test harness support retaining output files only for failed tests? Ideally with a cap on how much output data to save. If not, would this be difficult to configure? I'm not sure what you mean by "output files" in your question. If you mean output datasets that result from running a functional test for a tool, then I believe there is no difference if the test passed or failed. > > Thanks, > Eric > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

3 6

Re: [galaxy-dev] Urgent help needed
by Toqa Manasrah 06 Dec '11

06 Dec '11

Thank you so much Dannon. I appreciate your help but I need some more clarification. in the tool I am integrating (which is java classes) if the input parameters are ./main.bash eps0.3_40reads.fa population10_ref.fa 15 6 120 Then teh output is in eps0.3_40reads_I_6_15_CNTGS_DIST0_EM20.txt As u notice the output file name is not specified in teh command line, and actually I dont have the source files to go over this naming restriction. how I can use from_work_dir attribute to tell galaxy this the file to display? note: there are 11 output files and I am interested in this one only ..._CNTGS_DIST0_EM20.txt any other method to overcome such problem I have presentation tomorrwo. hope things will work fine. Thank you, ________________________________________ From: Dannon Baker [dannonbaker(a)me.com] Sent: Monday, December 05, 2011 2:41 PM To: Toqa Manasrah Subject: Re: [galaxy-dev] Urgent help needed Sure. See the link in my previous email (http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax) and look for the <data> section for an explanation of the attributes available to you. For an example of a tool that uses the from_work_dir element, see the cufflinks, cuff compare, top hat, or trinity wrappers in your galaxy install. -Dannon On Dec 5, 2011, at 2:25 PM, Toqa Manasrah wrote: > Thank you Dannon, > I mean the first choice. no way to specify the output name in tool and it has many putputs but I am intereseted in one only. > > Actually I dont know to use this attribute from_work_dir and I cant find help. can u help plz? > > ________________________________________ > From: Dannon Baker [dannonbaker(a)me.com] > Sent: Monday, December 05, 2011 1:54 PM > To: Toqa Manasrah > Cc: galaxy-dev(a)bx.psu.edu dev; Jennifer Jackson; galaxy-user > Subject: Re: [galaxy-dev] Urgent help needed > > Toqa, > > Just to make sure I've understood your question: the problem is that a tool that you're trying to wrap doesn't provide a way to specify a particular output filename? Take a look at the from_work_dir attribute of a <data> element. > > Or are you asking how to define outputs in general? For that, see the wiki at http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Coutputs.3E… > > Lastly, for new questions please start new email threads, instead of replying to old ones. This makes it much easier for us to track our responses and make sure people get appropriate help. > > -Dannon > > > On Dec 5, 2011, at 1:26 PM, Toqa Manasrah wrote: > >> Hi Everybody, >> I am running a tool which is not defining the output file name as parameter. >> the tool is running and I can see the output file in the history directory but not in the history frame. >> >> how could I solve the problem? I should see and download the output file from the history but I see an error message instead. the error message not a real error but rather galaxy cannot figure out which file to display? >> >> your help is appreciated, >> >> >> ________________________________________ >> From: Jennifer Jackson [jen(a)bx.psu.edu] >> Sent: Tuesday, November 08, 2011 8:22 AM >> To: Toqa Manasrah >> Cc: Galaxy Dev >> Subject: Re: [galaxy-dev] Installing Galaxy on local server >> >> Hello Tuqa, >> >> Galaxy will not stop by itself. That said, you will want to monitor the >> instance and any associated cluster, just as you would any other >> important processes. >> >> Please send all follow-up with a cc to the mailing list ("reply-all"). >> The development community is a great resource for administrative advice >> and they may have more to add to this reply, i.e. how stable they have >> found Galaxy to be under various conditions. >> >> Best, >> >> Jen >> Galaxy team >> >> On 11/8/11 4:01 AM, Toqa Manasrah wrote: >>> Thank you so much, >>> >>> I did the installation and I ran it under my server. >>> I have one thing concerning about now. if u can help plz. >>> >>> how to run Galaxy in such way that the html interface will continuously working and available for all users visiting the page? >>> >>> i mean how to guarantee that the "sh run.sh" will keep running without any interrupt? >>> >>> Thank you, >>> Tuqa >>> >>> ________________________________________ >>> From: Jennifer Jackson [jen(a)bx.psu.edu] >>> Sent: Thursday, November 03, 2011 6:47 PM >>> To: Toqa Manasrah >>> Cc: galaxy-dev(a)bx.psu.edu >>> Subject: Re: [galaxy-dev] Installing Galaxy on local server >>> >>> Hello Toqa, >>> >>> From your description, this sounds like a local installation for the >>> Galaxy instance itself. Then you can add on a cluster (local or cloud) >>> as needed. >>> >>> Help is under "Deploy Galaxy" at http://wiki.g2.bx.psu.edu/ >>> Also see the "Tool Shed" at http://wiki.g2.bx.psu.edu/Tool%20Shed >>> >>> Hopefully the documentation is helpful, >>> >>> Best, >>> >>> Jen >>> Galaxy team >>> >>> On 11/3/11 9:40 AM, Toqa Manasrah wrote: >>>> Hello, >>>> >>>> I am a graduate student at GSU. I am looking for installing GALAXY on >>>> our local server http://alla.cs.gsu.edu/~software. as a result I like my >>>> server homepage start with the GALAXY interface, like this one: >>>> http://rna1.engr.uconn.edu:7474/ . after that I wish to integrate some >>>> software tolls that we developed in ourdepartment. my primary question >>>> is which option Ihave to choose: Local or cloud? >>>> >>>> looking forward for your help and directions. >>>> >>>> Thank you. >>>> >>>> Tuqa >>>> >>>> >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> >>>> http://lists.bx.psu.edu/ >>> >>> -- >>> Jennifer Jackson >>> http://usegalaxy.org >>> http://galaxyproject.org/wiki/Support >>> >> >> -- >> Jennifer Jackson >> http://usegalaxy.org >> http://galaxyproject.org/wiki/Support >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/

2 1

DB schema upgrade problem (79 -> 85)
by Leon Mei 06 Dec '11

06 Dec '11

Dear all, Today I upgraded our NBIC server from an older release (fetched on July 5th, 2011) to the latest version in galaxy-dist. After execute "hg update" and "sh manage_db upgrade" and merge some local configurations, I successfully restarted the server. However once I access the server from a web browser, Galaxy crashed. Log file complains that some tables don't exist: == ProgrammingError: (ProgrammingError) (1146, "Table 'galaxy.user_quota_association' doesn't exist") u'SELECT user_quota_association.id AS user_quota_association_id, user_quota_association.user_id AS user_quota_association_user_id, user_quota_association.quota_id AS user_quota_association_quota_id, user_quota_association.create_time AS user_quota_association_create_time, user_quota_association.update_time AS user_quota_association_update_time \nFROM user_quota_association \nWHERE %s = user_quota_association.user_id' [1L] == A check in mysql confirms that only the following 3 tables were added during the migration from schema 79 to 85: * default_quota_association * quota * tool_shed_repository The following 9 tables are missing: cloud_*, user_quota_association, group_quota_association Has anyone experienced a similar problem? Any hint on how to solve this is highly appreciate! Thanks, Leon -- Hailiang (Leon) Mei Netherlands Bioinformatics Center BioAssist NGS Taskforce - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing Skype: leon_mei Mobile: +31 6 41709231

2 4

Re: [galaxy-dev] [galaxy-bugs] Problem with indexing a genome through tophat on a local copy of Galaxy...
by Jeremy Goecks 06 Dec '11

06 Dec '11

Hi David, There should be additional information in the galaxy database about why the job failed; take a look at stderr column of the failed job using some SQL like this: -- select * from job where state='error' and tool_id='tophat' and stderr like '%indexing reference%' order by id desc; -- If that doesn't work, try something simpler: -- select * from job where state='error' and tool_id='tophat' order by id desc; -- What are the errors that you see? Finally, please direct future questions about local Galaxy installations to galaxy-dev (cc'd) so that the community can learn from and participate in discussions. Thanks, J. On Dec 5, 2011, at 10:09 AM, David Matthews wrote: > Hi, > > We have a local version of Galaxy here at Bristol and it is very nice. However, it does not like to run TopHat on large genomes (i.e. Human) but is happy with small ones (e.g. viruses). The genome we have is hg19 but missing chr Y and this genome works fine on the PSU version. However, we seem to be hitting a wall when we run it here and it seems to relate to the indexing part right at the begining. Here is the error output we see in the red box: > > An error occurred running this job:Settings: > Output files: "/tmp/tmppA7Hrk/dataset_942.*.ebwt" > Line rate: 6 (line is 64 bytes) > Lines per side: 1 (side is 64 bytes) > Offset rate: 5 (one in 32) > FTable chars: 10 > Strings: unpacked > Max bucket size: default > Max bucket size, sqrt m > > And here is the stuff we see when we click on the green bug (the second set is just the last bit of a long list of stuff): > > > Error indexing reference sequence > Returning block of 520230381 > Getting block 6 of 7 > Reserving size (531691900) for bucket > Calculating Z arrays > Calculating Z arrays time: 00:00:00 > Entering block accumulator loop: > 10% > 20% > 30% > 40% > 50% > 60% > 70% > 80% > 90% > 100% > Block accumulator loop time: 00:00:59 > Sorting block of length 346541282 > (Using difference cover) > Sorting block time: 00:06:04 > Returning block of 346541283 > Getting block 7 of 7 > Reserving size (531691900) for bucket > Calculating Z arrays > Calculating Z arrays time: 00:00:00 > Entering block accumulator loop: > 10% > 20% > 30% > 40% > 50% > 60% > 70% > 80% > 90% > 100% > Block accumulator loop time: 00:00:40 > Sorting block of length 408062248 > (Using difference cover) > Sorting block time: 00:07:08 > Returning block of 408062249 > Exited Ebwt loop > fchr[A]: 0 > fchr[C]: 837200431 > fchr[G]: 1417119193 > fchr[T]: 1997326330 > fchr[$]: 2835690133 > Exiting Ebwt::buildToDisk() > Total time for backward call to driver() for mirror index: 01:16:32 > TopHat v1.2.0 > We are running on our HPC and have directed the job to land on a whole node with 8GB of ram on board. Any ideas why the index is failing? > > > Best Wishes, > David. > > __________________________________ > Dr David A. Matthews > > Senior Lecturer in Virology > Room E49 > Department of Cellular and Molecular Medicine, > School of Medical Sciences > University Walk, > University of Bristol > Bristol. > BS8 1TD > U.K. > > Tel. +44 117 3312058 > Fax. +44 117 3312091 > > D.A.Matthews(a)bristol.ac.uk > > > > > >

2 3

Can't view data files from published histories, only imported histories
by Liisa Koski 06 Dec '11

06 Dec '11

Hi, I found a weird bug. I am trying to view data files by clicking on the 'eye' icon from a published history on my local galaxy installation. When I click on the eye I get a 'Server Error' and in the log file I get the following. Error - <type 'exceptions.NameError'>: global name 'data' is not defined URL: http://domain:8080/u/user/d/8bdb720fee635874 File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File 'galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File 'galaxy_dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File 'galaxy_dist/lib/galaxy/web/controllers/dataset.py', line 693 in display_by_username_and_slug trans.response.set_content_type( data.get_mime() ) NameError: global name 'data' is not defined If I import the history into my own history pane I can click on the eye and see the data with no errors. Any ideas how I can see the data from the published histories? Thanks, Liisa Liisa Koski Bioinformatics Programmer

2 6

possibly weird config requests...
by Smithies, Russell 05 Dec '11

05 Dec '11

This may be a list of weird requests but our Galaxy server runs internally and is only accessible to our users so I'd like to make it a bit easier to manage. I could customize a copy but hopefully some of the "features" I'm after are already available. If not, I'm happy to work on them and integrate if others think they'll be useful. * Rather than email address as usernames, we'd rather just use our corporate logins (lastname, first initial, eg.mine is smithiesr) Or can I have an alias list (like samba)? * Is there a bulk user creation script? Rather than have everyone create their own account, I'd rather do them all at once. It's not that I don't trust users but... * Can I have every user's home dir automagiclly added as a data dir? eg. I'd like to have /home/smithiesr available. Would be nice if this was part of a bulk useradd script * Any chance of AD integration? I have no idea if Python plays well with Active Directory and/or LDAP. * Is it possible to add new tools without restarting Galaxy? I know tool configs can be reloaded from the admin console but I'd like our users to be able to incorporate their existing scripts as tools. Would a local tool-shed make this easier? * We use NTLM for authentication on most of our internal sites - am I going to have problems getting Galaxy/Python to authenticate? A few years ago we ran into the same requirement with a Java app so ended up hacking together a script that passed SSO creds to Internet Exploiter for authentication. Sorry if some of these sound dumb or are already implemented, I'm still reading thru the docs. Thanx, Russell Smithies Infrastructure Team T 03 489 9085 M 0274 734 600 E russell.smithies(a)agresearch.co.nz Invermay Agricultural Centre Puddle Alley, Private Bag 50034, Mosgiel 9053, New Zealand T +64 3 489 3809 F +64 3 489 3739 www.agresearch.co.nz ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

3 3