- galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Existing efforts to convert the QIIME pipeline to Galaxy?
by Hollander, Mattias de 02 Jan '12

02 Jan '12

Thanks Jim for giving this extensive update on how to improve the Qiime wrappers. We had contact about the Qiime wrappers before, and I have been using them in Galaxy on a weekly basis. Therefore I can say they have been proven to be very useful. I mainly edited the parameter and output sections of some configuration files, but all still use the wrapper script. The most important scripts are the split libraries and the workflow scripts (pick_otus_through_otu_table, alpha/beta diversity workflows). I use the Qiime scripts in a Galaxy Cloudman environment adjusted to a multicore server. Setup on EC2 shouldn't be a problem, although you need to place the requirement tag for all the needed dependecies on top of the xml file. I will have a look at the suggestions and I am willing to help to improve Qiime support for galaxy either by working on the configuration files or testing them out. So please keep me in the loop... Greetings, Mattias -----Original Message----- From: galaxy-dev-bounces(a)lists.bx.psu.edu on behalf of galaxy-dev-request(a)lists.bx.psu.edu Sent: Sat 12/31/2011 6:00 PM To: galaxy-dev(a)lists.bx.psu.edu Subject: galaxy-dev Digest, Vol 66, Issue 26 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.bx.psu.edu To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.bx.psu.edu You can reach the person managing the list at galaxy-dev-owner(a)lists.bx.psu.edu When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Re: Existing efforts to convert the QIIME pipeline to Galaxy? (Jim Johnson) ---------------------------------------------------------------------- Message: 1 Date: Thu, 29 Dec 2011 14:02:04 -0600 From: Jim Johnson <johns198(a)umn.edu> To: Jose Carlos Clemente <jose.clemente(a)gmail.com>, "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> Cc: Greg Caporaso <gregcaporaso(a)gmail.com>, Rob Knight <Rob.Knight(a)colorado.edu>, Jesse Stombaugh <jistombaugh(a)gmail.com>, Zuzolo Amanda <azuzolo(a)gmu.edu>, Antonio Gonz?lez <antgonza(a)gmail.com>, Gillevet Patrick <pgilleve(a)gmu.edu>, jj(a)umn.edu, Daniel McDonald <wasade(a)gmail.com> Subject: Re: [galaxy-dev] Existing efforts to convert the QIIME pipeline to Galaxy? Message-ID: <4EFCC73C.7010709(a)umn.edu> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" It is easiest to generate tools for galaxy when the applications or scripts can take arbitrarily named input files and generate output to given path names. Input directories, output directories are very convenient on the command line, but more of a challenge when crafting a galaxy tool. That said, many applications require a wrapper script to work with in galaxy. Thank you for the consistent script_info[] help/usage syntax in the qiime scripts, which enabled me to generate a skeleton galaxy tool_config file for each qiime script. I had some time last spring to work on integrating qiime into galaxy. Unfortunately, I haven't had any time since to work on this. I put those partial results on the Galaxy Tool Shed: http://toolshed.g2.bx.psu.edu/ There's a continuing effort at George Mason University to incorporate qiime into galaxy tools, so you may want to ask them what they need. I started by generating galaxy tool_config files, e.g. align_seqs.xml, by using python to get the script_info[] from the qiime script: $ cat generate_tool_config.bash #!/usr/bin/env bash python $1 > ${1%.*}.help cat tool_template.txt | sed "s/__TOOL_BINARY__/${1}/" | python -i $1 -h > ${1%.*}.log (I'll attach tool_template.txt ) This generated skeleton tool_config .xml files that I could then edit as needed. ( http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax ) I originally was calling all qiime scripts from a tool wrapper: qiime_wrapper.py But, if a script can be called with any input filepaths and write its results to any filepaths, and only writes to STDERR when it fails, then you could call that script directly. When should you use a tool_wrapper or call the qiime script directly? Many of the qiime scripts could probably be called directly, especially if it can be called with arbitary input/output file pathnames. The reasons for using a tool wrapper may be if input/output needs to be manipulated, moved, renamed in order to be used by the qiime script. You'll also need a tool wrapper if the names or number of the output files can not be determined from the parameter settings. ( http://wiki.g2.bx.psu.edu/Admin/Tools/Multiple%20Output%20Files ) If your tool relies on a file ext to determine a format, you'll have to rename the input. ( Galaxy dataset pathnames will look something like: /<your_galaxy_file_path>/072/dataset_72931.dat ) The format/type of a dataset is stored in its metadata, so the tool_config can use that information, especially if a script can take muliple alternative input formats. A tool_wrapper can also be used to manage the stdout or stderr from a tool. Galaxy currently interprets any output on stderr as a failure. A couple changes in galaxy should make somethings easier than when I first attempted this: - galaxy now accepts dataset requests with sub directories. ( https://bitbucket.org/galaxy/galaxy-central/issue/494/support-sub-dirs-in-e… ) That means that output HTML files with links into sub directories can be left intact, with the html copied to the output dataset and the linked files to its "extra_files_path". - if you know the pathname of an output relative to the working directory, galaxy can copy it automatically to the output dataset using the from_work_dir attribute. ( see example in: https://bitbucket.org/galaxy/galaxy-central/src/21b645303c02/tools/ngs_rna/… ) Datatypes You may want to create new datatypes to make it easier for the user to correctly select inputs to a tool from previous outputs. For example, the qiime mapping file is a tabular file with specific requirements. I put a 'qiimemapping' datatype in lib/galaxy/datatypes/metagenomics.py and datatypes_conf.xml so an input could generate a select list containing only qiimemapping datasets rather than all tabular ones. Generating a configfile You can generate configfiles in the galaxy tool_config .xml file. The configfile is generated by the Cheetah interpreter just as the commandline is. see: alpha_rarefaction.xml The qiime_wrapper.py was patterned after the mothur_wrapper.py with some of the same wrapper params to handle run time determined output (perhaps not needed): --galaxy_datasets a comma separated list of regex:output_dataset the wrapper searches the working_dir and copies the file that matches the regex to the outout dataset if the exact pathname is known, use the "from_work_dir" attribute instead --galaxy_datasetid would be an output dataset id that would be used to dynamically create additional new datasets at job termination ( http://wiki.g2.bx.psu.edu/Admin/Tools/Multiple%20Output%20Files "Number of Output datasets cannot be determined until tool run") --galaxy_new_datasets a comma separated list of regex:datatype used to dynamically create additional new datasets at job termination --galaxy_new_files_path the galaxy dir for dynamically generated output datasets On 12/14/11 12:11 PM, Jose Carlos Clemente wrote: > Hi Jim, > > we just had a meeting to discuss ideas for a QIIME GUI, and were wondering how far did you manage to get with your plan to develop a QIIME wrapper for Galaxy. Do you have something working that we could take a look at? Were there any particular issues with the integration? We might have some bandwidth to work on this over next year, but thought it would be good to check with you first. > > Thanks, > Jose > > > On Mon, Feb 7, 2011 at 20:24, Rob Knight <Rob.Knight(a)colorado.edu <mailto:Rob.Knight@colorado.edu>> wrote: > > Hi Jim, > > Just to echo Greg's comments; it would be excellent to connect QIIME to Galaxy: let me know what is necessary from my end to ensure that this happens. > > Thanks, > Rob > > On Feb 7, 2011, at 12:30 PM, Greg Caporaso wrote: > >> Hi Jim, >> This is great! We're very enthusiastic about hooking up QIIME with Galaxy, and definitely encourage you to work on it. It's something we've discussed doing, but no one in our lab is actively working on it. One particular use case that I've been thinking about would be to set up Galaxy/QIIME in our QIIME EC2 images so our EC2 users could access their images via the Galaxy web server. Are you planning to release all of the xml files under an open source/free distribution model? >> >> I'm definitely interested in progress updates. Let me know if there are ways that we can help out, for example by giving you a list of what we believe are the most commonly used QIIME scripts so you can focus your efforts on those. I'm also cc'ing some of the folks in the lab here who are interested in Galaxy integration. >> >> Greg >> >> On Mon, Feb 7, 2011 at 10:36 AM, Jim Johnson <johns198(a)umn.edu <mailto:johns198@umn.edu>> wrote: >> >> Greg, >> >> Tim te Beek let me know that you have an interest in incorporating Qiime into Galaxy. I'm currently working on that in support of a grant at the University of Minnesota. I'll keep you apprised of my progress if you are interested. >> >> Thanks, >> >> JJ >> >> James Johnson >> University of Minnesota Supercomputing Institute >> 117 Pleasant St. SE >> Minneapolis, MN 55455 >> Email: jj(a)umn.edu <mailto:jj@umn.edu> >> >> >> >> -------- Original Message -------- >> Subject: Re: [galaxy-dev] Existing efforts to convert the QIIME pipeline to Galaxy? >> Date: Mon, 07 Feb 2011 11:15:38 -0600 >> From: Jim Johnson <johns198(a)umn.edu> <mailto:johns198@umn.edu> >> Reply-To: jj(a)umn.edu <mailto:jj@umn.edu> >> Organization: MSI University of Minnesota >> To: Tim te Beek <tim.te.beek(a)nbic.nl> <mailto:tim.te.beek@nbic.nl> >> >> >> >> Tim, >> >> Sorry to hear we won't be doing this together. I'm finding some sporadic time to work on this. I also put my Mothur metagenomics wrapper code on http://community.g2.bx.psu.edu/ >> >> I looked at the initial work you did, and combined that with the script design ideas I used with Mothur. I'm writing a single script to call all the Qiime scripts, and generating a tool config (.xml) for each qiime script. Since the qiime scripts all contain a rather consistent script_info dictionary in the code, I wrote a script to generate an initial tool config from that script_info. I'm now starting the tedious process of editing those for each script. I'll attach the few I've started on. If you have any users that would able to beta test this when I have this more complete, please let me know. >> >> Thanks, >> >> JJ >> >> >> On 2/7/11 10:15 AM, Tim te Beek wrote: >>> Hi Jim, >>> >>> Sorry for the lack of feedback / updates on my part in converting QIIME to Galaxy. Following the initial difficulties with the conversion, the support project was put on hold pending further discussion. This morning the clients decided to forgo the Galaxy implementation, as their implementation would only ever be used by a single (technical) user, reducing the benefit of having a user friendly graphical interface. >>> >>> So I'm sorry to say I won't be able to contribute to the conversion, although your wrapper script did look like a viable solution to work around the unknown output file problems. Should you be able to complete the conversion yourself and be willing to share your implementation, I would still very much like to know. >>> >>> Perhaps the following could still be of some use to you: I contacted the QIIME development group about adding output file name arguments to the scripts, as can be seen here: >>> http://sourceforge.net/tracker/?func=detail&atid=1157167&aid=3164813&group_… <http://sourceforge.net/tracker/?func=detail&atid=1157167&aid=3164813&group_…> >>> Although it appears they wont be able to add output file name script parameters, they did say they're interested in Galaxy, and would be willing to help if there's a need for that. >>> >>> Additionally, we at NBIC do have a team that's more experienced at using Galaxy and converting tools to work in Galaxy, and if you want I could put you in contact with them about supporting this effort. In informal discussions they did seem to look favorable upon your solution, so perhaps that could be extended into a collaboration. >>> >>> Best regards, >>> Tim >>> >>> On Fri, Jan 14, 2011 at 20:01, Tim te Beek <tim.te.beek(a)nbic.nl <mailto:tim.te.beek@nbic.nl>> wrote: >>> >>> Ah ok, that's still fine. Thanks for the archive! I'll see what I can learn from how you've handled running command and handling in- & output for the non-trivial scripts, and apply that to the QIIME scripts. >>> Should you wish so I could get back to you if there's any significant new developments on my part in moving the QIIME scripts to Galaxy, other than the Trac / SVN changes. >>> >>> Have a good weekend! >>> Tim >>> >>> >>> On Fri, Jan 14, 2011 at 19:12, Jim Johnson <johns198(a)umn.edu <mailto:johns198@umn.edu>> wrote: >>> >>> I should probably be more clear, in that mothur and qiime overlap in functionality and in the algorithms they use. >>> They do use different data formats. >>> >>> >>> >>> On 1/14/11 12:02 PM, Tim te Beek wrote: >>>> Hi Jim, >>>> >>>> If you could send me the Mothur package, that would be great. If there's indeed as much overlap then it could be a great way to kick-start the QIIME pipeline conversion development. I can look into it starting monday, with hopefully nothing else to distract me the whole week. >>>> >>>> Best regards, >>>> Tim >>>> >>>> >>>> On Fri, Jan 14, 2011 at 18:49, Jim Johnson <johns198(a)umn.edu <mailto:johns198@umn.edu>> wrote: >>>> >>>> Tim, >>>> >>>> I just started looking at qiime myself. We are hoping to release a galaxy server to the University of Minnesota researchers in the next month, >>>> so there has been much to do in preparation. I'll take a look at what you've done. Thanks. >>>> >>>> I finished a galaxy interface to the Mothur metagenomics package http://www.mothur.org/wiki/Main_Page. It appears to have a lot of overlap with Qiime. >>>> I generated a number of datatypes, and made use of many of the tool config concepts that are available in galaxy. >>>> If you are interested, I can send that to you: mothur.tar.gz (55108 bytes). I hope to submit it tohttp://community.g2.bx.psu.edu/ in the next month. >>>> >>>> Regards, >>>> >>>> JJ >>>> >>>> >>>> On 1/14/11 9:52 AM, Tim te Beek wrote: >>>>> Hi Jim, >>>>> >>>>> Sorry to only respond just now: I've been tied up in other projects, so have not been able to get any work done on this up to today. >>>>> >>>>> What I currently have is not much, since my progress has been slowed by my unfamiliarity with both Galaxy and the QIIME tools (I'm working on this as an outsider for a small support project). But what I do have I've made available through the following Trac / SVN instance: >>>>> _ >>>>> _ >>>>> >>>>> https://trac.nbic.nl/brs2010p26/browser/trunk/galaxy_dist/tools/qiime >>>>> >>>>> >>>>> These configuration files so far can run: >>>>> >>>>> * check_id_map >>>>> * split_libraries >>>>> * pick_otus_through_otu_table >>>>> >>>>> For the first two the output is correctly directed to the Galaxy history, the last one generates quite a large set of output files which might have to be made available otherwise (the documentation on this gives a few options: https://bitbucket.org/galaxy/galaxy-central/wiki/ToolsMultipleOutput)) >>>>> >>>>> I had some trouble working around the fact that most QIIME scripts do not allow one to specify output filenames on the command line, which seems to be a requirement for Galaxy to import files into the history pane. What I've done is written a bash file to accompany each XML file, that expects as first command line argument the command that should be evaluated, and as additional arguments the output filenames expected by Galaxy. After running a tool I then move whatever output files were created to the paths expected by Galaxy. I must say I'm not (at all) sure if this current approach is something to recommend, or if better alternatives are available. Any feedback on this would be highly appreciated. >>>>> An additional source of pain was that the split_libraries script requires specific file extensions for it's fasta & quality files, which required passing them separately and symbolic linking them as .fna & .qual files. >>>>> >>>>> So far I've not taken the time to introduce new QIIME specific data types, but this could be something to consider given the large amount of 'tabular' input and output files. >>>>> >>>>> I hope this email and the scripts linked above can still be of some use to you (or vice-versa), but in either case any feedback or collaboration would be much appreciated. >>>>> >>>>> Best regards, >>>>> Tim >>>>> >>>>> >>>>> On Wed, Nov 24, 2010 at 15:48, Jim Johnson <ctoftey(a)comcast.net <mailto:ctoftey@comcast.net>> wrote: >>>>> > Tim, >>>>> > >>>>> > I hope to also look at incorporating QIIME into our local Galaxy instance at >>>>> > the University of Minnesota, but probably won't be able to start for a >>>>> > couple weeks. It would be good to develop that in coordination with others. >>>>> > >>>>> > I just finished incorporating "Mothur" metagenomics suite >>>>> > http://www.mothur.org/ (Dr. Patrick Schloss, Department of Microbiology & >>>>> > Immunology at The University of Michigan) into our Galaxy server at the >>>>> > University of Minnesota. I hope to contribute that to >>>>> > http://community.g2.bx.psu.edu/ after some testing by our researchers. If >>>>> > the Galaxy wrappings for Mothur are of any interest to you, I can send you a >>>>> > copy any time. >>>>> > >>>>> > Thanks, >>>>> > >>>>> > JJ >>>>> > >>>>> > James E Johnson >>>>> > Minnesota Supercomputing Institute >>>>> > University of Minnesota >>>>> > >>>>> > >>>>> > On Nov 23, 2010 at 07:22 AM, Tim te Beek wrote: >>>>> > >>>>> >> Hello all, >>>>> >> >>>>> >> Is anyone aware of any existing efforts to port the QIIME sequencing >>>>> >> pipeline (http://qiime.sourceforge.net/) to Galaxy? I would like to run >>>>> >> QIIME analyses through Galaxy to get better control of >>>>> >> intermediate processing steps, but before I start to convert (a subset of) >>>>> >> some 90 scripts, I'd first like to make sure this has not been done before >>>>> >> by anyone willing to share their work. >>>>> >> >>>>> >> So: has anyone converted the QIIME pipeline to Galaxy before, and would >>>>> >> they >>>>> >> be willing to share their scripts? >>>>> >> >>>>> >> Best regards, >>>>> >> Tim >>>>> > >>>>> > _______________________________________________ >>>>> > galaxy-dev mailing list >>>>> > galaxy-dev(a)lists.bx.psu.edu <mailto:galaxy-dev@lists.bx.psu.edu> >>>>> > http://lists.bx.psu.edu/listinfo/galaxy-dev >>>>> > >>>>> >>>> >>> >>> >>> >> >> >> > >

1 0

Zip file problems + RE: Excel upload problems
by Lukasse, Pieter 30 Dec '11

30 Dec '11

I have the same remark regarding zip files: there should be an easy way to configure a file type as a "non-standard zip": a zipped file that Galaxy just leaves untouched instead of unzipping it and passing the first file. For now I had to make 2 changes: 1- In datatypes_conf.xml : add the type extension for your type (in my case mscfileset): <datatype extension="mscfileset" type="galaxy.datatypes.data:Data" mimetype="application/zip" display_in_upload="true"/> 2- In the tools/data_source/upload.py code add something like the following marked in red below: [cid:image001.png@01CCBFE9.A17275E0] I think it should be possible to adjust the upload.py code in a generic way (see " if is_zipped : " block). Regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/> From: galaxy-dev-bounces(a)lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Siemen Henk Sikkema Sent: donderdag 15 december 2011 11:14 To: Peter Cock Cc: Greg Von Kuster; galaxy-dev(a)bx.psu.edu Subject: Re: [galaxy-dev] Excel upload problems Hi All, Thanks for the responses! I just got around to trying the solution by Joe Cruz. This did the trick for xls files, so that's the good news! The slightly less good news is that the same does not hold for xlsx files. Those files are pure zip files containing lots of internal xml stuff. Even when following the suggested procedure, Galaxy insists that it knows how to handle those files: by unpacking them and returning the first file from the zip (in this case a pretty useless xml file). On a more general note, having to do these tricks can restrict sharing of new tools a bit in my opinion. It would be great if there would be an easy way to tell Galaxy: "leave files of this type as is, I can handle it" :P Siemen On Thu, Dec 8, 2011 at 10:18 PM, Peter Cock <p.j.a.cock(a)googlemail.com<mailto:p.j.a.cock@googlemail.com>> wrote: > On Fri, Dec 9, 2011 at 3:35 AM, Peter Cock <p.j.a.cock(a)googlemail.com<mailto:p.j.a.cock@googlemail.com>> wrote: >> >> P.S. Why do you call your class XIs, surely Excel would be >> clearer? I've only just realised Xls is title case xls (as in the extension), I read it at the title case of xis (which made no sense). I still think using Excel as the class name would be clearer. On Thu, Dec 8, 2011 at 9:01 PM, Ross <ross.lazarus(a)gmail.com<mailto:ross.lazarus@gmail.com>> wrote: > +1 > I'd agree that calling it excel is suitably wry - because it doesn't IMHO. > Please name the new binary datatype something other than xls because > many (of my) tools create outputs with that extension and it's a > subclass of tabular here - the .xls extension fools excel into loading > them automagically. I don't entirely agree with your logic, but I would agree that as the format name it would be consistent with most of the other Galaxy formats to go for a longer name (e.g. "excel") over a short 3 letter extension (here "xls"). However, does this matter for the extension that Galaxy gives on downloading the dataset, or is that all done via the mime type? We want it to be easy for people to download an Excel file from Galaxy and open it, which means getting the extension right under Windows. Peter

1 1

amplicon reads(with errors)
by Bassam Tork 30 Dec '11

30 Dec '11

Dear All, Is there a way to generate 454 simulated amplicon reads with errors using galaxy. Happy New Year, Bassam Tork.

2 1

defining genome assembler in galaxy
by Bassam Tork 29 Dec '11

29 Dec '11

*Hello,* I am interested in defining vispa tool on galaxy *(http://alla.cs.gsu.edu/~software/VISPA/vispa.html).* *we usually run vispa using main.bash file as follows ./main.bash readFile referenceFile threads n t where: threads: number of threads n:number of mismatches t:mutation rate the output file name,we are interested in is not passed as parameter, but its named as follows: if inupt file name= readFile.fas then output file name is readFile_I_2_20_CNTGS_DIST0_EM20.txt ('readFile'+'_I_2_20_CNTGS_DIST0_EM20.txt') *My idea was to pass all above including main.bash as parameters to the python wrapper (called run.py, attached), as follows: where the last parameter sys.argv[7] is the output file, specified by xml file. But galaxy was running for more than 3 hours, although it should take only 7-10 minutes on our server with no results. Vispa should write its output to dataset_192_I_2_20_CNTGS_DIST0_EM20.txt , where dataset_192 .txt is the reads file.But dataset_192_I_2_20_CNTGS_DIST0_EM20.txt did not appear in galaxy-dist/database/files/000. run.py: simply coppies data from dataset_192_I_2_20_CNTGS_DIST0_EM20.txt to galaxy output file, but nothing is coppied since dataset_192_I_2_20_CNTGS_DIST0_EM20.txt was not created. How could the results appear?Could somebody help me ? *Your Help Is Highly Appreciated, Bassam Tork.* *PhD - GSU*

2 3

Re: [galaxy-dev] Written tutorial
by Jeremy Goecks 29 Dec '11

29 Dec '11

Jeff, The screencasts are in place of written tutorials; we've received feedback in the past that screencasts are often easier to understand and use. We'll try to locate the datasets used in these tutorials. Thanks, J. On Dec 29, 2011, at 9:43 AM, Rosenfeld, Jeffrey wrote: > Hi Jeremy, > > Thank you for the link. I have already seen those files, but I was > hoping for written instructions and the files for the video tutorials. > > Jeff > > On 12/29/2011 09:19 AM, Jeremy Goecks wrote: >> Jeff, >> >> Here are some tutorials on basic NGS analyses in Galaxy: >> >> http://main.g2.bx.psu.edu/u/james/p/exercises >> >> Best, >> J. >> >> On Dec 22, 2011, at 11:54 AM, Rosenfeld, Jeffrey wrote: >> >>> Hi, >>> >>> I am trying to teach some people to use Galaxy, and I was looking >>> for a written tutorial that went through the basics of taking sequence >>> reads, gromming and trimming them and then doing alignment. I see that >>> you have this type of exercise as screencasts: >>> http://main.g2.bx.psu.edu/u/aun1/p/ngs-analysis-service , but this is >>> not something written where users can follow instructions and do it >>> themselves. Also, there is no indication as to where to download the >>> example files used in the screencasts. >>> >>> Thanks, >>> >>> Jeff >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/

1 0

Re: [galaxy-dev] Toolshed
by Greg Von Kuster 29 Dec '11

29 Dec '11

Hello Ted, Please replay-all to back-and-forth communication like this so that the thread stays on the mailing list. You can also search the Galaxy mailing lists for answers to your questions - see http://galaxy.psu.edu/search/mailinglists/. See my answers below. On Dec 28, 2011, at 11:46 PM, Ted Goldstein wrote: > Greg, > Thanks. I reinstalled from galaxy-centlral and it works fine. I'd rather be on the dev branch. > > Two other topics: > 1. I would like to take a crack at > https://bitbucket.org/galaxy/galaxy-central/issue/554/show-column-names-hea… > Can you give me a clue about which files to work on? I haven't been working in this area for quite some time, and have not been involved in the discussion on the mail thread and bitbucket ticket for this, so I'm not the best person to answer your questions. That being said, if I understand the issue correctly, I would start by looking at the ColumnListParameter() class in ~/lib/galaxy/tools/parameters/basic.py. > > > 2. When I hit control-c, the server process stops answering but doesn't terminate. Is this correct? See this thread (there are others as well if you search the mailing list archives with various search strings): http://galaxy-users-list-archive.2308625.n4.nabble.com/restart-server-td413… > > Thanks! > Ted > > On Dec 28, 2011, at 5:30 AM, Greg Von Kuster wrote: > >> Hello Ted, >> >> I assume you cloned your local Galaxy instance from our distribution repo at https://bitbucket.org/galaxy/galaxy-dist. >> >> Unfortunately, in order to make use of the features that enable communication between your local Galaxy instance and a Galaxy tool shed, you have to update your local Galaxy instance from our Galaxy central repo at https://bitbucket.org/galaxy/galaxy-central. This is because the distribution repo is not updated as often as is necessary to keep current with the rapidly evolving tool shed code base and feature set. This issue should be eliminated at some point. >> >> >> On Dec 27, 2011, at 11:54 PM, Ted Goldstein wrote: >> >>> Hello, >>> I just installed Galaxy and am very impressed. But I am having problems adding in tools from the toolshed. When I click on adding a tool such as the Venn Diagram tool, it brings up the following message (even though I never installed it before: >>> >>> ============ >>> Found >>> >>> The resource was found at xyzzy.ucsc.edu:1234/admin_toolshed/install_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=9956e395db5a106ba074ae610c525fa17cec3b14:7b2276656e6e5f6c697374223a205b22447261772056656e6e204469616772616d2028504446292066726f6d206c69737473222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f70657465726a632f76656e6e5f6c697374222c2022626166373033316434373065225d7d&includes_tools=True; you should be redirected automatically. >>> WSGI Server >>> ================= >>> >>> >>> The page never redirects. As well the URL is malformed (missing the "http:"). If I fabricate this by hand. E.g. >>> http://xyzzy.ucsc.edu:1234/admin_toolshed/install_repository?tool_shed_url=… >>> >>> ============ >>> Not Found >>> >>> The resource could not be found. >>> No route for /admin_toolshed/install_repository >>> ============ >>> The log file shows: >>> .2.3.4 - - [27/Dec/2011:20:33:27 -0700] "GET /admin/browse_tool_sheds HTTP/1.1" 200 - "http://xyzzy.ucsc.edu:1234/admin" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> 1.2.3.4 - - [27/Dec/2011:20:33:30 -0700] "GET /admin/browse_tool_shed?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://xyzzy.ucsc.edu:1234/admin/browse_tool_sheds" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> 1.2.3.4 - - [27/Dec/2011:20:44:28 -0700] "GET /admin_toolshed/install_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=9956e395db5a106ba074ae610c525fa17cec3b14:7b2276656e6e5f6c697374223a205b22447261772056656e6e204469616772616d2028504446292066726f6d206c69737473222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f70657465726a632f76656e6e5f6c697374222c2022626166373033316434373065225d7d&includes_tools=True HTTP/1.1" 404 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> >>> =============== >>> >>> I looked through the mailing list and wiki and I don't see this problem. Could I have misinstalled or misconfigured something? >>> >>> Thanks for the great system! >>> >>> Best regards, >>> Ted >>> >>> >>> Ted Goldstein >>> ted(a)soe.ucsc.edu >>> Ph.D. Candidate, HausslerLab, CBSE, UCSC >>> (650) 743-4141 >>> http://soe.ucsc.edu/~ted >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Written tutorial
by Rosenfeld, Jeffrey 29 Dec '11

29 Dec '11

Hi, I am trying to teach some people to use Galaxy, and I was looking for a written tutorial that went through the basics of taking sequence reads, gromming and trimming them and then doing alignment. I see that you have this type of exercise as screencasts: http://main.g2.bx.psu.edu/u/aun1/p/ngs-analysis-service , but this is not something written where users can follow instructions and do it themselves. Also, there is no indication as to where to download the example files used in the screencasts. Thanks, Jeff

2 1

Possible bug
by Alfonso Núñez Salgado 28 Dec '11

28 Dec '11

As I've written before, I couldn't preview some kind of files and downloading them neither (not just some extensions but all). I don't really know if it's a real bug or not but i've saw other people whith the same problem. To solve it I've just changed the return value at "./lib/galaxy/web/controllers/dataset.py", at the "display" function (or method). In stead of returning the value returned by "open()" function, I do return the value returned by "read()" function. Originaly: ... return open(...) ... New: ... return open(...).read() ... Well, i don't know if its beacause of my system but i wish this can help Saludos Alfonso

1 0

Re: [galaxy-dev] Newbie question for toolshed development
by Greg Von Kuster 27 Dec '11

27 Dec '11

Including a tool in a section of one of the tool configs that populates the Galaxy tool panel is only relevant to the Galaxy instance, not a Galaxy tool shed. If a user installs a tool shed repository from a tool shed to their local Galaxy instance, they can choose a specific section label in which to load the tool into their tool panel at the time of installation. See http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_tool… On Dec 27, 2011, at 4:25 PM, Aaron Quinlan wrote: > Hi Greg, > > In a previous email (below) you mentioned that the tool_conf.xml is not necessary for tool shed repos. Consequently, is there no way to add section labels when a repo includes multiple tools? For example, I'd like to have section labels like the Picard tools have (e.g., "CONVERSION", "QC METRICS...") on the main site. > > Is this possible somehow? > > Best, > Aaron > > On Nov 23, 2011, at 9:51 AM, Greg Von Kuster wrote: > >>> Here is my current repo including just one tool: >>> >>> http://testtoolshed.g2.bx.psu.edu/repository/browse_repositories?sort=name&… >> >> This looks good, but a couple of comments: >> >> 1. There is no longer any use for suite_config.xml - it was used in the old version of the tool shed, but is no longer used. >> >> 2. Including tool_conf.xml is not necessary - nothing is done with it. >> >> 3. Support for data types used by your tools that are not included in the Galaxy distribution will be available soon, so if your tools use them, you can include code files that include classes that support them. More information about this will soon be available in the tool shed wiki, but let me know if you have questions about this. Your current tool doesn't fall into this category. >> >> 4. Make sure that your tool configs include the <requirements> tag sets if the tool has a dependency. This is the tag set that will enable the use of the future fabric scripts discussed above. See the section of our tool config syntax wiki for details: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Crequiremen… > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

jquery / js / python programming for galaxy / bugs, ideally for AWS ec2
by carl crott 23 Dec '11

23 Dec '11

Good morning all, So I'm looking to contribute I'm really interested in the power that amazon integration can bring to these tools and I'm wondering if anyone happens to know of bugs offhand or programming which specifically deals with AWS / EC2. If anyone happens to know of simple things offhand feel free to shout at me! -Carl -- Carl Crott Web Applications Engineer www.black-glass.com 412-610-0600

1 0