December 2011 - galaxy-dev - lists.galaxyproject.org

Problem uploading files from filesystem paths
by Liisa Koski 03 Jan '12

03 Jan '12

Hi, I'm trying to upload data to Data Libraries from filesystems paths as Admin. I get the following error. Any ideas? Thanks in advance, Liisa Traceback (most recent call last): File "/data/Galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/data/Galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, js Job Standard Error Traceback (most recent call last): File "/data/Galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/data/Galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, json_file, output_path ) File "/data/Galaxy/galaxy-dist/tools/data_source/upload.py", line 300, in add_file if datatype.dataset_content_needs_grooming( dataset.path ): File "/data/Galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/data/Galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib64/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1139, in _execute_child raise child_exception OSError: [Errno 13] Permission denied

6 9

Is there a way to pass key =value pairs to the output files in a programmatic way? (Tags? Metadata?)
by Thon deBoer 02 Jan '12

02 Jan '12

Hi, I am trying to pass some key-value pairs to the result of a tool (such as the original sample name) but I cannot find a way to do this in the galaxy XML tool definitions. I am simply trying to retain in all the data files I produce, the name of the original sample which the results are related to. Now I have to follow the history of a result just to see which original data file sample my results came from (and when using hidden files, this is almost impossible, since all results refer to some arbitrary number my data set had ("Tophat on data 30" is not going to help me, if I have 140 samples with all the same name)... Is there some way to pass some key-value pairs to a tools output? Can tags be assigned in the Tool definition? Can I set metadata tags programmatically? Any help would be appreciated... Regards, Thon de Boer, Ph.D. Bioinformatics Guru LinkedIn Profile

4 5

Zip file problems + RE: Excel upload problems
by Lukasse, Pieter 30 Dec '11

30 Dec '11

I have the same remark regarding zip files: there should be an easy way to configure a file type as a "non-standard zip": a zipped file that Galaxy just leaves untouched instead of unzipping it and passing the first file. For now I had to make 2 changes: 1- In datatypes_conf.xml : add the type extension for your type (in my case mscfileset): <datatype extension="mscfileset" type="galaxy.datatypes.data:Data" mimetype="application/zip" display_in_upload="true"/> 2- In the tools/data_source/upload.py code add something like the following marked in red below: [cid:image001.png@01CCBFE9.A17275E0] I think it should be possible to adjust the upload.py code in a generic way (see " if is_zipped : " block). Regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/> From: galaxy-dev-bounces(a)lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Siemen Henk Sikkema Sent: donderdag 15 december 2011 11:14 To: Peter Cock Cc: Greg Von Kuster; galaxy-dev(a)bx.psu.edu Subject: Re: [galaxy-dev] Excel upload problems Hi All, Thanks for the responses! I just got around to trying the solution by Joe Cruz. This did the trick for xls files, so that's the good news! The slightly less good news is that the same does not hold for xlsx files. Those files are pure zip files containing lots of internal xml stuff. Even when following the suggested procedure, Galaxy insists that it knows how to handle those files: by unpacking them and returning the first file from the zip (in this case a pretty useless xml file). On a more general note, having to do these tricks can restrict sharing of new tools a bit in my opinion. It would be great if there would be an easy way to tell Galaxy: "leave files of this type as is, I can handle it" :P Siemen On Thu, Dec 8, 2011 at 10:18 PM, Peter Cock <p.j.a.cock(a)googlemail.com<mailto:p.j.a.cock@googlemail.com>> wrote: > On Fri, Dec 9, 2011 at 3:35 AM, Peter Cock <p.j.a.cock(a)googlemail.com<mailto:p.j.a.cock@googlemail.com>> wrote: >> >> P.S. Why do you call your class XIs, surely Excel would be >> clearer? I've only just realised Xls is title case xls (as in the extension), I read it at the title case of xis (which made no sense). I still think using Excel as the class name would be clearer. On Thu, Dec 8, 2011 at 9:01 PM, Ross <ross.lazarus(a)gmail.com<mailto:ross.lazarus@gmail.com>> wrote: > +1 > I'd agree that calling it excel is suitably wry - because it doesn't IMHO. > Please name the new binary datatype something other than xls because > many (of my) tools create outputs with that extension and it's a > subclass of tabular here - the .xls extension fools excel into loading > them automagically. I don't entirely agree with your logic, but I would agree that as the format name it would be consistent with most of the other Galaxy formats to go for a longer name (e.g. "excel") over a short 3 letter extension (here "xls"). However, does this matter for the extension that Galaxy gives on downloading the dataset, or is that all done via the mime type? We want it to be easy for people to download an Excel file from Galaxy and open it, which means getting the extension right under Windows. Peter

1 1

amplicon reads(with errors)
by Bassam Tork 30 Dec '11

30 Dec '11

Dear All, Is there a way to generate 454 simulated amplicon reads with errors using galaxy. Happy New Year, Bassam Tork.

2 1

defining genome assembler in galaxy
by Bassam Tork 29 Dec '11

29 Dec '11

*Hello,* I am interested in defining vispa tool on galaxy *(http://alla.cs.gsu.edu/~software/VISPA/vispa.html).* *we usually run vispa using main.bash file as follows ./main.bash readFile referenceFile threads n t where: threads: number of threads n:number of mismatches t:mutation rate the output file name,we are interested in is not passed as parameter, but its named as follows: if inupt file name= readFile.fas then output file name is readFile_I_2_20_CNTGS_DIST0_EM20.txt ('readFile'+'_I_2_20_CNTGS_DIST0_EM20.txt') *My idea was to pass all above including main.bash as parameters to the python wrapper (called run.py, attached), as follows: where the last parameter sys.argv[7] is the output file, specified by xml file. But galaxy was running for more than 3 hours, although it should take only 7-10 minutes on our server with no results. Vispa should write its output to dataset_192_I_2_20_CNTGS_DIST0_EM20.txt , where dataset_192 .txt is the reads file.But dataset_192_I_2_20_CNTGS_DIST0_EM20.txt did not appear in galaxy-dist/database/files/000. run.py: simply coppies data from dataset_192_I_2_20_CNTGS_DIST0_EM20.txt to galaxy output file, but nothing is coppied since dataset_192_I_2_20_CNTGS_DIST0_EM20.txt was not created. How could the results appear?Could somebody help me ? *Your Help Is Highly Appreciated, Bassam Tork.* *PhD - GSU*

2 3

Re: [galaxy-dev] Written tutorial
by Jeremy Goecks 29 Dec '11

29 Dec '11

Jeff, The screencasts are in place of written tutorials; we've received feedback in the past that screencasts are often easier to understand and use. We'll try to locate the datasets used in these tutorials. Thanks, J. On Dec 29, 2011, at 9:43 AM, Rosenfeld, Jeffrey wrote: > Hi Jeremy, > > Thank you for the link. I have already seen those files, but I was > hoping for written instructions and the files for the video tutorials. > > Jeff > > On 12/29/2011 09:19 AM, Jeremy Goecks wrote: >> Jeff, >> >> Here are some tutorials on basic NGS analyses in Galaxy: >> >> http://main.g2.bx.psu.edu/u/james/p/exercises >> >> Best, >> J. >> >> On Dec 22, 2011, at 11:54 AM, Rosenfeld, Jeffrey wrote: >> >>> Hi, >>> >>> I am trying to teach some people to use Galaxy, and I was looking >>> for a written tutorial that went through the basics of taking sequence >>> reads, gromming and trimming them and then doing alignment. I see that >>> you have this type of exercise as screencasts: >>> http://main.g2.bx.psu.edu/u/aun1/p/ngs-analysis-service , but this is >>> not something written where users can follow instructions and do it >>> themselves. Also, there is no indication as to where to download the >>> example files used in the screencasts. >>> >>> Thanks, >>> >>> Jeff >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/

1 0

Re: [galaxy-dev] Toolshed
by Greg Von Kuster 29 Dec '11

29 Dec '11

Hello Ted, Please replay-all to back-and-forth communication like this so that the thread stays on the mailing list. You can also search the Galaxy mailing lists for answers to your questions - see http://galaxy.psu.edu/search/mailinglists/. See my answers below. On Dec 28, 2011, at 11:46 PM, Ted Goldstein wrote: > Greg, > Thanks. I reinstalled from galaxy-centlral and it works fine. I'd rather be on the dev branch. > > Two other topics: > 1. I would like to take a crack at > https://bitbucket.org/galaxy/galaxy-central/issue/554/show-column-names-hea… > Can you give me a clue about which files to work on? I haven't been working in this area for quite some time, and have not been involved in the discussion on the mail thread and bitbucket ticket for this, so I'm not the best person to answer your questions. That being said, if I understand the issue correctly, I would start by looking at the ColumnListParameter() class in ~/lib/galaxy/tools/parameters/basic.py. > > > 2. When I hit control-c, the server process stops answering but doesn't terminate. Is this correct? See this thread (there are others as well if you search the mailing list archives with various search strings): http://galaxy-users-list-archive.2308625.n4.nabble.com/restart-server-td413… > > Thanks! > Ted > > On Dec 28, 2011, at 5:30 AM, Greg Von Kuster wrote: > >> Hello Ted, >> >> I assume you cloned your local Galaxy instance from our distribution repo at https://bitbucket.org/galaxy/galaxy-dist. >> >> Unfortunately, in order to make use of the features that enable communication between your local Galaxy instance and a Galaxy tool shed, you have to update your local Galaxy instance from our Galaxy central repo at https://bitbucket.org/galaxy/galaxy-central. This is because the distribution repo is not updated as often as is necessary to keep current with the rapidly evolving tool shed code base and feature set. This issue should be eliminated at some point. >> >> >> On Dec 27, 2011, at 11:54 PM, Ted Goldstein wrote: >> >>> Hello, >>> I just installed Galaxy and am very impressed. But I am having problems adding in tools from the toolshed. When I click on adding a tool such as the Venn Diagram tool, it brings up the following message (even though I never installed it before: >>> >>> ============ >>> Found >>> >>> The resource was found at xyzzy.ucsc.edu:1234/admin_toolshed/install_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=9956e395db5a106ba074ae610c525fa17cec3b14:7b2276656e6e5f6c697374223a205b22447261772056656e6e204469616772616d2028504446292066726f6d206c69737473222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f70657465726a632f76656e6e5f6c697374222c2022626166373033316434373065225d7d&includes_tools=True; you should be redirected automatically. >>> WSGI Server >>> ================= >>> >>> >>> The page never redirects. As well the URL is malformed (missing the "http:"). If I fabricate this by hand. E.g. >>> http://xyzzy.ucsc.edu:1234/admin_toolshed/install_repository?tool_shed_url=… >>> >>> ============ >>> Not Found >>> >>> The resource could not be found. >>> No route for /admin_toolshed/install_repository >>> ============ >>> The log file shows: >>> .2.3.4 - - [27/Dec/2011:20:33:27 -0700] "GET /admin/browse_tool_sheds HTTP/1.1" 200 - "http://xyzzy.ucsc.edu:1234/admin" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> 1.2.3.4 - - [27/Dec/2011:20:33:30 -0700] "GET /admin/browse_tool_shed?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://xyzzy.ucsc.edu:1234/admin/browse_tool_sheds" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> 1.2.3.4 - - [27/Dec/2011:20:44:28 -0700] "GET /admin_toolshed/install_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=9956e395db5a106ba074ae610c525fa17cec3b14:7b2276656e6e5f6c697374223a205b22447261772056656e6e204469616772616d2028504446292066726f6d206c69737473222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f70657465726a632f76656e6e5f6c697374222c2022626166373033316434373065225d7d&includes_tools=True HTTP/1.1" 404 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.25) Gecko/20111212 Firefox/3.6.25" >>> >>> =============== >>> >>> I looked through the mailing list and wiki and I don't see this problem. Could I have misinstalled or misconfigured something? >>> >>> Thanks for the great system! >>> >>> Best regards, >>> Ted >>> >>> >>> Ted Goldstein >>> ted(a)soe.ucsc.edu >>> Ph.D. Candidate, HausslerLab, CBSE, UCSC >>> (650) 743-4141 >>> http://soe.ucsc.edu/~ted >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> >>> http://lists.bx.psu.edu/ >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0

Written tutorial
by Rosenfeld, Jeffrey 29 Dec '11

29 Dec '11

Hi, I am trying to teach some people to use Galaxy, and I was looking for a written tutorial that went through the basics of taking sequence reads, gromming and trimming them and then doing alignment. I see that you have this type of exercise as screencasts: http://main.g2.bx.psu.edu/u/aun1/p/ngs-analysis-service , but this is not something written where users can follow instructions and do it themselves. Also, there is no indication as to where to download the example files used in the screencasts. Thanks, Jeff

2 1

Possible bug
by Alfonso Núñez Salgado 28 Dec '11

28 Dec '11

As I've written before, I couldn't preview some kind of files and downloading them neither (not just some extensions but all). I don't really know if it's a real bug or not but i've saw other people whith the same problem. To solve it I've just changed the return value at "./lib/galaxy/web/controllers/dataset.py", at the "display" function (or method). In stead of returning the value returned by "open()" function, I do return the value returned by "read()" function. Originaly: ... return open(...) ... New: ... return open(...).read() ... Well, i don't know if its beacause of my system but i wish this can help Saludos Alfonso

1 0

Re: [galaxy-dev] Newbie question for toolshed development
by Greg Von Kuster 27 Dec '11

27 Dec '11

Including a tool in a section of one of the tool configs that populates the Galaxy tool panel is only relevant to the Galaxy instance, not a Galaxy tool shed. If a user installs a tool shed repository from a tool shed to their local Galaxy instance, they can choose a specific section label in which to load the tool into their tool panel at the time of installation. See http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_tool… On Dec 27, 2011, at 4:25 PM, Aaron Quinlan wrote: > Hi Greg, > > In a previous email (below) you mentioned that the tool_conf.xml is not necessary for tool shed repos. Consequently, is there no way to add section labels when a repo includes multiple tools? For example, I'd like to have section labels like the Picard tools have (e.g., "CONVERSION", "QC METRICS...") on the main site. > > Is this possible somehow? > > Best, > Aaron > > On Nov 23, 2011, at 9:51 AM, Greg Von Kuster wrote: > >>> Here is my current repo including just one tool: >>> >>> http://testtoolshed.g2.bx.psu.edu/repository/browse_repositories?sort=name&… >> >> This looks good, but a couple of comments: >> >> 1. There is no longer any use for suite_config.xml - it was used in the old version of the tool shed, but is no longer used. >> >> 2. Including tool_conf.xml is not necessary - nothing is done with it. >> >> 3. Support for data types used by your tools that are not included in the Galaxy distribution will be available soon, so if your tools use them, you can include code files that include classes that support them. More information about this will soon be available in the tool shed wiki, but let me know if you have questions about this. Your current tool doesn't fall into this category. >> >> 4. Make sure that your tool configs include the <requirements> tag sets if the tool has a dependency. This is the tag set that will enable the use of the future fabric scripts discussed above. See the section of our tool config syntax wiki for details: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Crequiremen… > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

1 0