December 2011 - galaxy-dev - lists.galaxyproject.org

Visualization/Display Application with related datasets
by Andrew Brock 17 Dec '11

17 Dec '11

Hi All, What is the best way to link data from multiple data sets with the least user input? The files in each dataset have links to the filenames which were used to generate them, which are valid names of other galaxy datasets in the file system. The only way I can see is to write a visualization tool, where the user has to manually select each dataset that was used along the workflow, however in my sample workflow this is 11 files with 4 different data types. I have a script which you can pass the filename of the last file generated, and when run from a command line shell will successfully load up all datasets used throughout the workflow. If run on the same computer as a display application, this works fine, however if the display application is hosted on an external server I need to be able to generate links to all the required files. The method of invoking this display application/visualisation is not a concern, however it would be nice if the user didn't have to manually select all the files used and what they were, when the information is already available. Andrew.

1 0

Automatic citation list from a tool, workflow, or history
by Peter Cock 15 Dec '11

15 Dec '11

Dear all, It has become a convention that each tool/wrapper in Galaxy includes citation instructions in their help text (although not all the tools do this - I think they should). It occurred to me this could be formalised, with explicit markup in the tool XML file, embedding the citation (at very least with an identifier like the DOI or ISBN, there is probably a good existing XML standard that could be followed). Then, Galaxy would be able to automatically pull out a list of citations the tool authors have requested be cited, removing duplicates (e.g. matching DOI), from a history or a workflow. The aim of this is (a) to make it easier to write up your methods by supplying all the references, and (b) to help ensure tool authors get the acknowledgement they deserve. Does this sound like a good idea? Peter

5 5

about ftp setting up
by Liao_Ruiqi 15 Dec '11

15 Dec '11

Hello there， I'm a student at Fudan University in China. I was trying to enable ftp uploading in my local version of GALAXY. I tried the setting in http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP, but I got the following problems: 1. The galaxyftp seems to be a pasql database, shall I build it myself? or dose it already exists in galaxy directory? 2. the galaxy_user seems to be a table, shall I build it myself? and how shall I update it and integrate it with galaxy? Thanks for your attention, and looking forward to your reply! Best wishes Liao Ruiqi

2 1

Excel upload problems
by Siemen Sikkema 15 Dec '11

15 Dec '11

Hello! For our future galaxy users in Netherlands Bioinformatics Centre we'd like to support Excel uploads. We have our own excel to tabular CLI tool and I have been trying to get it working with galaxy. The tool itself is not the problem, but getting excel files uploaded is… We're running a local instance (hg identify: 949e4f5fa03a+ tip) on Mac for development purposes. The issues with excel are two-fold: XLSX files get unzipped automatically and a useless XML file remains, while uploading an XLS file results in: "The uploaded binary file contains inappropriate content". I've tried adding <datatype extension="xls" type="galaxy.datatypes.binary:Binary" display_in_upload="true"/> <datatype extension="xlsx" type="galaxy.datatypes.binary:Binary" display_in_upload="true"/> to datatypes_conf.xml and selecting XLS and XLSX as datatype during upload but to no avail (the errors don't change). A temporary workaround we thought of was to first zip the files before uploading, that way Galaxy would unzip them and we'd be left with the raw excel files. At first this seemed to work but the conversion did not. Furthermore, downloading the files and trying to open them failed. A quick 'diff' between the original and mangled files show differences practically throughout the whole file! Now, my questions are as follows. Has work been underway to support excel natively? Is there a way to have Galaxy simply accept uploaded files without any interpretation? What happens inside Galaxy that corrupts the excel files during unpacking? Thank you very much and I apologize if I somehow missed something obvious or if these questions have been asked before. Kind regards, Siemen Sikkema

6 10

Data upload from a server
by Makis Ladoukakis 15 Dec '11

15 Dec '11

Dear Galaxy developers, I have installed a Galaxy local instance on our university's server and I am going to use it to analyze data also uploaded on the same server. Nevertheless the access to the server (and the data) is limited only to those involved with the project so making public urls for them is not an option. Downloading the data to my pc is also not an option as its volume is much greater than my hard drive. Is there another way for Galaxy to access those data? Thank you, Efthymios Ladoukakis

3 2

New tool not recognised unless i reboot
by Makis Ladoukakis 15 Dec '11

15 Dec '11

Dear galaxy users, I have downloaded a local instance of Galaxy on my linux distribution (ubuntu) and i am adding new tools i made myself. To add a new tool i follow all the steps (tool definition file and the modification of tool_conf.xml) and run 'sh run.sh' for Galaxy to include it in its interface. Nevertheless when i try to do the same later after i close Galaxy, when i run 'sh run.sh' again i get this error: socket.error: [Errno 98] Address already in use and the tool is not recognized. For the run.sh to work again i have to reboot my pc. In a virtual machine ubuntu image it's even worse. The 'run.sh' command keeps sending this error even when i reboot the image or even when i reboot the pc itself. Is there a way around this problem? Thank you, Makis

3 3

Looking for Galaxy related papers (or have written one)? There is a new Galaxy Group @ CiteULike
by Dave Clements 14 Dec '11

14 Dec '11

Hello all, A new Galaxy Group <http://www.citeulike.org/group/16008/order/group_rating>has been created at the CiteULike <http://citeulike.org/> social bookmarking service to organize papers that are about, use, or reference Galaxy. The Galaxy Group<http://www.citeulike.org/group/16008/order/group_rating>is open to any CiteULike user (and anyone can become a CiteULike user). Group members can add papers to the group, assign tags, and rate papers. CiteULike makes extensive use of *tags* to organize and categorize papers. The Galaxy group uses this set of tags<http://wiki.g2.bx.psu.edu/CiteULike#CiteULike_Tags>to categorize papers. If you want to help other Galaxy users and sites find papers that are relevant to them, or help the Galaxy Project keep track of what papers (and communities) are using Gaalxy, then please help by adding relevant papers to the Galaxy Group Library<http://www.citeulike.org/group/16008/order/group_rating>and/or rating and tagging the papers that are there. *We particularly need help with papers that were published before this year, as very few of them are in the initial list. * Links: http:/galaxyproject.org/wiki/CiteULike - Explanation of tags http://www.citeulike.org/group/16008/order/group_rating - Galaxy Group list of papers Thanks, Dave Clements PS: To cite the Galaxy Project, see http://galaxyproject.org/wiki/Citing%20Galaxy. -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ http://galaxyproject.org/wiki/

1 0

Re: [galaxy-dev] error when running UNIT_TEST script
by James Boocock 14 Dec '11

14 Dec '11

Hi When running the run_unit_tests.sh script on my server i get the following Error. Wondering what It means and whether It is something I have done or something within galaxy #Start Error Doctest: galaxy.web.form_builder.TextArea ... ok TextField (galaxy.web.form_builder) Doctest: galaxy.web.form_builder.TextField ... ok ====================================================================== ERROR: Failure: ImportError (No module named tables) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/loader.py", line 379, in loadTestsFromName addr.filename, addr.module) File "/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py", line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/home/james/galaxy-dist/lib/galaxy/datatypes/converters/bed_to_genetrack_converter.py", line 16, in <module> from genetrack.scripts import tabs2genetrack File "/home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/__init__.py", line 38, in <module> from genetrack.hdflib import PositionalData File "/home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/hdflib.py", line 5, in <module> from tables import openFile ImportError: No module named tables ---------------------------------------------------------------------- Ran 60 tests in 11.380s #End Error Cheers James

2 1

Question about the <repeat> tag
by Roger Ngo 14 Dec '11

14 Dec '11

Hello all, I have a question on how the <repeat> tag works in a Galaxy tool XML file. Right now I have a perl script that takes in various arguments. One of the arguments I have is a repeated one which is just a file selection feature. The script works something like this: script.pl $arg1 $arg2 $varyingNumOfArgs By using <repeat>, is the execution of the script something like this? script.pl $arg1 $arg2 $repeatedArg1 $repeatedArg2 ... $repeatedArgN where repeatedArgs1 ... repeatedArgsN are basically arguments specified by the <repeat> tag. If so, is there a certain way to access these repeated arguments and pass them into my script in the <command> tag? This is the part where I am not quite sure how <repeat> works. Thanks

2 2

Re: [galaxy-dev] [galaxy-user] enabling regular users to upload large data volumes to a local Galaxy server
by Shantanu Pavgi 14 Dec '11

14 Dec '11

Moving this to galaxy-dev list. Just wanted to share my notes on both upload options. Although both ftp_upload_dir and 'user library import' options help in getting data into the galaxy there are some differences in both options: 1. ftp upload option deletes original copies of files after galaxy import, whereas the user library option doesn't delete original copies from the file system 2. ftp option doesn't provide an option to 'sym-link' to files without copying them in galaxy, which 'user library' option supports 3. ftp option scans configured location recursively and displays all files in the web interface, whereas with 'user import' option you will need to organize files in sub-directories 4. ftp option requires galaxy application user to have full permissions over 'ftp_upload_dir', whereas 'user import' will work even with read-execute permissions (execute - for scanning the dirs) for the galaxy application user. Both upload options are useful and have certain advantages-disadvantages depending on your use case. And actually you don't need a FTP server setup for transferring files into ftp_upload_dir location as long as you take care of file system permissions. I think the galaxy server treats 'ftp_upload_dir' location as FTP staging area - where files can be dropped off, then galaxy imports these files in it's application specific naming convention and deletes original copies of these imported files from 'ftp_upload_dir' location. If one takes care of appropriate file permissions for the galaxy application user and user-depositing files then 'ftp_upload_dir' location can be used without setting up FTP server. You can drop of these files using any other transfer protocol as well - e.g. we have used SCP. However, these file permission issues can be handled easily using a FTP server program and hence FTP server option is definitely a recommended approach. -- Shantanu. On Dec 13, 2011, at 10:55 AM, Yury V Bukhman wrote: > Perhaps someone else will find this of interest. As it turned out, we didn't need FTP for a local server. Instead, one can set an option in universe_wsgi.ini (or universe_wsgi.webapp.ini if you are running multiple web server processes) to allow regular users to upload a set of files from a file system into a data library. Thus we have set up a drop box on a shared drive, with a subfolder for each user, where they can put their files, and gave each user their own data library in Galaxy where those files can be uploaded. The drop box folder is mounted on the Galaxy server. > > The option in universe_wsgi.webapp.ini is: > user_library_import_dir = /mnt/dropbox > > It seems to work... > > Yury > > > > On 11/14/2011 5:25 PM, Chorny, Ilya wrote: >> FTP server. >> >> Ilya >> >> -----Original Message----- >> From: galaxy-user-bounces(a)lists.bx.psu.edu [mailto:galaxy-user-bounces@lists.bx.psu.edu] On Behalf Of Yury V Bukhman >> Sent: Monday, November 14, 2011 3:01 PM >> To: galaxy-user(a)lists.bx.psu.edu >> Subject: [galaxy-user] enabling regular users to upload large data volumes to a local Galaxy server >> >> Hi, >> >> we are running a local Galaxy server, administered by a bioinformatics core group. Our end users increasingly come to us with sets of large NGS files that they can't upload to Galaxy on their own through a web browser. We copy their data to a Galaxy filesystem and upload into data libraries from there using the admin interface. However, the users would prefer to be able get their data onto the server on their own. >> What's the best solution to that? Should we set up FTP upload? Are there other tricks? Any advice would be appreciated. >> >> Thanks. >> >> Yury >> >> >> >> -- >> Yury V. Bukhman, Ph.D. >> Associate Scientist, Bioinformatics Group Leader Great Lakes Bioenergy Research Center University of Wisconsin - Madison >> 445 Henry Mall, Rm. 513 >> Madison, WI 53706, USA >> Phone: 608-890-2680 Fax: 608-890-2427 >> Email: ybukhman(a)glbrc.wisc.edu >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ > > -- > Yury V. Bukhman, Ph.D. > Associate Scientist, Bioinformatics Group Leader > Great Lakes Bioenergy Research Center > University of Wisconsin - Madison > 445 Henry Mall, Rm. 513 > Madison, WI 53706, USA > Phone: 608-890-2680 Fax: 608-890-2427 > Email: ybukhman(a)glbrc.wisc.edu > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0