- galaxy-dev - lists.galaxyproject.org

show stopper for my galaxy installation
by Fernandez Edgar 23 Jan '15

23 Jan '15

Hello gents, I started this new thread because I have a showstopper for my installation of galaxy. Here's my situation: 1. I have a galaxy server that has internet access. 2. My galaxy server is also my torque server and my only torque submit node. 3. I have three torque compute nodes that DOESN'T have internet access. Now, I've tried uploading a file (larger than 2Gb) via and URL and it fails. However, I've added my galaxy server as a compute node and re-tried to upload the same file and I made sure the job will run on that machine and it works. Now, my compute nodes are in a network that is completely protected from the outside world. So no internet access. What are my option in this case? Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication * Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218

3 5

Manually installing rsem_datatypes package from toolshed
by Ryan G 22 Jan '15

22 Jan '15

I'm trying to manually install rsem_datatypes package into my local Galaxy install. I can't use the Toolshed for various reasons. Anyway, I downloaded the package using hg clone https://toolshed.g2.bx.psu.edu/repos/jjohnson/rsem_datatypes I see two files: 1) datatypes_conf.xml, and 2) rsem.py I copied the <datatype extension ...> lines into my config/datatypes_conf.xml file. I also copied rsem.py to lib/galaxy/datatypes/. When I restart Galaxy, it doesn't seem that Galaxy knows how to handle rsem datatypes, as I get this error: galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,520 Error importing datatype module galaxy.datatypes.rsem: 'module' object has no attribute 'rsem' Traceback (most recent call last): File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 210, in load_datatypes module = getattr( module, mod ) AttributeError: 'module' object has no attribute 'rsem' I'm obviously missing something but not sure what. I suspect maybe the rsem.py file is not in the right place? Ryan

2 2

Local Galaxy + Google Drive ?
by Nicholas Kline 22 Jan '15

22 Jan '15

Hello and Happy New Year, Back in September 2014, Google announced unlimited storage for users of Google Drive for Education: "Unlimited storage: No more worrying about how much space you have left or about which user needs more gigabytes. Drive for Education supports individual files up to 5TB in size and will be available in coming weeks." source: http://googleforeducation.blogspot.com/2014/09/announcing-drive-for-educati… Pardon me if I missed a discussion about this, but I was wondering if there are any plans to include a feature in Galaxy that would allow it to use Google Drive for storage instead of the local disk? Thanks :)

2 1

galaxy and torque - resource allocation
by Fernandez Edgar 22 Jan '15

22 Jan '15

Hello gents, Hope everyone had a great holiday break! Wish you guys all the best for 2015! I have a couple of questions about how resources (CPU and memory) is allocated when you have a galaxy and torque installation. So I've setup torque with some default and maximum amount of CPU and memory allocations. However, I have some worries when it comes to application (like tophat for example). By default, it takes half the CPU of a server unless specified otherwise. How is the CPU allocation is specified to application like tophat through galaxy? Also, how does galaxy react if a job needs more memory than the limit set by torque? Any information would help me a lot! My sincere salutations to you all!!! Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication * Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218

3 9

GALAXY_DATA_INDEX_DIR problem
by Lukasse, Pieter 22 Jan '15

22 Jan '15

Hi, In the documentation page https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax it mentions that "the reserved parameter name GALAXY_DATA_INDEX_DIR - it points to the ~/tool-data<https://bitbucket.org/galaxy/galaxy-central/src/tip/tool-data> directory." However, when we use this in a tool that is installed via toolshed we get a different interpretation of the variable: it seems to point to something like this at runtime: ./database/tmp/tmp-toolshed-gmfcrIr59Sq Ideas? Pieter Lukasse Wageningen UR, Plant Research International Department of Bioinformatics (Bioscience) Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands T: +31-317481122; M: +31-628189540; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

1 0

dataset collections
by Jorrit Boekel 22 Jan '15

22 Jan '15

Hi all, I’m toying around a little in galaxy-dist with the dataset collections feature. Since I know this is work in progress, I was wondering about some things I haven’t really found online. It seems to work really well to run a tool on a list of datasets, and a new job is run for each list item. But when I want to reduce to a smaller amount of list items, I understand I need to write some sort of merge tool myself, dependent on the data (all proteomics data here currently). This works well for reducing a dataset to a single file, but I am not sure about how to reduce to a new smaller collection. In the tool I’m writing, I let the user choose the size of the collection. Is there some way to tell galaxy dynamically how many outputs to expect AND put them in a collection? Something like: <outputs> <output type=“data_collection” amount_of_files=“3”/> </outputs> Where 3 is set by the user in a param also. Also, when running with two or more lists as input, is there some sort of correlation between the lists? It seems like it takes the files in dataset no order, so just checking. By the way, thanks very much John and everyone else involved in collections for doing and pushing this stuff. If there are smaller issues I can help with, I’d be thrilled. Can’t stress enough how much this feature means for galaxy adoption in our lab and possibly field. cheers, — Jorrit Boekel Proteomics systems developer BILS / Lehtiö lab Scilifelab Stockholm, Sweden

2 1

Setting a file download link in Galaxy help
by XiaoTao Jiang 22 Jan '15

22 Jan '15

Dear all I have installed Galaxy in local and I want to set a file download link in galaxy help. I want the user to download a small data-set from the help with a link. However, the general HTML download syntax does not take effect, look the figure I set. Anyone knows how to set the download link? [image: 内嵌图片 1] Thanks for all your attention. Regards Jimmy -- Department of Engineering The University of HongKong

2 1

Re: [galaxy-dev] PATH set in tool env.sh leaking through to set_metadata.py
by John Chilton 21 Jan '15

21 Jan '15

On Wed, Jan 21, 2015 at 6:54 PM, John Chilton <jmchilton(a)gmail.com> wrote: > On Wed, Jan 21, 2015 at 10:35 AM, Wolfgang Maier > <wolfgang.maier(a)biologie.uni-freiburg.de> wrote: >> >> >> On 21.01.2015 16:11, John Chilton wrote: >>> >>> So the tool dependency stuff didn't change - but the local job runner >>> now behaves like the other job runners (DRMAA, PBS, etc...) so you >>> have uncovered a bug for all of them I think. Is this a tool shed tool >>> or a locally created dependency? >>> >> >> This is a tool shed tool (package_mimodd_0_1_5), but one I uploaded myself >> (I discovered the bug when I did a test install after updating our local >> Galaxy). >> >>> I need to think about how to solve this more generally but if you want >>> a quick work around - I think it would work to just replace "python" >>> in $GALAXY_ROOT/set_metadata.sh with a hard-coded path to Galaxy's >>> python or perhaps just "python2". >> >> >> Since I can change the tool shed tool a more "global" solution could be to >> simply remove the "python" symlink from the python3 virtualenv after its >> created and have the tool use "python3" explicitly at runtime. I'll try >> whether that works, but I think it should. > > This sounds like a better short-term solution to me - this way older > versions of Galaxy will still be able to use the recipe. I created a > Trello issue to track the longer term fix of isolating metadata > generation from the tool environment - which is needed but could be a > little disruptive so I would rather push it off until after the next > release. Opps - I forgot the link and forgot to cc the mailing list. https://trello.com/c/aMCwYZsN -John > > Thanks! > > -John > >> >>> >>> Sorry for the inconvenience and thanks for bringing this to our attention. >>> >> >> No problem. I'm glad if I could help with something instead of just asking >> questions all the time. >> >> Wolfgang >> >> >> >>> >>> On Tue, Jan 20, 2015 at 12:52 PM, John Chilton <jmchilton(a)gmail.com> >>> wrote: >>>> >>>> Are you using the local job runner (this will be the case if you >>>> haven't explicitly configured something like pbs or DRMAA in your >>>> job_conf.xml file)? >>>> >>>> -John >>>> >>>> On Tue, Jan 20, 2015 at 12:34 PM, Wolfgang Maier >>>> <wolfgang.maier(a)biologie.uni-freiburg.de> wrote: >>>>> >>>>> On 01/20/2015 06:20 PM, Wolfgang Maier wrote: >>>>>> >>>>>> >>>>>> >>>>>> I have not seen this error before (I believe not with >>>>>> latest_2014.10.06) >>>>> >>>>> >>>>> >>>>> update: confirmed this now. It's enough to hg update to >>>>> latest_2014.10.06 >>>>> and things are working again. >>>>> >>>>> The difference is that when building the dependency shell command the >>>>> latest >>>>> release seems to put the call to set_metadata.sh into that command, >>>>> while >>>>> before it seems it was run separately. >>>>> >>>>> >>>>> Wolfgang >>>>> >>

1 0

PATH set in tool env.sh leaking through to set_metadata.py
by Wolfgang Maier 21 Jan '15

21 Jan '15

Hi, I am not sure whether I am diagnosing this problem approximately right or not, but it seems that with the latest release of Galaxy (stable branch, latest_2015.01.13) enviroment variables set in a tool-dependency's env.sh file and sourced before a job run are now leaking through to ./scripts/set_metadata.py called after the tool run finishes. In my specific case, the sourced env.sh adds a python 3 virtualenv directory to PATH, which then causes set_metadata.py to fail with: File "./scripts/set_metadata.py", line 130 except Exception, e: ^ SyntaxError: invalid syntax because that (amazingly enough :)) seems to be the first line not compatible with Python 3. I have not seen this error before (I believe not with latest_2014.10.06) so I guess my question is: has the way that set_metadata.py is called been changed recently and would you consider the current behavior a bug or is this expected and ok ? Best, Wolfgang

3 6

Oct '14 and Jan'15 release, trouble installing tools from the toolshed, resetting metadata for installed repositories
by Yves Gagnon 21 Jan '15

21 Jan '15

Hello fellow galaxy users : I noticed that trying to install some tools from the main toolshed, sometimes it works, others it does not. In a specific case, trying to install clustalw and its dependency, I get this error message : URL: (redactedURL)/admin_toolshed/prepare_for_install File '(redacted path)/galaxy_dist_dev/eggs/WebError-0.8a-py2.6.egg/weberror/evalexception/middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '(redacted path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '(redacted path)/galaxy_dist_dev/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '(redacted path)/galaxy_dist_dev/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '(redacted path)/galaxy_dist_dev/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1095 in prepare_for_install install_repository_manager.handle_tool_shed_repositories( installation_dict, using_api=False ) File '(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/install_manager.py', line 636 in handle_tool_shed_repositories new_tool_panel_section_label=new_tool_panel_section_label ) File '(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py', line 200 in create_repository_dependency_objects self.app.installed_repository_manager.activate_repository( repository_db_record ) File '(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/installed_repository_manager.py', line 90 in activate_repository repository_tools_tups = irmm.get_repository_tools_tups() File '(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 69 in get_repository_tools_tups index, shed_conf_dict = self.tpm.get_shed_tool_conf_dict( self.metadata_dict.get( 'shed_config_filename' ) ) TypeError: 'NoneType' object is not iterable Same goes for resetting all metadata on installed repositories after update, 23 out of ~200 tools work, for the rest I get this error in paster.log : tool_shed.galaxy_install.metadata.installed_repository_metadata_manager ERROR 2015-01-20 15:18:35,357 Error attempting to reset metadata on repository tabular_to_fasta Traceback (most recent call last): File "(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py", line 114, in reset_metadata_on_selected_repositories self.set_repository( repository ) File "(redacted path)/galaxy_dist_dev/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py", line 142, in set_repository super( InstalledRepositoryMetadataManager, self ).set_repository( repository ) File "(redacted path)/galaxy_dist_dev/lib/tool_shed/metadata/metadata_generator.py", line 1066, in set_repository self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get' My galaxy contains this line for tool_conf files, and I moved all config files from the distrib root to the config folder: tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml Did I do something wrong while updating to the latest releases??? Thanks for your help! Yves Gagnon

1 1