- galaxy-dev - lists.galaxyproject.org

Current directory in the galaxy job file script
by Ping Luo 22 Sep '15

22 Sep '15

I installed the latest Galaxy (v15.07) and tested with a job. I found my local environment setup file was not executed. The line of code that executes the local setup file is [ -f "config/local_env.sh" ] && . config/local_env.sh Further looking into the galaxy_xx.sh file shows the current directory of the script is $working_directory before "cd $working_directory" is executed. This is not the case in previous versions, where the job's current directory is the root of the Galaxy server in the beginning, until "cd $working_directory" is executed. I think making the current directory first at the root of the Galaxy server is better since I don't need to put a full path in galaxy.ini for the environment_setup_file. Will you change this back to it was? Thanks, Ping

1 0

uWSGI --daemonize option should be replaced with --daemonize2
by Federico Zambelli 22 Sep '15

22 Sep '15

Hello guys, I'm configuring a new Galaxy production instance and I did many tries with uWSGI. I have found that the -d (or --daemonize) option that the Galaxy wiki suggests should be replaced with the --daemonize2 option. In fact, when using the -d option for uWSGI it often happens that some (or sometimes all) of the uwsgi workers does not correctly bind to the Galaxy app when uWSGI is loaded just after Galaxy, as when you set up an init.d galaxy launch script. I don't know exactly why it happens but I think it has to do with the fact that uWSGI daemonize before the Galaxy server is fully loaded and this was giving me a lot of headaches. Example: This is a typical uWSGI log that I had at launch when using -d (or --daemonize) using six workers spawned uWSGI master process (pid: 32142) spawned uWSGI worker 1 (pid: 32183, cores: 4) spawned uWSGI worker 2 (pid: 32184, cores: 4) spawned uWSGI worker 3 (pid: 32185, cores: 4) galaxy.queue_worker INFO 2015-09-22 13:53:22,348 Binding and starting galaxy control worker for main.1 galaxy.queue_worker INFO 2015-09-22 13:53:22,351 Binding and starting galaxy control worker for main.2 spawned uWSGI worker 4 (pid: 32188, cores: 4) spawned uWSGI worker 5 (pid: 32189, cores: 4) galaxy.queue_worker INFO 2015-09-22 13:53:22,356 Binding and starting galaxy control worker for main.4 spawned uWSGI worker 6 (pid: 32191, cores: 4) *** Stats server enabled on 127.0.0.1:9191 fd: 20 *** galaxy.queue_worker INFO 2015-09-22 13:53:22,360 Binding and starting galaxy control worker for main.5 As you can see worker 3 and worker 6 are not bound to galaxy. And this behaviour is not deterministic: sometimes more workers are missing and in some cases I had no workers bound to Galaxy and so it was not accessible. Some other times instead all the workers were correctly bound to Galaxy. Using --daemonize2 ensures that uWSGI daemonize after the app (Galaxy) is fully loaded thus removing this strange behaviour. Best, Federico

1 0

parse history file/uploaded file with dynamic_options script
by Linda Bakker 22 Sep '15

22 Sep '15

Hi All, I am a bit lost here. In my xml I have the following sections: <inputs> <param type="data" format="tabular" name="amplitudes" label="amplitude file"/> <param name="samples" type="select" refresh_on_change="true" display="checkboxes" multiple="true" label="Mark quality control samples" dynamic_options='get_column_names(amplitudes)'/> </inputs> <code file="dynamic_options.py" /> In my dynamic_options.py script, I want to parse the headers from 'amplitudes' file and show them in the checkboxes section. My problem seems to be the following. In python, amplitudes is a galaxy object: <galaxy.model.HistoryDatasetAssociation object at 0x7f4fc020a210> How do I parse/access this file to get the headers? It seems to be partly the same problem as asked last week in this conversation: http://dev.list.galaxyproject.org/Question-about-using-dynamic-options-and-… Thanks! Linda -- Linda Bakker Bioinformatics Research Assistant Plant Research International, Wageningen UR Building 107 (Radix), room W1.Fa.054 Droevendaalsesteeg 1, 6708 PB, Wageningen, The Netherlands tel: +31-317-480756 email: linda.bakker(a)wur.nl

2 2

Need urgent help: ItemOwnershipException: History is not owned by user
by Zuzanna K. Filutowska 21 Sep '15

21 Sep '15

Dear All, I have recently started to receive errors like that while new LDAP users are trying to access the history or upload a file, while they can log into galaxy just fine. Everything under /home/galaxy is owned by galaxy:galaxy. I am using apache proxy. Details of error: galaxy.web.framework.decorators ERROR 2015-09-21 12:48:09,338 Uncaught exception in exposed API method: Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py", line 135, in decorator rval = func( self, trans, *args, **kwargs) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py", line 234, in create target_history = self.history_manager.get_owned( decoded_id, trans.user, current_history=trans.history ) File "/home/galaxy/galaxy-dist/lib/galaxy/managers/secured.py", line 94, in get_owned return self.error_unless_owner( item, user, **kwargs ) File "/home/galaxy/galaxy-dist/lib/galaxy/managers/secured.py", line 104, in error_unless_owner raise exceptions.ItemOwnershipException( "%s is not owned by user" % ( self.model_class.__name__ ) ) ItemOwnershipException: History is not owned by user -- Pozdrawiam, -- Zuzanna K. Filutowska, HPC Systems Administrator Poznan Supercomputing and Networking Center Institute of Bioorganic Chemistry Polish Academy of Sciences Seize the day boys! Make your lifes extraordinary! --John Keating

4 11

Time Out
by Daniel Siqueira de Oliveira 21 Sep '15

21 Sep '15

Hello, i'm repeatexplorer user and i am with difficult for download of a archive. The archive weighs 13.5GB and always arriving at 600-700MB of download, the download fail. I need a solution. Thank you. -- Daniel Siqueira de Oliveira Acadêmico de Ciências Biológicas- UFSM Iniciação científica- LABDROS Fone: (55)- 9699-5834

1 0

Problem with load balancing.
by Zuzanna K. Filutowska 18 Sep '15

18 Sep '15

Dear All, I am trying to run galaxy in multiple processes. I followed the documentation: https://wiki.galaxyproject.org/Admin/Config/Performance/Scaling And while the python processes seem to run fine, the balacing is broken - while accessing Galaxy server via www, I can only download a text file with "internal server error" inside. Here is my httpd.conf: http://www.wklej.org/id/1799330/ And here is my universe_wsgi.ini file: http://www.wklej.org/id/1799334/ Any help would be appreciated. -- Pozdrawiam, -- Zuzanna K. Filutowska, HPC Systems Administrator Poznan Supercomputing and Networking Center Institute of Bioorganic Chemistry Polish Academy of Sciences Seize the day boys! Make your lifes extraordinary! --John Keating

1 1

Error when Installing package_galaxy_utils_1_0
by Leslie Herrera 18 Sep '15

18 Sep '15

Hello: My lab is currently in the process of setting Galaxy up on our server. I have been installing the appropriate tools, but I am having difficulty installing the package_galaxy_utils_1_0. The error message is below. If this is something that is occuring on our end, would you please send instuctions on how to fix it? If I have sent this email to wrong address or you need more information, please let me know. Thank you very much for your help. Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.38.tar.gz Extracting in /tmp/tmp3dIIS_ Now working in /tmp/tmp3dIIS_/distribute-0.6.38 Building a Distribute egg in /scratch/tmp-toolshed-mtdKjroZR/galaxy_sequence_utils-1.0.0 Traceback (most recent call last): File "setup.py", line 248, in <module> scripts = scripts, File "/usr/lib/python2.7/distutils/core.py", line 111, in setup _setup_distribution = dist = klass(attrs) File "/tmp/tmp3dIIS_/distribute-0.6.38/setuptools/dist.py", line 225, in __init__ _Distribution.__init__(self,attrs) File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__ self.finalize_options() File "/tmp/tmp3dIIS_/distribute-0.6.38/setuptools/dist.py", line 257, in finalize_options ep.require(installer=self.fetch_build_egg) File "/tmp/tmp3dIIS_/distribute-0.6.38/pkg_resources.py", line 2027, in require working_set.resolve(self.dist.requires(self.extras),env,installer)) File "/tmp/tmp3dIIS_/distribute-0.6.38/pkg_resources.py", line 2237, in requires dm = self._dep_map File "/tmp/tmp3dIIS_/distribute-0.6.38/pkg_resources.py", line 2466, in _dep_map self.__dep_map = self._compute_dependencies() File "/tmp/tmp3dIIS_/distribute-0.6.38/pkg_resources.py", line 2499, in _compute_dependencies common = frozenset(reqs_for_extra(None)) File "/tmp/tmp3dIIS_/distribute-0.6.38/pkg_resources.py", line 2496, in reqs_for_extra if req.marker_fn(override={'extra':extra}): File "/tmp/tmp3dIIS_/distribute-0.6.38/_markerlib/markers.py", line 109, in marker_fn return eval(compiled_marker, environment) File "<environment marker>", line 1, in <module> NameError: name 'sys_platform' is not defined /scratch/tmp-toolshed-mtdKjroZR/galaxy_sequence_utils-1.0.0/distribute-0.6.38-py2.7.egg Traceback (most recent call last): File "setup.py", line 9, in <module> use_setuptools() File "/scratch/tmp-toolshed-mtdKjroZR/galaxy_sequence_utils-1.0.0/distribute_setup.py", line 152, in use_setuptools return _do_download(version, download_base, to_dir, download_delay) File "/scratch/tmp-toolshed-mtdKjroZR/galaxy_sequence_utils-1.0.0/distribute_setup.py", line 132, in _do_download _build_egg(egg, tarball, to_dir) File "/scratch/tmp-toolshed-mtdKjroZR/galaxy_sequence_utils-1.0.0/distribute_setup.py", line 123, in _build_egg raise IOError('Could not build the egg.') IOError: Could not build the egg. -- Leslie A Herrera Starr Lab Junior Specialist MCB, 139 Briggs Hall University of California, Davis laherrera(a)ucdavis.edu

2 2

Re: [galaxy-dev] need help badly
by Fernandez Edgar 17 Sep '15

17 Sep '15

Hello gents, I figured out something: it has nothing to do with galaxy but maybe one of you could explain me why I'm having this behavior : I connect ssh on the torque master (where the galaxy web service is running) and: 1. su - galaxy 2. cd /home/galaxy/galaxy-prod/database/job_working_directory/000 I execute a job #1234, it fails with the error mentioned below. I execute the exact same job #1235 and while it's running I enter the directory 1235 --> job finishes successfully. My directory /home/galaxy is an nfs mount in /etc/fstab. Any ideas? Cordialement / Regards, Edgar Fernandez De : Fernandez Edgar Envoyé : September-17-15 9:44 AM À : galaxy-dev(a)bx.psu.edu Objet : [galaxy-dev] need help badly Hello gents, I hope everyone is doing ok... I'm in need of your help so badly it's a little bit urgent: So my galaxy wasn't used in a while but the last time everything was working well: 1. The web-site was functional 2. The local handler upload the files locally on the master 3. Torque was shooting all the jobs correctly and getting an answer back. But today, one of my students tried some typical trimmomatic and all his jobs came back with the following error: Job output not returned from cluster. However, when I checked his datasets with that error, the view data (icon that is an eye) had some results. Also, when I looking inside the logs, I got: Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o' But that file galaxy_406.o existed and had TrimmomaticPE: Completed successfully I tried changing the value from 0 to 5 of "retry_job_output_collection" in the config/galaxy.ini file without any success. I've look through the internet and I saw a lot of people had that error but no one found a solution. Can you guys please help? Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication * Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218

3 5

need help badly
by Fernandez Edgar 17 Sep '15

17 Sep '15

Hello gents, I hope everyone is doing ok... I'm in need of your help so badly it's a little bit urgent: So my galaxy wasn't used in a while but the last time everything was working well: 1. The web-site was functional 2. The local handler upload the files locally on the master 3. Torque was shooting all the jobs correctly and getting an answer back. But today, one of my students tried some typical trimmomatic and all his jobs came back with the following error: Job output not returned from cluster. However, when I checked his datasets with that error, the view data (icon that is an eye) had some results. Also, when I looking inside the logs, I got: Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o' But that file galaxy_406.o existed and had TrimmomaticPE: Completed successfully I tried changing the value from 0 to 5 of "retry_job_output_collection" in the config/galaxy.ini file without any success. I've look through the internet and I saw a lot of people had that error but no one found a solution. Can you guys please help? Cordialement / Regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication * Bur. : 1-514-343-6111 poste 16568 Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-218

1 0

Galaxy Tools and Collections Remote Hackathon (Thursday and Friday)
by John Chilton 17 Sep '15

17 Sep '15

Hello All, This is just a reminder that the Galaxy Tools and Collections Remote Hackathon will be tomorrow Thursday the 17th and Friday the 18th of September. More details can be found at: https://github.com/galaxyproject/tools-iuc/issues/239 This Github issue includes a big list of potential tasks to work on. In addition to this list, feel free to stop by IRC or our Google Hangout all day and get advice and help on updating your own existing tools to use collections effectively. We will have a planning session Google hangout each morning at 10:00 EDT (GMT-04:00) and a wrap up session around 15:00 EDT. If can you can make it to the hangout at these times please stop by, but we will keep the Google hangout open all day. The link for the Google event which includes a link to the Hangout session is: https://plus.google.com/u/0/events/cnjf2mqg7vl6aigs5fkai6dh4rk In addition to the hangout I would encourage participants to join the #galaxyproject IRC channel on irc.freenode.net. More information at https://wiki.galaxyproject.org/Support/IRC. Finally, this is supposed to be a fun, community building event. There are many different ways to help out including some that do not involve coding, any contribution no matter how small is appreciated. This is not competitive and there is no reason to work all night or to excess. -John (on behalf of the IUC)

2 1