- galaxy-dev - lists.galaxyproject.org

installation/conda problems
by Matthias Bernt 06 Sep '17

06 Sep '17

Dear list, I think I have messed up the environment for one tool (IntaRNA) -- can someone tell me what to do to get a clear environment? Here is what happened. I have started the installation (via ansible) and accidentally aborted it. Then the tool gave me the following error: ``` Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks. Fatal error: Exit code 1 (Error occurred. Please check Tool Standard Error) ``` Since uninstalling and reinstallation (via ansible) gave the same result I tried to reinstall from the Galaxy UI. Now I got (even if if the admin interface list the tool as installed): ``` Fatal error: Exit code 127 (Error occurred. Please check Tool Standard Error) /gpfs1/data/galaxy_server/galaxy-dev/jobs_dir/000/362/tool_script.sh: line 25: IntaRNA: command not found ``` Is there a list of directories and files that I could clear to get rid of any trace of the tool and start anew? Best, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

3 3

Issues with new galaxy and galaxy_lib
by Evan Clark 04 Sep '17

04 Sep '17

Is there a way to change the location of the galaxy lib directory? We migrated our galaxy and it is looking for the lib on the wrong location. Additionally this is causing errors like the one below the first error. galaxy.jobs.runners DEBUG 2017-09-01 09:35:13,673 executing external set_meta script for job 13: GALAXY_LIB="/var/web_services/galaxy/lib"; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH"; else PYTHONPATH="$GALAXY_LIB"; fi; export PYTHONPATH; fi; GALAXY_VIRTUAL_ENV="/var/web_services/galaxy/.venv"; if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate"; fi; python "/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/set_metadata_7mPxU0.py" "/mnt/ceph/home/galaxy/galaxy/scratch/tmpEn3vYB" "/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/working/galaxy.json" "/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/metadata_in_HistoryDatasetAssociation_13_WlHC7F,/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/metadata_kwds_HistoryDatasetAssociation_13_w_LWxj,/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/metadata_out_HistoryDatasetAssociation_13_Wjmehj,/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/metadata_results_HistoryDatasetAssociation_13_r2SVjv,/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/galaxy_dataset_13.dat,/mnt/ceph/home/galaxy/galaxy/scratch/jobs_directory/000/13/metadata_override_HistoryDatasetAssociation_13_OXLKhY" 5242880 edu/koko/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:48.0) Gecko/ Firefox/48.0" galaxy.jobs.runners DEBUG 2017-09-03 15:33:54,164 execution of external set_meta for job 19 finished galaxy.jobs.output_checker DEBUG 2017-09-03 15:33:54,241 Tool produced standard error failing job - [Traceback (most recent call last): File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 19, in <module> from galaxy import util ImportError: No module named galaxy ]

1 0

unipept visualization
by Matthias Bernt 01 Sep '17

01 Sep '17

Dear list, I can not get the wonderful unipept visualization to work on our Galaxy instance (a test instance on the dev branch). I get the message: "Unfortunately we could not identify a suitable plugin...." The file I try to visualize was generated by unipept (pept2lca) and is of format d3_hierarchy. Could someone tell me if I need to install / configure something to get this running? Thanks a lot. Best, Matthias P.S. This might be also of interest for others, since this visualization is used in the Metaproteomics tutorial of the GTN. -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

3 7

conda problem
by Matthias Bernt 01 Sep '17

01 Sep '17

Dear list, When running rmarkdown_wgcna I get the following error in the logs: PaddingError: Placeholder of length '80' too short in package bioconda::r-wgcna-1.51-r3.3.1_0. The package must be rebuilt with conda-build > 2.0. The command that seems to lead to this is: Executing command: /gpfs1/data/galaxy_server/galaxy-dev/database/dependencies/_conda/bin/conda create -y --name mulled-v1-3bfe8da4ed798836281cc963619917 317ae960c9a828f7ea05d1670c65b19619 pandoc=1.15.0.6-0 r-getopt=1.20.0 r-rmarkdown=1.2 r-plyr=1.8.4 r-highcharter=0.4.0 r-dt=0.2 r-htmltools=0.3.5 r-wgcna=1.51 Any ideas? Best, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

2 2

[hxr@hx42.org: [SECURITY] 2017-08 security patch. Update now!]
by E. Rasche 29 Aug '17

29 Aug '17

Two high priority security vulnerabilities were recently discovered by Eric Rasche and Manabu Ishii respectively. These vulnerabilities were to cross site scripting and session fixation attacks. Detailed descriptions of these categories of vulnerabilities can be found at: - https://www.owasp.org/index.php/Cross-site_Scripting_(XSS) - https://www.owasp.org/index.php/Session_fixation Per our new security policies (https://github.com/galaxyproject/galaxy/blob/dev/SECURITY_POLICY.md) we have created patches for versions of Galaxy including 16.10, 17.01, and 17.05. **With Git** If you have cloned Galaxy from the official GitHub repository and are on a supported release_branch, you can simply `git pull` to pull in the security patch which has been applied to those branches. **Manual Patching** The 16.10 patch can apply cleanly to 17.01 as well and can be fetched using: wget https://gist.githubusercontent.com/jmchilton/760bf8ba6055b9a47a48529fcc49a4… The 17.05 patch can be fetched using: wget https://gist.githubusercontent.com/jmchilton/2623e37a2ccb1820770278e348dfef… Once the patch has been downloaded and copied to the root of your Galaxy directory, it can be applied using the following patch command: % patch -p1 < 2017augsecurity_1610.patch -or- % patch -p1 < 2017augsecurity_1705.patch If you are having trouble applying the patch feel free to email galaxy-committers(a)lists.galaxyproject.org and we will try to help. -Eric (on behalf of the Galaxy Committers) Post script: this mail was intended to go out on Thursday the 24th of August, however I failed to send it then. My apologies to the community, it will be more timely in the future.

1 0

Error while building reference genome index
by laurent.fournier＠ulg.ac.be 29 Aug '17

29 Aug '17

Hi, As the topic says, i am trying to populate the "built in" list. Fetching with the tool "Create DBKey and Reference Genome" in the Local datas works just fine but when i try to complete the procedure with "BWA-MEM index" (as well as "SAM FASTA index"), the job stops quickly. There are some shell outputs : export PATH=$PATH:'/home/mass/ifilesets/GAL/APP/galaxy/database/dependencies/_conda/envs/__samtools(a)1.3.1/bin' ; python "/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/set_metadata_WKziP5.py" "/home/mass/GAL/DATATMP/tmpsX8JCw" "/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/working/galaxy.json" "/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/metadata_in_HistoryDatasetAssociation_81_RSHoLb,/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/metadata_kwds_HistoryDatasetAssociation_81_cGTIQh,/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/metadata_out_HistoryDatasetAssociation_81_6hf9L8,/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/metadata_results_HistoryDatasetAssociation_81_0CUrCX,/home/mass/GAL/DATA/032/dataset_32404.dat,/home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/metadata_override_HistoryDatasetAssociation_81_XBr_WZ" 5242880; sh -c "exit $return_code" galaxy.jobs.runners.drmaa DEBUG 2017-08-29 12:06:50,938 (80) submitting file /home/mass/ifilesets/GAL/APP/galaxy/database/jobs_directory/000/80/galaxy_80.sh galaxy.jobs.runners.drmaa DEBUG 2017-08-29 12:06:50,938 (80) native specification is: -p all_5hrs galaxy.jobs.runners.drmaa INFO 2017-08-29 12:06:51,045 (80) queued as 2333510 galaxy.jobs DEBUG 2017-08-29 12:06:51,045 (80) Persisting job destination (destination id: slurm_default) galaxy.jobs.runners.drmaa DEBUG 2017-08-29 12:06:51,238 (80/2333510) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2017-08-29 12:06:52,247 (80/2333510) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2017-08-29 12:06:57,373 (80/2333510) state change: job finished normally galaxy.jobs.output_checker INFO 2017-08-29 12:06:57,416 Job 80: Fatal error: Exit code 1 () galaxy.jobs.output_checker DEBUG 2017-08-29 12:06:57,417 Tool exit code indicates an error, failing job. galaxy.jobs DEBUG 2017-08-29 12:06:57,482 (80) setting dataset 32404 state to ERROR galaxy.jobs INFO 2017-08-29 12:06:57,578 Collecting metrics for Job 80 galaxy.jobs DEBUG 2017-08-29 12:06:57,584 job 80 ended (finish() executed in (170.899 ms)) galaxy.model.metadata DEBUG 2017-08-29 12:06:57,586 Cleaning up external metadata files and then, in galaxy's frontend : Traceback (most recent call last): File "/home/mass/ifilesets/GAL/APP/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_sam_fasta_index_builder/2a1ac1abc3f7/data_manager_sam_fasta_index_builder/data_manager/data_manager_sam_fasta_index_builder.py", line 87, in <module> if __name__ == "__main__": main() File "/home/mass/ifilesets/GAL/APP/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_sam_fasta_index_builder/2a1ac1abc3f7/data_manager_sam_fasta_index_builder/data_manager/data_manager_sam_fasta_index_builder.py", line 82, in main build_sam_index( data_manager_dict, options.fasta_filename, target_directory, options.fasta_dbkey, sequence_id, sequence_name, data_table_name=options.data_table_name or DEFAULT_DATA_TABLE_NAME ) File "/home/mass/ifilesets/GAL/APP/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/data_manager_sam_fasta_index_builder/2a1ac1abc3f7/data_manager_sam_fasta_index_builder/data_manager/data_manager_sam_fasta_index_builder.py", line 30, in build_sam_index assert os.path.exists( fasta_filename ), 'FASTA file "%s" is missing, cannot build samtools index.' % fasta_filename AssertionError: FASTA file "/home/mass/ifilesets/GAL/APP/galaxy/tool-data/mm10/seq/mm10.fa,/home/mass/ifilesets/GAL/APP/galaxy/tool-data/mm10/seq/mm10.fa" is missing, cannot build samtools index. BUT if i check, the unexisting file is right where it is supposed to : [f043030@big-cloud19 tmp]$ ls -l /home/mass/ifilesets/GAL/APP/galaxy/tool-data/mm10/seq/ total 2720208 -rw-r--r--. 1 4294967294 4294967294 2785490220 Aug 28 09:47 mm10.fa Any ideas on what i do wrong ? Regards, Laurent Fournier. --- Laurent FOURNIER IT Specialist University of Liège, GIGA +1 laurent.fournier(a)ulg.ac.be +32 497 87 96 46 www.giga.ulg.ac.be

1 0

Re: [galaxy-dev] galaxy-dev Digest, Vol 134, Issue 19
by Matthias Bernt 28 Aug '17

28 Aug '17

Dear ..., what are the file permissions for the file and the folder? Is the drmaa_external_runner.py python script run with sudo? Which release/branch are you on? Best, Matthias > We are setting up run as real user but have run into a strange issue. > When the drmaa external script is called it could not load the job json > file. All permissions are set properly and the file exists. Upon > attempting to run the script by itself this error is created. When > loading a test script to open a file there is no issue. If the drmaa > external script is run as root the same issue occurs. > > /mnt/ceph/home/galaxy/galaxy/files_directory/database/slurm/11.jt_json > Traceback (most recent call last): > File "scripts/drmaa_external_runner.py", line 157, in <module> > main() > File "scripts/drmaa_external_runner.py", line 146, in main > load_job_template_from_file(jt, json_filename) > File "scripts/drmaa_external_runner.py", line 43, in > load_job_template_from_file > f = open(filename, 'r') > IOError: [Errno 13] Permission denied: > '/mnt/ceph/home/galaxy/galaxy/files_directory/database/slurm/11.jt_json'

1 0

Error with run as real user, unable to read file
by Evan Clark 25 Aug '17

25 Aug '17

We are setting up run as real user but have run into a strange issue. When the drmaa external script is called it could not load the job json file. All permissions are set properly and the file exists. Upon attempting to run the script by itself this error is created. When loading a test script to open a file there is no issue. If the drmaa external script is run as root the same issue occurs. /mnt/ceph/home/galaxy/galaxy/files_directory/database/slurm/11.jt_json Traceback (most recent call last): File "scripts/drmaa_external_runner.py", line 157, in <module> main() File "scripts/drmaa_external_runner.py", line 146, in main load_job_template_from_file(jt, json_filename) File "scripts/drmaa_external_runner.py", line 43, in load_job_template_from_file f = open(filename, 'r') IOError: [Errno 13] Permission denied: '/mnt/ceph/home/galaxy/galaxy/files_directory/database/slurm/11.jt_json'

1 0

Firefox drag and drop not working
by Jochen Bick 25 Aug '17

25 Aug '17

Hi all, we recently updated to our galaxy server and now we would like to use the new feature drag and drop items from the history. It does not work with Firefox but with chrome. In Firefox it says: The address isn’t valid The URL is not valid and cannot be loaded. Web addresses are usually written like http://www.example.com/ Make sure that you’re using forward slashes (i.e. /). Is there anything to enable, install, or set inside Firefox? Cheers Jochen

3 3

job_working_directory: no subdirectories created
by D K 23 Aug '17

23 Aug '17

Hi Galaxy-devs, I recently ran into odd behavior when jobs were submitted at the same time to our cluster (jobs wouldn't return correctly, e.g. ). I changed "cleanup_job" in galaxy.ini to "never" to see what was going on in my job_working_directory (/scratch/galaxy/job_working/). It looks like subdirectories are no longer being created in my parent job_working_directory (e.g. the tool_script.sh along with the outputs are always placed and executed in /scratch/galaxy/job_working). Isn't there usually a bunch of subdirectories created (e.g. 000/XXX, etc...) I suspect my issues arise because script execution is happening in the parent directory rather than in temporary ones that get created within the parent directory. Could this be the issue and if so does anyone know how this could have happened or how I can fix this? I've also noticed that my environment variables are pointing to the parent directory rather than a subdirectory: SGE_CWD_PATH=/scratch/galaxy/job_working As an example, the contents of /scratch/galaxy/job_working look like this (note that there are no usual subdirectories like 000/XXX, etc...): [galaxy@galaxy job_working]$ ls | more __instrument_core_epoch_end __instrument_core_epoch_start __instrument_core_galaxy_slots conda-env conda-metadata-env galaxy_20771.e galaxy_20771.ec galaxy_20771.o galaxy_20771.sh galaxy_20772.e galaxy_20772.ec galaxy_20772.o galaxy_20772.sh metadata_in_HistoryDatasetAssociation_46704_NHpqc0 metadata_in_HistoryDatasetAssociation_46705_nUT573 metadata_kwds_HistoryDatasetAssociation_46704_5IADnM metadata_kwds_HistoryDatasetAssociation_46705__z5cTO metadata_out_HistoryDatasetAssociation_46704_G8phbW metadata_out_HistoryDatasetAssociation_46705_qTOxoE metadata_override_HistoryDatasetAssociation_46704_v0f_Wl metadata_override_HistoryDatasetAssociation_46705_8pzLX_ metadata_results_HistoryDatasetAssociation_46704_fSFQsP metadata_results_HistoryDatasetAssociation_46705_r4vUOP set_metadata_2A6wuD.py set_metadata_2Dasxn.py tool_script.sh working Any help would be greatly appreciated!!

1 0