- galaxy-dev - lists.galaxyproject.org

Galaxy+Slurm (with elastic cluster) error: "Job output not returned from cluster"
by Marco Tangaro 02 Nov '16

02 Nov '16

Dear all, I've a issue using Galaxy with elastic cluster support. It is provided by integrating SLURM and a worker node is added as soon as jobs are submitted through the Galaxy portal. When a job is submitted, the node takes some minutes to be configured. After ~7 minutes, Galaxy give me a failure message "Job output not returned from cluster". On the contrary if the node is already up, everything is ok. I tried only a very simple job getting the UCSC human genome. I'm using the master galaxy branch with postgresq+nginx+uwsgi+proftpd. Here is my job_conf.xml configuration: https://gist.github.com/mtangaro/c0528c3d9a7b44b3bab35dbd947f2c81 I'm not a slurm expert. I went through the mailing list archive, but I did not solved the issue. Thanks a lot for your help. Marco

2 1

IEs and fancy visualisations
by Björn Grüning 02 Nov '16

02 Nov '16

This is for all Interactive Environment fans that like awesome visualisations! Our friends from IOBIO have created a blog post about an fantastic collaboration between IOBIO and Galaxy: http://iobio.io/2016/11/02/galaxy-integration/ Thanks Chase and Yi - great work! Bjoern

1 0

Compiling dependencies in galaxy yields error
by evan clark 01 Nov '16

01 Nov '16

I was installing a tool from the galaxy toolshed and one of the dependencies was missing due to a bad link. I modified the install files to use the same version of the the software except from a local directory, however galaxy renders a large error when compiling. If I compile the source in the same location outside of galaxy it compiles and installs correctly. The specific package I am attempting to install is ghostscript for homer package.

2 4

Local Galaxy Workflow Sharing
by Rachel Bell 31 Oct '16

31 Oct '16

Hi, I am looking to share a workflow I've created in Local Galaxy with an individual user via email address - where they will access the workflow on main galaxy. When I select 'Share/Download' the following error message appears: [cid:59dde9c4-0f20-4cb6-9c14-1875f09657c4] I'd really appreciate some guidance on this, I'm guessing it is maybe a permissions thing in one of the config files? Thanks in advance, Rachel Bell Bioinformatics Technician Premaitha Health Rutherford House Manchester Science Park Manchester, M15 6SZ, UK T: +44 (0) 161 667 1068 M: +44 (0) 7875 191 219 www.premaitha.com<http://www.premaitha.com/> This e-mail including any attachments is intended for the addressee only. It is private and confidential. If you have received this message in error, we apologise, but please notify us and remove it from your system. You must take no action based on this email, nor must you copy or disclose it or any part of its contents to any person or organisation. Please note that email communications may be monitored. Premaitha Limited, trading as Premaitha Health, is registered in England with registered number 8436676. Its registered office is at Rutherford House, Manchester Science Park, Manchester, M15 6SZ, UK.

2 1

November 2016 Galactic News
by Dave Clements 31 Oct '16

31 Oct '16

1 0

Error for Picard CollectInsertSizeMetrics
by Raphenya, Amogelang 31 Oct '16

31 Oct '16

1 0

Job_id in a galaxy tool
by Katherine Beaulieu 31 Oct '16

31 Oct '16

Hello Galaxy Community, Is it possible to get the job_id listed on the info 'eye' page within a tool wrapper? Katherine

2 1

galaxy - velvet devteam revision 08256557922f
by Fernandez Edgar 31 Oct '16

31 Oct '16

Hey Guys, I need your help understanding what’s happening with velvet devteam revision 08256557922f. After installing it, velvet was giving me the following error: galaxy.jobs.handler WARNING 2016-10-20 12:15:11,986 (1599) Tool 'toolshed.g2.bx.psu.edu/repos/devteam/velvet/velveth/1.2.10.0' removed from tool config, unable to run job galaxy.jobs.handler ERROR 2016-10-20 12:15:12,165 (1599) Error checking job readiness I realized velvet needed that galaxy was restarted. Now, velvet finishes his jobs (becomes green) however, the following message is on galaxy’s Job output not returned from cluster And you can see in the logs of the handler: galaxy.jobs DEBUG 2016-10-31 10:17:08,548 (1669) Working directory for job is: /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669 galaxy.jobs.handler DEBUG 2016-10-31 10:17:08,567 (1669) Dispatching to drmaa runner galaxy.jobs DEBUG 2016-10-31 10:17:08,621 (1669) Persisting job destination (destination id: torque) galaxy.jobs.runners DEBUG 2016-10-31 10:17:08,630 Job [1669] queued (63.397 ms) galaxy.jobs.handler INFO 2016-10-31 10:17:08,652 (1669) Job dispatched galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Building dependency shell command for dependency 'velvet' galaxy.tools.deps DEBUG 2016-10-31 10:17:08,891 Find dependency velvet version 1.2.10 galaxy.tools.deps DEBUG 2016-10-31 10:17:08,896 Resolver tool_shed_packages returned <galaxy.tools.deps.resolvers.galaxy_packages.GalaxyPackageDependency object at 0x7f80f807bd50> (isnull? False) galaxy.jobs.command_factory INFO 2016-10-31 10:17:08,945 Built script [/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh] for tool command[PACKAGE_BASE=/home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f; export PACKAGE_BASE; . /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir/velvet/1.2.10/devteam/velvet/08256557922f/env.sh; velveth 2>&1 | grep "Version" | sed -e 's/Version //' > /home/galaxy/galaxy-prod/database/tmp/GALAXY_VERSION_STRING_1669 2>&1; python /home/galaxy/galaxy-prod/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/velvet/08256557922f/velvet/velveth_wrapper.py '/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat' '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/dataset_4306_files' 21 -fasta -short /home/galaxy/galaxy-prod/database/files/003/dataset_3537.dat] galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Building dependency shell command for dependency 'samtools' galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Find dependency samtools version None galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver tool_shed_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,144 Resolver galaxy_packages returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps DEBUG 2016-10-31 10:17:09,145 Resolver conda returned <galaxy.tools.deps.resolvers.NullDependency object at 0x7f812d147d90> (isnull? True) galaxy.tools.deps WARNING 2016-10-31 10:17:09,145 Failed to resolve dependency on 'samtools', ignoring galaxy.jobs.runners DEBUG 2016-10-31 10:17:09,153 (1669) command is: mkdir -p working; cd working; /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/tool_script.sh; return_code=$?; cd '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669'; python "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/set_metadata_KQTExg.py" "/home/galaxy/galaxy-prod/database/tmp/tmpLU0HQy" "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/working/galaxy.json" "/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_in_HistoryDatasetAssociation_4437_kIYXWa,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_kwds_HistoryDatasetAssociation_4437_ArCXyb,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_out_HistoryDatasetAssociation_4437_l28qWr,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_results_HistoryDatasetAssociation_4437_oTDjOd,/home/galaxy/galaxy-prod/database/files/004/dataset_4306.dat,/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/metadata_override_HistoryDatasetAssociation_4437_8VzdJw" 5242880; sh -c "exit $return_code" galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,202 (1669) submitting file /home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/galaxy_1669.sh galaxy.jobs.runners.drmaa INFO 2016-10-31 10:17:09,210 (1669) queued as 1573.esilbac3a.esi.umontreal.ca galaxy.jobs DEBUG 2016-10-31 10:17:09,214 (1669) Persisting job destination (destination id: torque) galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:09,642 (1669/1573.esilbac3a.esi.umontreal.ca) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2016-10-31 10:17:14,848 (1669/1573.esilbac3a.esi.umontreal.ca) state change: job finished normally galaxy.jobs.runners ERROR 2016-10-31 10:17:24,906 (1669/1573.esilbac3a.esi.umontreal.ca) Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-16.04/database/jobs_directory/001/1669/galaxy_1669.o' galaxy.model.metadata DEBUG 2016-10-31 10:17:25,314 loading metadata from file for: HistoryDatasetAssociation 4437 galaxy.jobs INFO 2016-10-31 10:17:25,541 Collecting metrics for Job 1669 galaxy.jobs DEBUG 2016-10-31 10:17:25,554 job 1669 ended (finish() executed in (636.363 ms)) galaxy.model.metadata DEBUG 2016-10-31 10:17:25,560 Cleaning up external metadata files Do you have an idea why I’m getting this problem? Best regards, Edgar Fernandez System Administrator (Linux) Direction Générale des Technologies de l'Information et de la Communication Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-210 • Bur. : 1-514-343-6111 poste 16568

1 0

Local galaxy samtools HISAT 2 issue
by evan clark 31 Oct '16

31 Oct '16

I am setting up a production galaxy server and am having an issue with running HISAT 2. It appears that it is not able to find samtools and renders an error within galaxy. Below is the error that it produces. I am unsure how to get the full error log for HISAT. |Traceback(most recent call last): File"/home/eclark28/galaxy_production/galaxy/lib/galaxy/jobs/runners/__init__.py", line 609,in finish_job job_state.job_wrapper.finish( stdout, stderr, exit_code) File"/home/eclark28/galaxy_production/galaxy/lib/galaxy/jobs/__init__.py", line 1265,in finish dataset.datatype.set_meta( dataset, overwrite=False) File"/home/eclark28/galaxy_production/galaxy/lib/galaxy/datatypes/binary.py", line 347,in set_meta exit_code= subprocess.call( args=command, stderr=open( stderr_name,'wb' ) ) File"/home/eclark28/galaxy_production/python/lib/python2.7/subprocess.py", line 523,in call return Popen(*popenargs, **kwargs).wait() File"/home/eclark28/galaxy_production/python/lib/python2.7/subprocess.py", line 711,in __init__ errread, errwrite) File"/home/eclark28/galaxy_production/python/lib/python2.7/subprocess.py", line 1343,in _execute_child raise child_exception OSError:[Errno 2] No suchfile or directory|

2 1

galaxy venv problem
by Beginner TI 31 Oct '16

31 Oct '16

Hi, During the installation or galaxy instance, I set up a directory to install all the python dependencies. Basically, whenever I install python modules, I do something like: python setup.py install --prefix=/my/python/dir as a result, there are tons of stuff installed there. I added this path to PYTHONPATH. The reason I do this is that I don't want to use sudo to install galaxy python modules. Now the problem...I was hoping not to use virtual env. However galaxy is set to use it's own venv, and during the course of starting, it will unset the PYTHONPATH. That caused that installed tools didn't work because it could not find the modules installed. Here is one example of how it is not working. The tool generated the following command in the script: gca -b /media/libraryfiles/galaxy-data-store/database/files/000/dataset_44.dat --span=3000 -g /media/libraryfiles/dep/cistrome-static-lib/ceaslib/GeneTable/mm9 --name=gca_out >& /media/libraryfiles/galaxy-data-store/database/files/000/dataset_95.dat cp gca_out.xls /media/libraryfiles/galaxy-data-store/database/files/000/dataset_94.dat I could run that command in the terminal successfully, but not in the galaxy. It keeps reporting error that gca_out.xls isn't there. With the PYTHONPATH unset, when I run that command, I would get: Traceback (most recent call last): File "/media/libraryfiles/dep/python/bin/gca", line 32, in <module> import CEAS.inout as inout ImportError: No module named CEAS.inout Once I put the PYTHONPATH back, export PYTHONPATH=/media/libraryfiles/dep/python/lib/python2.7/site-packages then that gca command would generate the correct output file. I tried to use sh run.sh --skip-venv However, it keeps giving me errors. It seems those were from not able to install the dependencies listed in requirements.txt. I then installed those dependencies to my python directory as well. During the course had to remove some that came with system, and install the newer ones. Now it seems to be working. I'm just wondering, is this a good way to using system python with all the dependencies I installed? Thanks, Rui

2 1