I am a new user of Galaxy.

I have a Galaxy instance running (sort of) on a local research cluster. I issued the command "hg update stable” today and it retrieved no files. SO presume I am up-to-date on the stable release. I start the instance as a user named “galaxy”.

Right now I am still running in “local” mode. I hope to migrate to DRMAA LSF eventually.

I have tried to set up ProFTP to upload files but have not succeeded so I use Galaxy Web-upload.

The upload was working nicely and I had added a couple of new tools and they were working with the uploaded files.

Getting LSF/DRMAA to work was giving me fits and ultimately I deleted all my history files in an effort to start over.

Presently, files being uploaded appear in history as say job 1 ( in a new history) The job status in the history panel of the web GUI

changes from purple to yellow then then to red indicating some sort of error. There is no viewable error text captured, but I can click on the “eye” icon and see the

first megabyte of the data (for tiny files I can see the entire content and it’s intact). In the Galaxy file system, however, these files appear but have a different number , say, dataset_399.dat

On my system the uploaded files appear in /PATH/galaxy-dist/database/files/000

My first question is why is the data going into the “000” subdirectory and not one “owned’ by the user who is uploading?

My second question is why is the dataset being labeled as dataset_399.dat and not dataset_001.dat?

My third question is why do the uploaded files not appear as selectable options ( say I have paired-end fastq files and tool wants to have choices about filenames)? This problem is present for programs that seek one input file as well.

I presume that Galaxy is confused because the numbering in history is not the same as the numbering in the file upload archive (e.g. /PATH/galaxy-dist/database/files/000 in my case) so my last question is how do I “reset” my system to get the dataset and history numbers to be the same?

Here’s how I launch the Galaxy instance

sh /shared/app/Galaxy/galaxy-dist/run.sh -v --daemon --pid-file=Nov6Localdaemon.pid.txt --log-file=Nov6Local1639daemon.log.txt

Entering daemon mode

Here are the last lines of the log

Starting server in PID 26236.

serving on 0.0.0.0:8089 view at http://127.0.0.1:8089

galaxy.tools.actions.upload_common DEBUG 2013-11-06 16:48:49,624 Changing ownership of /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 with: /usr/bin/sudo -E scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 hazards 502

galaxy.tools.actions.upload_common WARNING 2013-11-06 16:48:49,750 Changing ownership of uploaded file /shared/app/Galaxy/galaxy-dist/database/tmp/upload_file_data_QZGHm4 failed: sudo: no tty present and no askpass program specified

galaxy.tools.actions.upload_common DEBUG 2013-11-06 16:48:49,751 Changing ownership of /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO with: /usr/bin/sudo -E scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO hazards 502

galaxy.tools.actions.upload_common WARNING 2013-11-06 16:48:49,775 Changing ownership of uploaded file /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO failed: sudo: no tty present and no askpass program specified

galaxy.tools.actions.upload_common INFO 2013-11-06 16:48:49,805 tool upload1 created job id 170

galaxy.jobs DEBUG 2013-11-06 16:48:50,678 (170) Persisting job destination (destination id: local)

galaxy.jobs.handler INFO 2013-11-06 16:48:50,698 (170) Job dispatched

galaxy.jobs.runners.local DEBUG 2013-11-06 16:48:50,994 (170) executing: python /shared/app/Galaxy/galaxy-dist/tools/data_source/upload.py /depot/shared/app/Galaxy/galaxy-dist /shared/app/Galaxy/galaxy-dist/database/tmp/tmpTq22ot /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO 406:/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/dataset_406_files:/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/galaxy_dataset_406.dat

galaxy.jobs DEBUG 2013-11-06 16:48:51,030 (170) Persisting job destination (destination id: local)

galaxy.jobs.runners.local DEBUG 2013-11-06 16:48:53,335 execution finished: python /shared/app/Galaxy/galaxy-dist/tools/data_source/upload.py /depot/shared/app/Galaxy/galaxy-dist /shared/app/Galaxy/galaxy-dist/database/tmp/tmpTq22ot /shared/app/Galaxy/galaxy-dist/database/tmp/tmpEsyGfO 406:/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/dataset_406_files:/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/galaxy_dataset_406.dat

galaxy.jobs.runners.local DEBUG 2013-11-06 16:48:53,463 executing external set_meta script for job 170: /depot/shared/app/Galaxy/galaxy-dist/set_metadata.sh /shared/app/Galaxy/galaxy-dist/database/files /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170 . /depot/shared/app/Galaxy/galaxy-dist/universe_wsgi.ini /shared/app/Galaxy/galaxy-dist/database/tmp/tmpTq22ot /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/galaxy.json /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/metadata_in_HistoryDatasetAssociation_410_pUkPU9,/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/metadata_kwds_HistoryDatasetAssociation_410_iouZ6k,/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/metadata_out_HistoryDatasetAssociation_410_3c6Nj2,/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/metadata_results_HistoryDatasetAssociation_410_A6L2LO,/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/galaxy_dataset_406.dat,/shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/metadata_override_HistoryDatasetAssociation_410_8Wub22

galaxy.jobs.runners.local DEBUG 2013-11-06 16:48:54,789 execution of external set_meta for job 170 finished

galaxy.jobs DEBUG 2013-11-06 16:48:54,815 (170) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170 galaxy 50982

galaxy.jobs ERROR 2013-11-06 16:48:54,839 (170) Failed to change ownership of /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170, failing

Traceback (most recent call last):

File "/depot/shared/app/Galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 868, in finish

self.reclaim_ownership()

File "/depot/shared/app/Galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 1452, in reclaim_ownership

self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) )

File "/depot/shared/app/Galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 1438, in _change_ownership

assert p.returncode == 0

AssertionError

galaxy.jobs DEBUG 2013-11-06 16:48:54,876 fail(): Moved /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170/galaxy_dataset_406.dat to /shared/app/Galaxy/galaxy-dist/database/files/000/dataset_406.dat

-rw------- 1 galaxy galaxy 93 Nov 6 16:48 dataset_406.dat

I tried this as sudo user

scripts/external_chown_script.py /shared/app/Galaxy/galaxy-dist/database/job_working_directory/000/170 svcgalaxy 100000

Traceback (most recent call last):

File "scripts/external_chown_script.py", line 16, in <module>

import drmaa

File "build/bdist.linux-x86_64/egg/drmaa/__init__.py", line 41, in <module>

File "build/bdist.linux-x86_64/egg/drmaa/wrappers.py", line 43, in <module>

RuntimeError: could not find drmaa library. Please specify its full path using the environment variable DRMAA_LIBRARY_PATH

I have no idea why that refers to DRMAA anything because in this case I was running local

In fact however I get that same error when I start galaxy with this ”job_conf.xml”

cat job_conf.xml

<?xml version="1.0"?>

<job_conf>

<!-- "workers" is the number of threads for the runner's work queue.

The default from <plugins> is used if not defined for a <plugin>.

-->

</plugins>

<!-- Additional job handlers - the id should match the name of a

[server:<id>] in universe_wsgi.ini.

-->

</handlers>

<!-- Destinations define details about remote resources and how jobs

should be executed on those remote resources.

-->

<!-- "TODO: The real user options should maybe not be considered runner params."

-->

<param id="galaxy_external_runjob_script">scripts/drmaa_external_runner.py</param>

<param id="galaxy_external_killjob_script">scripts/drmaa_external_killer.py</param>

<param id="galaxy_external_chown_script">scripts/external_chown_script.py</param>

</destination>

</destinations>

</job_conf>

The other difference is that in the DRMAA case, the GUI upload job in the HISTORY panel goes “grey” and stays that way; nothing is ever uploaded.

In local mode I can upload files but there are errors in HISTORY and I cannot get tools to see the uploaded files.

In DMAA mode I cannot get the GUI upload to happen at all.

BTW in user “galaxy”’s .bashrc file I have this line

export DRMAA_LIBRARY_PATH=/opt/lsf/7.0/linux2.6-glibc2.3-x86_64/lib/libdrmaa.so

/shared/app/Galaxy/galaxy-dist/database/files