Hi Edgar,

I am a pretty big newbie to Galaxy but I have had similar problems on my Dev build with grabbing outputs from the working directories. Its a temporary fix but I have put routines in a bash wrapper that uses a working directory that I made and know. If your cluster has access to a shared drive that would work that way it may work.

Josh

On Thu, Sep 17, 2015 at 11:05 PM, Fernandez Edgar < edgar.fernandez@umontreal.ca> wrote:

Hello gents,

I figured out something: it has nothing to do with galaxy but maybe one of you could explain me why I’m having this behavior :

I connect ssh on the torque master (where the galaxy web service is running) and:

1.   su - galaxy
   
   

2.   cd /home/galaxy/galaxy-prod/database/job_working_directory/000
   
   
   
   

I execute a job #1234, it fails with the error mentioned below.

I execute the exact same job #1235 and while it’s running I enter the directory 1235 à job finishes successfully.

My directory /home/galaxy is an nfs mount in /etc/fstab.

Any ideas?

Cordialement / Regards,

Edgar Fernandez

*De :* Fernandez Edgar *Envoyé :* September-17-15 9:44 AM *À :* galaxy-dev@bx.psu.edu *Objet :* [galaxy-dev] need help badly

Hello gents,

I hope everyone is doing ok…

I’m in need of your help so badly it’s a little bit urgent:

So my galaxy wasn’t used in a while but the last time everything was working well:

1.   The web-site was functional
   
   

2.   The local handler upload the files locally on the master
   
   

3.   Torque was shooting all the jobs correctly and getting an answer
   

back.

But today, one of my students tried some typical trimmomatic and all his jobs came back with the following error:

Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that is an eye) had some results.

Also, when I looking inside the logs, I got:

Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'

But that file galaxy_406.o existed and had

TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the config/galaxy.ini file without any success.

I’ve look through the internet and I saw a lot of people had that error but no one found a solution.

Can you guys please help?

Cordialement / Regards,

*Edgar Fernandez*

System Administrator (Linux)

Direction Générale des Technologies de l'Information et de la Communication

( Bur. : *1-514-343-6111 <1-514-343-6111> poste 16568*

*Université de Montréal*

PAVILLON ROGER-GAUDRY, bureau X-218

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

Hi Edgar,

Hm, I haven't gotten it to work but galaxy is supposed to support looking in the working directory for the file. Its likely for reasons similar to your error. If you have access to the program's script you could try to force it to write the results somewhere else. I would be surprised if ".." didn't work. the job_working_directory/000 folder always exists. Of course these are similar to my prior suggestion.

I haven't had to interact with more cloud stuff than the google APIs for classes or shared drives. If you want to do a script do something along the following and at it to the programs run at system start up.

Point A. For folder in job_working_directory/000 as $var cd $var cd .. end loop GOTO A.

Lastly, and I probably should have started here, any chance that its a folder permissions issue?

Good Luck Josh

On Thu, Sep 17, 2015 at 11:35 PM, Fernandez Edgar < edgar.fernandez@umontreal.ca> wrote:

Hello Josh,

Thank you so much for the answer.

I thought of that but it doesn’t solve my problem.

I need to create a routine that looks inside the new job’s directory :

So for example, a process that runs every second and looks inside

/home/galaxy/galaxy-prod/database/job_working_directory/000

And if there is a new job (new folder), it looks/enters inside with cd.

However, is it possible to program galaxy server to create the job’s directory and enter the directory so when a torque slave writes inside the directory, the master is already inside?

Cordialement / Regards,

Edgar Fernandez

*De :* Josh Woodring [mailto:woodring.josh94@gmail.com] *Envoyé :* September-17-15 11:14 AM *À :* Fernandez Edgar *Cc :* galaxy-dev@bx.psu.edu *Objet :* Re: [galaxy-dev] need help badly

Hi Edgar,

I am a pretty big newbie to Galaxy but I have had similar problems on my Dev build with grabbing outputs from the working directories. Its a temporary fix but I have put routines in a bash wrapper that uses a working directory that I made and know. If your cluster has access to a shared drive that would work that way it may work.

Josh

On Thu, Sep 17, 2015 at 11:05 PM, Fernandez Edgar < edgar.fernandez@umontreal.ca> wrote:

Hello gents,

I figured out something: it has nothing to do with galaxy but maybe one of you could explain me why I’m having this behavior :

I connect ssh on the torque master (where the galaxy web service is running) and:

1.   su - galaxy
   
   

2.   cd /home/galaxy/galaxy-prod/database/job_working_directory/000
   
   
   
   

I execute a job #1234, it fails with the error mentioned below.

I execute the exact same job #1235 and while it’s running I enter the directory 1235 à job finishes successfully.

My directory /home/galaxy is an nfs mount in /etc/fstab.

Any ideas?

Cordialement / Regards,

Edgar Fernandez

*De :* Fernandez Edgar *Envoyé :* September-17-15 9:44 AM *À :* galaxy-dev@bx.psu.edu *Objet :* [galaxy-dev] need help badly

Hello gents,

I hope everyone is doing ok…

I’m in need of your help so badly it’s a little bit urgent:

So my galaxy wasn’t used in a while but the last time everything was working well:

1.   The web-site was functional
   
   

2.   The local handler upload the files locally on the master
   
   

3.   Torque was shooting all the jobs correctly and getting an answer
   

back.

But today, one of my students tried some typical trimmomatic and all his jobs came back with the following error:

Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that is an eye) had some results.

Also, when I looking inside the logs, I got:

Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'

But that file galaxy_406.o existed and had

TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the config/galaxy.ini file without any success.

I’ve look through the internet and I saw a lot of people had that error but no one found a solution.

Can you guys please help?

Cordialement / Regards,

*Edgar Fernandez*

System Administrator (Linux)

Direction Générale des Technologies de l'Information et de la Communication

( Bur. : *1-514-343-6111 <1-514-343-6111> poste 16568*

*Université de Montréal*

PAVILLON ROGER-GAUDRY, bureau X-218

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

--

Nil mihi rescribas, tu tamen ipse veni!

Hi Edgar,

I vote for a permission problem. I had to recently debug a volume/file permission problem and the symptoms are identical: Galaxy claims it can't find a file that actually does exist. Given that you said /home/galaxy is an NFS mount I strongly suspect something there has changed. Unfortunately I don't know enough about NFS to be more help.

Cheers, Keith

On Sep 17, 2015, at 12:34 PM, Josh Woodring woodring.josh94@gmail.com wrote:

Hi Edgar,

Hm, I haven't gotten it to work but galaxy is supposed to support looking in the working directory for the file. Its likely for reasons similar to your error. If you have access to the program's script you could try to force it to write the results somewhere else. I would be surprised if ".." didn't work. the job_working_directory/000 folder always exists. Of course these are similar to my prior suggestion.

I haven't had to interact with more cloud stuff than the google APIs for classes or shared drives. If you want to do a script do something along the following and at it to the programs run at system start up.

Point A. For folder in job_working_directory/000 as $var cd $var cd .. end loop GOTO A.

Lastly, and I probably should have started here, any chance that its a folder permissions issue?

Good Luck Josh

On Thu, Sep 17, 2015 at 11:35 PM, Fernandez Edgar edgar.fernandez@umontreal.ca wrote: Hello Josh,

Thank you so much for the answer.

I thought of that but it doesn’t solve my problem.

I need to create a routine that looks inside the new job’s directory :

So for example, a process that runs every second and looks inside

/home/galaxy/galaxy-prod/database/job_working_directory/000

And if there is a new job (new folder), it looks/enters inside with cd.

However, is it possible to program galaxy server to create the job’s directory and enter the directory so when a torque slave writes inside the directory, the master is already inside?

Cordialement / Regards,

Edgar Fernandez

De : Josh Woodring [mailto:woodring.josh94@gmail.com] Envoyé : September-17-15 11:14 AM À : Fernandez Edgar Cc : galaxy-dev@bx.psu.edu Objet : Re: [galaxy-dev] need help badly

Hi Edgar,

I am a pretty big newbie to Galaxy but I have had similar problems on my Dev build with grabbing outputs from the working directories. Its a temporary fix but I have put routines in a bash wrapper that uses a working directory that I made and know. If your cluster has access to a shared drive that would work that way it may work.

Josh

On Thu, Sep 17, 2015 at 11:05 PM, Fernandez Edgar edgar.fernandez@umontreal.ca wrote:

Hello gents,

I figured out something: it has nothing to do with galaxy but maybe one of you could explain me why I’m having this behavior :

I connect ssh on the torque master (where the galaxy web service is running) and:

1.   su - galaxy
   
   

2.   cd /home/galaxy/galaxy-prod/database/job_working_directory/000
   
   
   
   

I execute a job #1234, it fails with the error mentioned below.

I execute the exact same job #1235 and while it’s running I enter the directory 1235 à job finishes successfully.

My directory /home/galaxy is an nfs mount in /etc/fstab.

Any ideas?

Cordialement / Regards,

Edgar Fernandez

De : Fernandez Edgar Envoyé : September-17-15 9:44 AM À : galaxy-dev@bx.psu.edu Objet : [galaxy-dev] need help badly

Hello gents,

I hope everyone is doing ok…

I’m in need of your help so badly it’s a little bit urgent:

So my galaxy wasn’t used in a while but the last time everything was working well:

1.   The web-site was functional
   
   

2.   The local handler upload the files locally on the master
   
   

3.   Torque was shooting all the jobs correctly and getting an answer back.
   
   
   
   

But today, one of my students tried some typical trimmomatic and all his jobs came back with the following error:

Job output not returned from cluster.

However, when I checked his datasets with that error, the view data (icon that is an eye) had some results.

Also, when I looking inside the logs, I got:

Job output not returned from cluster: [Errno 2] No such file or directory: '/home/galaxy/galaxy-prod/database/job_working_directory/000/406/galaxy_406.o'

But that file galaxy_406.o existed and had

TrimmomaticPE: Completed successfully

I tried changing the value from 0 to 5 of "retry_job_output_collection" in the config/galaxy.ini file without any success.

I’ve look through the internet and I saw a lot of people had that error but no one found a solution.

Can you guys please help?

Cordialement / Regards,

Edgar Fernandez

System Administrator (Linux)

Direction Générale des Technologies de l'Information et de la Communication

( Bur. : 1-514-343-6111 poste 16568

Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-218

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/

--

Nil mihi rescribas, tu tamen ipse veni!

-- Nil mihi rescribas, tu tamen ipse veni! ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/