Hi,

It's seems like you got me ! yey ! :D I was already working on my fork for singularity , since we don't need any sudo rights for the galaxy user "setup manually by the admin and the paths hard-coded for each tool "

I will test it out and let you know !

Best regards from Greece, Nikos

On 14/06/2017 07:46 μμ, John Chilton wrote:

Sorry for the late response - but earlier this week Björn Grüning and I added various support to Galaxy's development branch for Singularity.

The following pull request added Singularity support in job running (https://github.com/galaxyproject/galaxy/pull/4175) - here job destinations may describe the paths to Singularity containers. It may also work with Singularity 2.3 to just specify Docker containers and let it auto-convert - I haven't tested that.

For "best practice" tools - these are tools that have only requirements that can resolve via Conda in the best practice Conda channels we can do even more - we can build Singularity containers and use them automatically - either ahead of time or build on the demand during job execution.

Here is the PR that added the on-demand building support for such Singularity containers to Galaxy https://github.com/galaxyproject/galaxy/pull/4185 and here is the tooling PR (https://github.com/galaxyproject/galaxy-lib/pull/64) and cached container PR (https://github.com/galaxyproject/galaxy/pull/4179) support. If you or anyone has a chance to try it out and beta test - let the list know how it goes. I plan to talk a little about this work and related Docker and Conda work at the GCC so I hope to see you there!

-John

On Tue, May 2, 2017 at 8:51 AM, Nikos Nikoloutsakos <nikoloutsa@admin.grnet.gr mailto:nikoloutsa@admin.grnet.gr> wrote:

Hello,

Are there any galaxy forks using Singularity images instead of
Docker ?
Is it trivial enough to replace "docker run" with "singularity exec" ?

Thank you,
Nikos


-- 
Nikos Nikoloutsakos
Greek Research and Technology Network (GRNET)
☎ +30 210 7471127 <tel:%2B30%20210%207471127>
✉ nikoloutsa@grnet.gr <mailto:nikoloutsa@grnet.gr>
hpc.grnet.gr <http://hpc.grnet.gr>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>

To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/ <http://galaxyproject.org/search/>

Hi Nikos,

we are discussion various enhancement with the Singularity developers, one is to suppress warnings.

Can you try this for the time being? This is working fine for me.

$tool_directory:rw,$working_directory:rw,$default_file_path:rw

Looking more closely to your error message it seems to be that Singularity is interpreting the default_ro, which should not happen. This should be replace by either ro or rw. Where do you specify this string?

Cheers, Bjoern

hi,

i have question when submitting jobs as real user

defaults = "$galaxy_root:default_ro,$tool_directory:default_ro"

singularity gives a WARNING since the home dir is already mounted "WARNING: Not mounting requested bind point (already mounted in container): default_ro"

can this be avoided ?

Thanks,

On 14/06/2017 07:46 μμ, John Chilton wrote:

...

Sorry for the late response - but earlier this week Björn Grüning and I added various support to Galaxy's development branch for Singularity.

The following pull request added Singularity support in job running (https://github.com/galaxyproject/galaxy/pull/4175) - here job destinations may describe the paths to Singularity containers. It may also work with Singularity 2.3 to just specify Docker containers and let it auto-convert - I haven't tested that.

For "best practice" tools - these are tools that have only requirements that can resolve via Conda in the best practice Conda channels we can do even more - we can build Singularity containers and use them automatically - either ahead of time or build on the demand during job execution.

Here is the PR that added the on-demand building support for such Singularity containers to Galaxy https://github.com/galaxyproject/galaxy/pull/4185 and here is the tooling PR (https://github.com/galaxyproject/galaxy-lib/pull/64) and cached container PR (https://github.com/galaxyproject/galaxy/pull/4179) support. If you or anyone has a chance to try it out and beta test - let the list know how it goes. I plan to talk a little about this work and related Docker and Conda work at the GCC so I hope to see you there!

-John

On Tue, May 2, 2017 at 8:51 AM, Nikos Nikoloutsakos <nikoloutsa@admin.grnet.gr mailto:nikoloutsa@admin.grnet.gr> wrote:

Hello,

Are there any galaxy forks using Singularity images instead of
Docker ?
Is it trivial enough to replace "docker run" with "singularity exec" ?

Thank you,
Nikos


-- 
Nikos Nikoloutsakos
Greek Research and Technology Network (GRNET)
☎ +30 210 7471127 <tel:%2B30%20210%207471127>
✉ nikoloutsa@grnet.gr <mailto:nikoloutsa@grnet.gr>
 hpc.grnet.gr <http://hpc.grnet.gr>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/>

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/ <http://galaxyproject.org/search/>

-- Nikos Nikoloutsakos Greek Research and Technology Network (GRNET) ☎ +30 210 7471127 ✉ nikoloutsa@grnet.gr hpc.grnet.gr

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/

Yup - I messed that up - sorry.

Can you try the fix in this PR ( https://github.com/galaxyproject/galaxy/pull/4208) and comment on the PR to let me know if they fix the problem?

-John

On Mon, Jun 19, 2017 at 10:24 AM, Nikos Nikoloutsakos < nikoloutsa@admin.grnet.gr> wrote:

Hi Bjoern, i'm not setting the <param id="singularity_volumes">

it seems the default_ro is not interpreting the preprocess_volumes in class SingularityContainers

volumes_raw = self._expand_volume_str(self.destination_info.get("singularity_volumes", "$defaults")) volumes = docker_util.DockerVolume.volumes_from_str(volumes_raw) #preprocessed_volumes_str = preprocess_volumes(volumes_raw, self.container_type) #volumes = docker_util.DockerVolume.volumes_from_str(preprocessed_ volumes_str)

Cheers, Nikos

On 16/06/2017 07:09 μμ, Björn Grüning wrote:

Hi Nikos,

we are discussion various enhancement with the Singularity developers, one is to suppress warnings.

Can you try this for the time being? This is working fine for me.

$tool_directory:rw,$working_directory:rw,$default_file_path:rw

Looking more closely to your error message it seems to be that Singularity is interpreting the default_ro, which should not happen. This should be replace by either ro or rw. Where do you specify this string?

Cheers, Bjoern

hi,

i have question when submitting jobs as real user

defaults = "$galaxy_root:default_ro,$tool_directory:default_ro"

singularity gives a WARNING since the home dir is already mounted "WARNING: Not mounting requested bind point (already mounted in container): default_ro"

can this be avoided ?

Thanks,

On 14/06/2017 07:46 μμ, John Chilton wrote:

Sorry for the late response - but earlier this week Björn Grüning and I added various support to Galaxy's development branch for Singularity.

The following pull request added Singularity support in job running (https://github.com/galaxyproject/galaxy/pull/4175) - here job destinations may describe the paths to Singularity containers. It may also work with Singularity 2.3 to just specify Docker containers and let it auto-convert - I haven't tested that.

For "best practice" tools - these are tools that have only requirements that can resolve via Conda in the best practice Conda channels we can do even more - we can build Singularity containers and use them automatically - either ahead of time or build on the demand during job execution.

Here is the PR that added the on-demand building support for such Singularity containers to Galaxyhttps://github.com/galaxyproject/galaxy/pull/4185 and here is the tooling PR (https://github.com/galaxyproject/galaxy-lib/pull/64) and cached container PR (https://github.com/galaxyproject/galaxy/pull/4179) support. If you or anyone has a chance to try it out and beta test - let the list know how it goes. I plan to talk a little about this work and related Docker and Conda work at the GCC so I hope to see you there!

-John

On Tue, May 2, 2017 at 8:51 AM, Nikos Nikoloutsakos <nikoloutsa@admin.grnet.gr mailto:nikoloutsa@admin.grnet.gr nikoloutsa@admin.grnet.gr> wrote:

Hello,

Are there any galaxy forks using Singularity images instead of
Docker ?
Is it trivial enough to replace "docker run" with "singularity exec" ?

Thank you,
Nikos


--
Nikos Nikoloutsakos
Greek Research and Technology Network (GRNET)
☎ +30 210 7471127 <+30%2021%200747%201127> <tel:%2B30%20210%207471127>
✉ nikoloutsa@grnet.gr <mailto:nikoloutsa@grnet.gr> <nikoloutsa@grnet.gr>
 hpc.grnet.gr <http://hpc.grnet.gr> <http://hpc.grnet.gr>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/ <https://lists.galaxyproject.org/> <https://lists.galaxyproject.org/>

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/ <http://galaxyproject.org/search/> <http://galaxyproject.org/search/>

-- Nikos Nikoloutsakos Greek Research and Technology Network (GRNET) ☎ +30 210 7471127 <+30%2021%200747%201127> ✉ nikoloutsa@grnet.gr hpc.grnet.gr

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/

-- Nikos Nikoloutsakos Greek Research and Technology Network (GRNET) ☎ +30 210 7471127 <+30%2021%200747%201127> ✉ nikoloutsa@grnet.gr hpc.grnet.gr