Yes. Exactly.

I created a few cards from some of your specific suggestions:

https://trello.com/c/oC9iC4dx
https://trello.com/c/j9RQxd6M
https://trello.com/c/MFyTxcgq

>From my perspective - empowering users to find workflows however and
request missing tools and giving deployers access to more easily
install missing tools from the workflow screen would be bigger wins -
but are perhaps a little more challenging.

There is a lot that needs to be done to improve workflow importing
generally and workflow importing from the Tool Shed specifically. I
would call the current implementation something of a prototype - the
deployer experience is confusing, the user experience is less than
ideal, I am pretty sure Galaxy's concept of the workflows and their
lineages and disconnected from the tool sheds - so things like updates
haven't been thought through, etc....

The problem is probably obvious - the ambitions of the tool shed and
workflow subsystems are exceeded by the number of developer hours
dedicated to them. Given that more people use tools than workflows in
the tool shed and by wide margin - I think the current devteam focus
on updating tools, improving the experience for tool developers,
improving tool search, trying to fix tool testing, imporving
dependency management, etc... makes sense. On the flip side of the
coin - I have put a lot of effort into workflows but focused on
expressiveness, etc... - again because I think those are more pressing
problems right now than tool shed publication. I doesn't mean anyone
thinks this isn't important - it is just we have to triage right?

Three questions I have when thinking about workflows in the tool shed -

1) I don't think anyone would disagree that we need to spend more time
on these things - but do you think we are missing allocating our time
on other things related to the tool shed and workflows and should
instead we be more focused on this use case - workflows coming from
the tool shed?

2) Is there anything I or anyone else can do to encourage more outside
development? It is no secret that my focus is generally on doing
things that will encourage other developers to build awesome stuff.
Any ideas how to create incentives for hacking on the tool shed.

3) Does putting workflows in the tool shed make sense fundamentally -
if we do find time to invest a lot more work on this - should
enhancing that process be the focus or should we find/build something
more tailored to workflow distribution and optimized for user facing
applications (the tool shed is meant to be deployer facing instead of
user facing IMO).

-John

On Wed, Apr 1, 2015 at 9:56 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
> Hello all,
>
> I've been playing with the Tool Shed lately for publishing
> Galaxy workflows - and realised the download process is
> still very complicated, especially the final hurdle - importing
> the workflow in order to actually use it.
>
> For the following example I am starting from a clean state -
> an almost fresh Galaxy installation with NOTHING as yet
> installed from the Tool Shed (see end of email), and I want
> to install a single workflow:
>
> https://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species
>
> 1. Goto Galaxy in browser, eg http://localhost/galaxy-dev/
> 2. Goto "User", "Register"
> 3. Create account using address added to admin_users
> earlier in config/galaxy.ini
> 4. Click "Return to the home page"
> 5. Since logged in as admin, can goto "Admin" settings
> 6. Select "Search and browse tool sheds"
> 7. Select "Galaxy Main Tool Shed"
> 8. Enter "species" in the search box
> 9. Select "blast_top_hit_species"
> 10. Note the list of dependencies
> 11. Click "Install into Galaxy"
> 12. Leave the dependency handling ticked.
> 13. Ponder that the section selection will apply to? The current
> repository has no tools but some of the dependencies do...
> I don't want to bundle the BLAST tools in the same section as
> the text manipulation tools. Go with the default (don't pick a section).
> 14. Click "install"
> 15. Wait as the following are installed:
>
> Name Description Owner Revision
> blast_top_hit_species Workflow to count species of top nr BLASTX hits
> of a transcriptome peterjc db0c1bb92308
> ncbi_blast_plus NCBI BLAST+ devteam 2fe07f50a41e
> blast_datatypes BlastXml and other blast datatypes devteam 5482a8cd0f36
> package_blast_plus_2_2_29 NCBI BLAST+ 2.2.29 (binaries only) iuc a2ec897aac2c
> package_biopython_1_65 Downloads and compiles version 1.65 of the
> Biopython package. biopython dc595937617c
> package_numpy_1_9 Contains a tool dependency definition that downloads
> and compiles version 1.9 of the the python numpy package. iuc
> 84e97f5cd3ab
> package_atlas_3_10 Contains a tool dependency definition that
> downloads and compiles version 3.10 of the the Automatically Tuned
> Linear Algebra Software. iuc 37ff27ea37ed
> fasta_to_tabular FASTA-to-Tabular converter devteam 9d189d08f2ad
> sample_seqs Sub-sample sequences files (e.g. to reduce coverage)
> peterjc 02c13ef1a669
> unique Advanced unique lines program using GNU sort bgruening 7ce75adb93be
>
> 16. Get bored, especially if the page appears to have stalled.
> Try reloading the page which goes back to the "Admin" welcome.
> Now how do I get back to the installation status...
> 17. Click on "Manage installed tool shed repositories"
> 18. If at this point some of the dependency requirements are in
> error state, try a round of repairing repositories (and watch at
> the terminal, e.g. compiling numpy is slow).
> 19. Admire all the nice green "tick" icons and "Installed" entries.
> 20. Notice the tiny grey "cog" icon against the workflow
> repository, "blast_top_hit_species".
> 21. Read the legend, "This repository contains exported workflows",
> which is encouraging but there is nothing on the page hinting at
> *how* to actually use the new workflow.
> 22. Try the context menu to see if that lets you use the workflow. No.
> 23. At this point I fully explored the workflow menus looking for an
> import option, and gave up and returned to "Manage installed tool
> shed repositories".
> 23. Ignore the context menu, click on "blast_top_hit_species".
> 24. Click on the top right "repository actions" menu
> 25. Find nothing about importing a workflow.
> 26. Click on the "Species of top BLAST hits" entry under
> "Contents of this repository"
> 27. Observe pretty SVG image of workflow.
> 24. Click on the top right "repository actions" menu
> 25. Notice the menu now offers "import workflow to galaxy".
> 26. Click on "import workflow to galaxy".
> 26. Back at SVG with green message "Workflow Species of top
> BLAST hits imported successfully. "
> 27. Return to "Analyze Data" (or "Workflows") and finally the
> workflow is available (although probably only for the Galaxy
> administrator's personal account).
> 28. Cry because all the new tools are dumped in the top level
> tool menu (see step 13).
>
> Possible suggestions to make this easier:
>
> At step 21, include a clue in the legend?
>
> At step 22 and 25, include "Import all these workflows" in menu?
>
> At step 24, change the button to "repository/workflow actions"
> since it includes some actions specific to the viewed workflow?
>
> How about the common use case where the Galaxy Admin
> was asked to import the workflow for a specific user? Can
> we have an action "Import workflow for specific user"?
>
> Also I think the tool section behaviour for dependencies needs
> clarification (step 13/28). From a tool author perspective, I would
> like to be able to set the default section for each tool (see also
> past requests to set the order they are added to the tool panel).
>
> Regards,
>
> Peter
>
> --
>
>
> This was how I setup my current galaxy development instance
> tracking the default dev branch on github:
>
> $ cd ~/repositories
> $ git clone https://github.com/galaxyproject/galaxy.git
> $ cd galaxy
>
> I made minor config changes, including adding myself as
> an admin user via the admin_users setting:
>
> $ emacs config/galaxy.ini
> ...
>
> I also setup Apache to serve this instance at
> http://localhost/galaxy-dev/ and set ToolShed path to be
> ~/repositories/shed_tools/
>
> This was working fine. To hopefully reset this back to a what
> is almost a clean slate (for this bug report), I next removed
> the auto-generated XML files, data files and SQLite database,
> and tool shed files:
>
> $ rm -f ~/repositories/galaxy/*.xml
> $ rm -f ~/repositories/galaxy/config/*.xml
> $ rm -f ~/repositories/galaxy/database/universe.sqlite
> $ rm -rf ~/repositories/galaxy/database/files/*
> $ rm -rf ~/repositories/shed_tools/
>
> (Dangerous - which is why I explicitly used the full paths!)
>
> I then restart Galaxy,
>
> $ cd ~/repositories/galaxy
> $ ./run.sh
> ...
>
> Note it creates a new empty SQLite database etc.
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/