- galaxy-dev - lists.galaxyproject.org

Count characters in column
by Thyssen, Gregory - ARS 18 Mar '13

18 Mar '13

Dear Galaxy Community I frequently use pileup files in galaxy. The sequence read column contains characters like .,AaCcTtGgNn$* etc. I would like to be able to add a column to my tables that represents the number of instances of a given character in the read column. I can do this in excel with this type of formula: =len(I2)-len(substitute(I2,"A","") But of course, Excel can only handle a limited number of total lines. Do we have this functionality in Galaxy? If so how? If not..... please? Thank you, Gregory Thyssen, PhD Molecular Biologist Cotton Fiber Bioscience USDA-ARS-Southern Regional Research Center 1100 Robert E Lee Blvd New Orleans, LA 70124 gregory.thyssen(a)ars.usda.gov 504-286-4280 This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

2 1

Re: [galaxy-dev] Issue 634 in galaxy-central: Optional column select arguments?
by Peter Cock 18 Mar '13

18 Mar '13

On Wed, Nov 30, 2011 at 8:51 PM, Bitbucket <issues-reply(a)bitbucket.org> wrote: > --- you can reply above this line --- > > Issue 634: Optional column select arguments? > https://bitbucket.org/galaxy/galaxy-central/issue/634/optional-column-selec… > > Nate Coraor / natefoo on Wed, 30 Nov 2011 21:51:21 +0100: > > Changes: > responsible: nothing -> dan > This issue was moved to Trello as: https://trello.com/card/634-optional-column-select-arguments/506338ce32ae45… I'd still like to define an optional column parameter as follows: <param name="startcol" type="datacolumn" dataref="inputtabular" multiple="False" numerical="False" optional="True" label="Column containing start co-ordindate" help="Leave empty to include everything from the first base/residue"/> i.e. I want the user to either select ONE of the columns of the input tabular data file (in which case getting "1" or "c1" etc as the string is fine) OR pick no column at all (in which case I want an empty string or similar in the command string. The bug is that Galaxy defaults to column one, and the UI doesn't allow no column to be picked. It occurred to me a simple UI idea would be to include a dummy 'No column' entry in the drop down before column 1, column 2, etc. Would that be acceptable? Peter

1 0

Re: [galaxy-dev] BioBlend: Problem Running Example file
by Clare Sloggett 18 Mar '13

18 Mar '13

Oops sorry, meant to keep the list cc'd - resending. On 18 March 2013 16:18, Clare Sloggett <sloc(a)unimelb.edu.au> wrote: > Hi Rob, > > Were you using a very old version of the library and examples or was it > quite recent? In any case, try grabbing the script from > https://github.com/afgane/bioblend/tree/master/docs/examples , we have > done a lot of bugfixing and documentation recently. If you just want to > test your setup there are also now some much simpler, less end-to-end > example scripts that don't use any admin permissions, and we'll add a few > more. > > The create_library step is the step where you're going to run into > problems if you don't have admin rights to the Galaxy instance. If you're > running on localhost though you should be able to set up the account as an > admin account. > > Finally... we've put up some much better docs on using workflows (and > libraries) in bioblend, which might be useful: > http://bioblend.readthedocs.org/ and specifically > http://bioblend.readthedocs.org/en/latest/api_docs/galaxy/docs.html#run-a-w… > > A lot of this is very recent so if you run into any bugs or anything that > is just not clear, let me know, the feedback is very helpful! > > Cheers, > Clare > > On 8 March 2013 16:01, Rob Leclerc <robert.leclerc(a)gmail.com> wrote: > >> I had trouble running blend4j, so I tried to jump into python (a language >> I have little experience with). >> >> I tried running the example run_import_workflow.py >> >> *% python run_imported_workflow.py http://localhost:80808c25bc83f6f9e4001dd21eb7b64f063f >> * >> >> but I get an error: >> >> Initiating Galaxy connection >> Importing workflow >> Creating data library 'Imported data for API demo' >> >> Traceback (most recent call last): >> File "run_imported_workflow.py", line 53, in <module> >> library_dict = gi.libraries.create_library(library_name) >> File >> "build/bdist.macosx-10.6-intel/egg/bioblend/galaxy/libraries/__init__.py", >> line 27, in create_library >> File "build/bdist.macosx-10.6-intel/egg/bioblend/galaxy/client.py", >> line 53, in _post >> File "build/bdist.macosx-10.6-intel/egg/bioblend/galaxy/__init__.py", >> line 132, in make_post_request >> File >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-1.1.0-py2.7.egg/requests/models.py", >> line 604, in json >> return json.loads(self.text or self.content) >> File >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson-3.1.0-py2.7-macosx-10.6-intel.egg/simplejson/__init__.py", >> line 454, in loads >> return _default_decoder.decode(s) >> File >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson-3.1.0-py2.7-macosx-10.6-intel.egg/simplejson/decoder.py", >> line 374, in decode >> obj, end = self.raw_decode(s) >> File >> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/simplejson-3.1.0-py2.7-macosx-10.6-intel.egg/simplejson/decoder.py", >> line 393, in raw_decode >> return self.scan_once(s, idx=_w(s, idx).end()) >> simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 >> (char 0) >> >> >> Is there anything I am missing from the stock configuration which would >> cause this not to run out of the box? >> >> Cheers, >> Rob >> >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> http://lists.bx.psu.edu/ >> > > > > -- > > Clare Sloggett > Research Fellow / Bioinformatician > Life Sciences Computation Centre > Victorian Life Sciences Computation Initiative > University of Melbourne, Parkville Campus > 187 Grattan Street, Carlton, Melbourne > Victoria 3010, Australia > Ph: 03 903 53357 M: 0414 854 759 > -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759

1 0

Re: [galaxy-dev] [galaxy-iuc] Tool Dependency Packages
by Greg Von Kuster 17 Mar '13

17 Mar '13

Repositories that contain only a tool dependency definition file are a special kind of repository. The tool dependency is considered an orphan tool dependency because it is not associated with any tools in the same repository. This orphan designation is not so important in the tool shed itself, but there will soon be features on the Galaxy side that will be available for administering orphan tool dependencies installed into a Galaxy instance. The orphan designation is different in Galaxy than it is in the tool shed. The important best practice to use for these special repositories is that the version of the tool dependency package definition should never change within the repository because doing so will not allow for corrections to the definition of the older tool dependency package definition in a previous changeset revision. This is the main reason that the package version is included in the repository name. Restricting the repository to a single package version allows for any number of changeset revision fixes to the definition file. Since some of the instruction sets for installing and compiling a dependency package are fairly complex, fixes to them must be allowed. Including a tool dependency package version in the repository name also makes it easier to locate the specific package definition, although this is a secondary reason for naming them so. Let me know if this is not clear, as this is a very important detail. Greg Von Kuster On Mar 17, 2013, at 5:22 PM, Björn Grüning wrote: > Hello all! > > Anyone knows why the tool-dependency-packages have a version number in > there repository names? I want to create a new one and wondered why I > have to include the version number, because the hg-revisions can keep > track of different versions of a dependency, or? Is there any > best-practise guideline? Should we create one? > > Thanks, > Bjoern > > _______________________________________________ > galaxy-iuc mailing list > galaxy-iuc(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-iuc

1 0

hg push to remote tool shed fails with abort: authorization failed
by jorrit poelen 15 Mar '13

15 Mar '13

Hey y'all, Even through http://wiki.galaxyproject.org/ToolShedRepositoryFeatures#Pushing_changes_to… shows a working example of a hg push, I have not been able to push changes to a remote toolshed successfull. Here's an example of an unsuccessful attempt:. $ vi README $ hg commit -m 'bla' -u jorrit $ hg push pushing to http://jorrit@testtoolshed.g2.bx.psu.edu/repos/jorrit/obotest searching for changes http authorization required realm: user: jorrit password: abort: authorization failed tried this with hg versions: Mercurial Distributed SCM (version 2.4.2+20130102) and Mercurial Distributed SCM (version 2.0.2) Curious to hear how I can get a hg push to remote repo to work from commandline! -jorrit

2 3

configure loacl galaxy cluster question!
by Hakeem Almabrazi 15 Mar '13

15 Mar '13

I have followed the instructions on how to setup a local cluster closely (http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster) Frankly, the (Galaxy Configuration) section was not clear for me. I am not sure if those outlined steps should be applied to the server's universe_wsgi.ini or to the nodes'? So I might have overlooked some steps there but here is a summary of what I am doing at the server: All my nodes have been configured so the galaxy user can ssh/scp between nodes without pwd. Also the galaxy is a sudo in the galaxy (server). Torque has been configured and tested between the nodes. So the cluster is working fine. In universal_wsgi.ini file at the server node -------------------------- Start_job_runners=pbs,drama Drama_external_runjob_~ Drama_external_killer~ External_chown_~ Pbs_application_server = galaxyhost (server) Pbs_stage_path=/tmp/galaxy_stage/ Pbs_dataset_server= galaxyhost (server) ##This is the same like pbs_application_server Also, ln -s /nfsexport/galaxy_stage /usr/local/galaxy/galaxy-dis/database/tmp outputs_to_working_directory= False (if I changed this to True, the galaxy will not start) --------------------------------------------- After restarting the galaxy at the server node. The job seems to be submitted and its status is "R". When I "top" the processes on the node where the job was sent to, I see two processes; ssh and scp ran by the galaxy server. This tells me something is being copied over to the node. But I am not sure what and to where? After while the job status changed to "W". qstat Job id Name User Time Use S Queue ------------------------- ---------------- --------------- -------- - ----- 68.ngsgalaxy01 ...xy(a)idtdna.com galaxy 0 W batch Here is what I say from the log when the job is sent. >>>>>>>>>>>>>>>>>> galaxy.jobs DEBUG 2013-03-15 15:34:54,183 (341) Working directory for job is: /usr/local/galaxy/galaxy-dist/database/job_working_directory/000/341 galaxy.jobs.handler DEBUG 2013-03-15 15:34:54,183 dispatching job 341 to pbs runner galaxy.jobs.handler INFO 2013-03-15 15:34:54,231 (341) Job dispatched galaxy.tools DEBUG 2013-03-15 15:34:54,309 Building dependency shell command for dependency 'samtools' galaxy.jobs.runners.pbs DEBUG 2013-03-15 15:34:54,391 (341) submitting file /usr/local/galaxy/galaxy-dist/database/pbs/341.sh galaxy.jobs.runners.pbs DEBUG 2013-03-15 15:34:54,391 (341) command is: PACKAGE_BASE=/usr/local/galaxy/software/samtools/0.1.16; export PACKAGE_BASE; . /usr/local/galaxy/software/samtools/0.1.16/env.sh; samtools flagstat "/usr/local/galaxy/galaxy-dist/database/files/000/dataset_319.dat" > "/usr/local/galaxy/galaxy-dist/database/files/000/dataset_384.dat" galaxy.jobs.runners.pbs DEBUG 2013-03-15 15:34:54,394 (341) queued in default queue as 70.ngsgalaxy01.idtdna.com galaxy.jobs.runners.pbs DEBUG 2013-03-15 15:34:54,966 (341/70.ngsgalaxy01.idtdna.com) PBS job state changed from N to R >>>>>>>>>>>>>>>>>> Here is the log when the ssh/scp on the node is finished. >>>>>>>>>>>>>>>>>>>> galaxy.jobs.runners.pbs DEBUG 2013-03-15 15:37:00,815 (341/70.ngsgalaxy01.idtdna.com) PBS job state changed from R to W >>>>>>>>>>>>>>>>>>>> Here is the log when I qdel that job >>>>>>>>>>>>>>>>>>>> galaxy.jobs.runners.pbs WARNING 2013-03-15 15:39:20,016 Exit code was invalid. Using 0. galaxy.jobs DEBUG 2013-03-15 15:39:20,033 (341) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /usr/local/galaxy/galaxy-dist/database/job_working_directory/000/341 galaxy 10020 galaxy.jobs ERROR 2013-03-15 15:39:20,071 (341) Failed to change ownership of /usr/local/galaxy/galaxy-dist/database/job_working_directory/000/341, failing Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 336, in finish self.reclaim_ownership() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 909, in reclaim_ownership self._change_ownership( self.galaxy_system_pwent[0], str( self.galaxy_system_pwent[3] ) ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 895, in _change_ownership assert p.returncode == 0 AssertionError galaxy.datatypes.metadata DEBUG 2013-03-15 15:39:20,160 Cleaning up external metadata files galaxy.jobs.runners.pbs WARNING 2013-03-15 15:39:20,172 Unable to cleanup: [Errno 2] No such file or directory: '/usr/local/galaxy/galaxy-dist/database/pbs/341.o' galaxy.jobs.runners.pbs WARNING 2013-03-15 15:39:20,173 Unable to cleanup: [Errno 2] No such file or directory: '/usr/local/galaxy/galaxy-dist/database/pbs/341.e' galaxy.jobs.runners.pbs WARNING 2013-03-15 15:39:20,173 Unable to cleanup: [Errno 2] No such file or directory: '/usr/local/galaxy/galaxy-dist/database/pbs/341.ec' 10.7.10.201 - - [15/Mar/2013:15:39:22 -0500] "GET /api/histories/5a1cff6882ddb5b2 HTTP/1.0" 200 - "http://10.7.10.31/history" "Mozilla/5.0 (Windows NT 5.2; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0" 10.7.10.201 - - [15/Mar/2013:15:39:22 -0500] "GET /api/histories/5a1cff6882ddb5b2/contents?ids=bbbfa414ae315caf HTTP/1.0" 200 - "http://10.7.10.31/history" "Mozilla/5.0 (Windows NT 5.2; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0" >>>>>>>>>>>>>>>>>>>>>> Is there anything I am not doing or doing wrong? Regards,

1 0

Login issue with apache proxy
by Lionel SPINELLI 15 Mar '13

15 Mar '13

Hello all, I recently installed a galaxy server in my lab with Apache proxy and load balancing. Just after the installation, I was able to login correctly and everything seemed to work fine. The day after, login issues appeared. Login from the login page is not possible while login from other page seems to wrok even if the main page display the same login menu (Login/register) and not the user options. I have searched in the mailing list archive and saw some related issues linked to caching in browser. So I have tried to clear several time all my history (cookies and others) but the login issue remains. I am using firefox because I know Chrome causes some other issues. I have installed Galaxy from the last tarball obtained from thelink in the page http://wiki.galaxyproject.org/Admin/Get%20Galaxy Can somebody tell me how can I solve this issue? Thanks regards Lionel

1 0

galaxy from subdirectory - new history with a page refresh
by Shantanu Pavgi (Campus) 14 Mar '13

14 Mar '13

I have setup a galaxy instance from the root host (without sub-directory) and it's working fine. I have setup another galaxy instance on the same system, but it's being served from a sub-directory using the same front-end Apache. The sub-directory instance creates a new history on every visit or page refresh. A page refresh occurs immediately after submitting a job so it doesn't appear in the right side history panel (by that time Galaxy has created a new history). Essentially Galaxy is creating one job per history! The cookie_path and prefix values are configured properly in the universe_wsgi.ini file. The Apache configuration contains sub-directory specific rewrite rules as well. Any clues on how to debug this further would be helpful. Anyone else seen such issue before? -- Thanks, Shantanu

1 0

Error of GATK tools test
by shenwiyn 14 Mar '13

14 Mar '13

Hi everyone, I have installed GATK tools in my local galaxy .But when I want to test whether they work successfully ,something error appear: The error appears under the BAM files we input : Unspecified genome build, click the pencil icon in the history item to set the genome build Could anyone tell me how to test the GATK tools and why this error appear?Thank you very much for you help. shenwiyn

1 0

Trouble Shooting example_watch_folder.py
by Rob Leclerc 14 Mar '13

14 Mar '13

There were several issues and bugs I encountered getting this script to work. I've written down some notes for a quickstart for anyone else who might be following in my footsteps. 1. Create input and output directores where you will drop and emit files. In my example I used: /Users/Rob/Documents/galaxy-drop/input /Users/Rob/Documents/galaxy-drop/output 2. Edit the galaxy-dist configuration script (universe_wsgi.ini) and set: allow_library_path_paste = true 3. In example_watch_folder.py, comment out line 30 and replace it with: #library_id = library[0]['id] This causes a "key error : 0" -- remove the indexing library_id = library['id'] 4. In example_watch_folder.py change the sleep duration on line 59 to 15 seconds. If you're still getting zero byte .dat files, then you'll need to extend the duration even more. time.sleep(15) 5. CREATE A SAMPLE Workflow & Dataset: 1. Start Galaxy 2. Go to USCS and upload the Exons from chr22 to your clean history file 3. Use the Text Manipulation tool to select the first 5 lines of the gene exon file 4. Extract the workflow 5. IMPORTANT: If you don't do the following step you'll get a 500 Internal Server. 6. Change the output filename in the workflow and then save the workflow. 7. Test that the workflow is accessable by visiting 'http://<galaxy_server/api/workflows' in your browser, and then visit 'http:// <galaxy_server/api/workflows/<workflow_id> 8. Download the exons file to local 6. In your browser go to: http://localhost:8080/api/workflows and identify the id of the workflow you've just created. Let's assume it is (4b187121143038ff) and that your api_key for admin is <api_key> 7. Run the script: ./example_watch_folder.py <api_key> http://localhost:8080/api/ /Users/Rob/Documents/galaxy-drop/input /Users/Rob/Documents/galaxy-drop/output "My API Import" 4b187121143038ff 8. Copy the exons file to /Users/Rob/Documents/galaxy-drop/input when the process is complete the file will be placed in /Users/Rob/Documents/galaxy-drop/output 9. In your browser go to: http://localhost:8080/api/histories locate the recently created history and then navigate to: http://localhost:8080/api/histories/<history_id>/contents where <history_id> is the id of the history that was created when you ran the dataset on the last workflow. This is my result. The second file is the output file. [ { "id": "0d16186aaff7cbfd", "name": "Galaxy1-[Exons_chr22].bed", "type": "file", "url": "/api/histories/2d9035b3fc152403/contents/0d16186aaff7cbfd" }, { "id": "81991b32733a4fc4", "name": "top200", "type": "file", "url": "/api/histories/2d9035b3fc152403/contents/81991b32733a4fc4" ] 10. You can navigate to the JSON description of the file in your browser: http://localhost:8080/api/histories/2d9035b3fc152403/contents/81991b32733a4… and the output file can be downloaded here: http:localhost:8080/datasets/81991b32733a4fc4/display?to_ext=data Cheers! Rob Rob Leclerc, PhD <http://www.linkedin.com/in/robleclerc> <https://twitter.com/#!/robleclerc> P: (US) +1-(917)-873-3037 P: (Shanghai) +86-1-(861)-612-5469 Personal Email: rob.leclerc(a)aya.yale.edu

1 0