September 2012 - galaxy-dev - lists.galaxyproject.org

FTP problem
by Rémy Dernat 14 Sep '12

14 Sep '12

Hi, I follow this tutorial to configure FTP with galaxy : wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP However, even if I can [ log into / upload file to / donwload file to ] my ftp server with my Galaxy account, I can not retrieve these data on my Galaxy Web server. It says : "Your FTP upload directory contains no files." The directory with my email account is created and the file exists. I have to say that my proftpd account is a special account : ftpgalaxy (like the howto). This is not the user used by the galaxy daemon process. If I change the user to the galaxy daemon user (which is 'galaxy'), I can not login anymore into my ftp server. The only way to login on my ftp server is tu put my UID/GID used by 'ftpgalaxy' account in my SELECT query. Those two servers are not on the same computer. But those 2 accounts exists with the same UID/GID on both computers. Any help would be appreciated. Regards, Remy

1 1

none availability genome.fa files through mapping modules on local server
by Sandrine Imbeaud 14 Sep '12

14 Sep '12

Hello, I apologize for this probably very simple application. We have installed our own Galaxy server and started using in-house the NGS modules. However, during the mapping procedure using either BWA for illumina or BFAST tools, no reference genome index is available. To solve the problem, we have followed the tutorials and have uploaded the hg19.fa file and put it locally in the Galaxy-dist/database folder. We also have modified the *_index.loc files indicating the path to the file. We restarted the Galaxy server. However, still no reference is available through the NGS mapping modules. Is there anyone that may help use solving this probably simple problem? Kind regards / Sandrine

1 0

Re: [galaxy-dev] [galaxy-user] No output produced.....
by Jeremy Goecks 14 Sep '12

14 Sep '12

The problem is likely your use of the from_work_dir attribute. I can see by the logging code that you're using an old(er) version of Galaxy, and it's likely that this attribute isn't functioning correctly in your instance. We've recently improved the implementation of from_work_dir, and you'll need to update your Galaxy instance to get these improvements. Let us know if you still have problems after updating. Finally, please direct tool and installation questions to the galaxy-dev mailing list rather than the galaxy-user mailing list. The galaxy-user mailing list is for tool usage and analysis questions. Best, J. On Sep 13, 2012, at 9:06 PM, <Neil.Burdett(a)csiro.au> wrote: > Hi, > I have my own image registration tool that I’ve created on my own local instance of galaxy. > > The method takes in two images (*.nii.gz) formats and registers them together, and produces one registered *.nii.gz file and a *.trsf matrix file. > > The first issue encountered was the method was expecting *.nii.gz files as inputs but was receiving *.dat files. I navigated around this problem as shown by the files below: > > - <<tool id="RegisterAliBabaAffine" name="RegisterAffine"> > < <description>two images</description> > < <command interpreter="bash">$__root_dir__/tools/registration/reg-wrapper.sh $moving $fixed $outputTRSF $outputImage</command> > - < <inputs> > < <param format="binary" name="moving" type="data" label="Moving Image" /> > < <param format="binary" name="fixed" type="data" label="Fixed Image" /> > < <param type="hidden" name="outputTRSF" value="output.trsf" label="trsf file" help="Output File must have .trsf extension" /> > < <param type="hidden" name="outputImage" value="output.nii.gz" label="Image output file" help="Output Image File must have .nii.gz extension" /> > </inputs> > - < <outputs> > < <data format="input" name="output_TRSF" from_work_dir="output.trsf" /> > < <data format="input" name="output_Image" from_work_dir="output.nii.gz" /> > </outputs> > < <help>This tool uses Affine Registration to register two images.</help> > </tool> > > #!/bin/bash > MOVING=`mktemp --suffix .nii.gz` > FIXED=`mktemp --suffix .nii.gz` > cat $1 > $MOVING > cat $2 > $FIXED > /usr/local/MILXView.12.08.1/BashScripts/RegisterAliBabaAffine -m $MOVING -f $FIXED -t $3 -o $4 > RC=$? > if [[ $RC == 0 ]]; then > OUTPUTTRSF=`mktemp --suffix .trsf` > OUTPUTIMG=`mktemp --suffix .nii.gz` > cat $OUTPUTTRSF > $3 > cat $OUTPUTIMG > $4 > rm $OUTPUTTRSF > rm $OUTPUTIMG > fi > > rm $MOVING > rm $FIXED > > exit $RC > > This allows them to pass the *.nii.gz files that the registration method is expecting. > > Everything works fine and I can see output generated in the job_working_dir and the history turns green... > > galaxy@bmladmin-OptiPlex-745:~$ ls -lrt ~/galaxy-dist/database/job_working_directory/000/27/ > total 2940 > -rw------- 1 galaxy nogroup 0 Sep 13 10:15 tmpRfHsOP_stderr > -rw-r--r-- 1 galaxy nogroup 241 Sep 13 10:35 output.trsf > -rw------- 1 galaxy nogroup 80 Sep 13 10:35 tmplmK0V2_stdout > -rw-r--r-- 1 galaxy nogroup 2998272 Sep 13 10:38 output.nii.gz > > However, the problem occurs when the files are copied from ~/galaxy-dist/database/job_working_directory/000/27/ to ~/galaxy-dist/database/files/000/. When this happens the files become size = 0. > > Any ideas? > > -rw-r--r-- 1 galaxy nogroup 0 Sep 13 09:36 /home/galaxy/galaxy-dist/database/files/000/dataset_40.dat > -rw-r--r-- 1 galaxy nogroup 0 Sep 13 09:36 /home/galaxy/galaxy-dist/database/files/000/dataset_41.dat > -rw-r--r-- 1 galaxy nogroup 0 Sep 13 10:38 /home/galaxy/galaxy-dist/database/files/000/dataset_43.dat > -rw-r--r-- 1 galaxy nogroup 0 Sep 13 10:38 /home/galaxy/galaxy-dist/database/files/000/dataset_42.dat > > > The output in galaxy.log indicates it is successful: > > /home/galaxy/galaxy-dist/tools/registration/reg-wrapper.sh /home/galaxy/galaxy-dist/database/files/000/dataset_23.dat /home/galaxy/galaxy-dist/database/files/000/dataset_20.dat output.trsf output.nii.gz galaxy.jobs DEBUG 2012-09-13 10:38:10,334 The tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs DEBUG 2012-09-13 10:38:10,361 finish(): Moved /home/galaxy/galaxy-dist/database/job_working_directory/000/27/output.trsf to /home/galaxy/galaxy-dist/database/files/000/dataset_42.dat as directed by from_work_dir galaxy.jobs DEBUG 2012-09-13 10:38:10,380 finish(): Moved /home/galaxy/galaxy-dist/database/job_working_directory/000/27/output.nii.gz to /home/galaxy/galaxy-dist/database/files/000/dataset_43.dat as directed by from_work_dir galaxy.jobs DEBUG 2012-09-13 10:38:10,609 job 27 ended > > Is the issue copying *.nii.gz files and *.trsf file into *.dat files? Anyway around this? > > I’ve also modified ~/galaxy-dist/lib/galaxy/jobs/__init__.py (line 363) to change shutil.move > > To shutil.copy2 (same results) > > Also put in a different output path to copy to. But essentially we have files with size in ~/galaxy-dist/database/job_working_directory/000/id/, but they files are size 0 after the move into ~/galaxy-dist/database/files/000 > > > Thanks > > Neil > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/

1 0

markupsafe module confliction problem
by Fenglou Mao 14 Sep '12

14 Sep '12

Galaxy imported a library which already existed in the python system folder. See the error message below. Is that because RedHat 6 used a "python-marksafe", but Galaxy installation program did not recognize it? Or Galaxy require a higher version of markupsafe module? /home/xxxxx/work/galaxy-dist/ lib/galaxy/__init__.py:79: UserWarning: Module markupsafe was already imported from /usr/lib64/python2.6/site-packages/markupsafe/__init__.pyc, but /home/xxxxx/work/galaxy-dist/eggs/MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg is being added to sys.path self.check_version_conflict() The system package is python-markupsafe-0.9.2-4.el6.x86_64 Galaxy has a newer version, it is MarkupSafe-0.12-py2.6-linux-x86_64-ucs4.egg Any idea how to resolve this problem?

1 0

Re: [galaxy-dev] Small enhancment to the tarball parsing.
by Greg Von Kuster 14 Sep '12

14 Sep '12

Hello Bjorn, I've committed your patch in change set revision 7669:5088185a5bca, which is currently running on the test Galaxy tools shed. The fix will be moved to the main Galaxy tool shed at the time of the next Galaxy release, currently scheduled for the end of next week. Regarding the main tool shed tracking Galaxy dist, that means that the main tool shed gets updated only when the Galaxy dist repository gets updated - this is now going to occur about every two weeks. Running the code from the Galaxy central repository may not fully work if going against the main Galaxy tool shed, but the test Galaxy tool shed tracks the Galaxy central code, so things should work there. Until recently, both the main tool shed and the test tool shed tracked Galaxy central, but the community requested that the main tool shed tracks the stable Galaxy dist repository instead of the development Galaxy central repository. Regarding contributed fixes, please keep them coming, as they are very much appreciated! I may have misunderstood one of your previous messages if I responded in a way that was confusing. If so, I apologize. Thanks so much for all of your help and contributions! Greg Von Kuster On Sep 10, 2012, at 2:45 PM, Björn Grüning wrote: > Hi Greg, > > what did you think about the attached patch? > In my setup i have 'deployment folder' where i symlink all different > wrapper under a new logical structure for toolshed deployment. That > directory gets tarred. Unfortunately a few files ending up as hard-link > in the tarball. At least TarInfo returns true. > > TarInfo.islnk() > Return True if it is a hard link. > > If im not wrong it does not hurt to also allow hard-links. Also it would > be nice to know which file is the offender, if a tarball gets rejected. > > One question to the galaxy-dist. The toolshed is now tracking > galaxy-dist? What does that mean? That all development is happening in > galaxy-dist and not in central? That would mean that it is not possible > to run central with both galaxy and the toolshed? Im a little bit > confused. When i will find a bug in galaxy or the toolshed in > galaxy-central the bug should not be fixed and posted? > > Thanks, > Bjoern > > > > -- > Björn Grüning > Albert-Ludwigs-Universität Freiburg > Institute of Pharmaceutical Sciences > Pharmaceutical Bioinformatics > Hermann-Herder-Strasse 9 > D-79104 Freiburg i. Br. > > Tel.: +49 761 203-4872 > Fax.: +49 761 203-97769 > E-Mail: bjoern.gruening(a)pharmazie.uni-freiburg.de > Web: http://www.pharmaceutical-bioinformatics.org/ > <toolshed_detailed_error.diff>

1 0

enable emboss tools
by Paul Tanger 14 Sep '12

14 Sep '12

Hi, I'm trying to enable all the emboss tools on a local galaxy install. I found this tidbit (below) which I thought might help, but when I look at the tool_conf.xml, I don't see a list of emboss tools to uncomment. What should I try next? Kelly Vincent kpvincent at bx.psu.edu Fri Jan 14 11:26:37 EST 2011 Previous message: [galaxy-dev] Restricted user builds Next message: [galaxy-dev] API connection error Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Rob, When you start the Galaxy server, it looks in tool_conf.xml for which tools to load. Note that functional tests use tool_conf.xml.sample, and we leave the EMBOSS tools commented out in that file, so your tool_conf.xml file will still have them commented out if you haven't changed that. Uncomment the relevant lines and the EMBOSS tools should show up. After making changes to tool_conf.xml, you'll have to restart the server.

3 7

Picard Reorder tool's reference genome box
by Shantanu Pavgi 14 Sep '12

14 Sep '12

Our local Galaxy instance doesn't have an auto-fill option working with Picard-Reorder-SAM/BAM tool's 'select a reference genome' box. We do have a 'tool-data/picard_index.loc' file with genome build-ids for picard tools. Do we need to configure anything else to get reference genome field working? Any pointers will be really helpful. Galaxy revision: ec29ce8e27a1 Picard tools version: 1.56 -- Thanks, Shantanu

1 0

toolshed upload problem
by David Hoover 14 Sep '12

14 Sep '12

I've created a repository in the test toolshed and tried uploading gzipped tarballs with all files required. I have several tool config files (.xml) wrapping a single python script with some tool-data dependencies. If I upload a single tool config file with the dependencies, everything is ok. However, if I try uploading more than one config file at a time, it complains with the error message for all but one of the config files: *.xml - This file requires an entry for "'gatk2_picard_indexes'" in the tool_data_table_conf.xml file. Upload a file named tool_data_table_conf.xml.sample to the repository that includes the required entry to correct this error. The tool_data_table_conf.xml.sample file already exists within the tarball. Nothing changes if I upload the file separately. Any ideas? David Hoover Helix Systems Staff

2 2

tool_data_table.xml and *.loc not installed with repository
by Birgit Crain 14 Sep '12

14 Sep '12

Hi I got a problem with the tool_data_table.xml and tool-data/*.loc files when testing the download of my repository. My repository contains a *.loc.sample file and a tool_data_table_conf.xml.sample (they were unchanged from my last upload to the repository). When installing the repository I get the following messages: 172.22.128.28 - - [10/Sep/2012:11:46:16 -0700] "GET /admin_toolshed/browse_tool_shed?tool_shed_url=http%3A%2F%2Ftesttoolshed.g2.bx.psu.edu%2F HTTP/1.1" 302 - "http://lx-galaxy-1:8080/admin_toolshed/browse_tool_sheds" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" 172.22.128.28 - - [10/Sep/2012:11:46:39 -0700] "GET /admin_toolshed/prepare_for_install?tool_shed_url=http://testtoolshed.g2.bx.psu.edu/&repository_ids=96d1ef4698eaa22d&changeset_revisions=982e19c29ec0 HTTP/1.1" 200 - "http://testtoolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?rep…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" Adding new row (or updating an existing row) for repository 'testing_cgatools' in the tool_shed_repository table. galaxy.web.controllers.admin_toolshed DEBUG 2012-09-10 11:47:27,751 Appending to tool panel section: CG tools 172.22.128.28 - - [10/Sep/2012:11:47:27 -0700] "POST /admin_toolshed/prepare_for_install?encoded_repo_info_dicts=606805d7b3160cf8502df2b105df5510f61a250c%3A7b2274657374696e675f636761746f6f6c73223a205b22746869732069732061207465737420222c2022687474703a2f2f74657374746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f62637261696e2d636f6d706c65746567656e6f6d6963732f74657374696e675f636761746f6f6c73222c2022393832653139633239656330222c20223233222c202262637261696e2d636f6d706c65746567656e6f6d696373222c206e756c6c5d7d&includes_tools=True&tool_shed_url=http%3A%2F%2Ftesttoolshed.g2.bx.psu.edu%2F&includes_tool_dependencies=False HTTP/1.1" 302 - "http://lx-galaxy-1:8080/admin_toolshed/prepare_for_install?tool_shed_url=ht…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" 172.22.128.28 - - [10/Sep/2012:11:47:27 -0700] "GET /admin_toolshed/initiate_repository_installation?shed_repository_ids=f2db41e1fa331b3e&reinstalling=False&encoded_kwd=80e690a1cf32f1294e27873f210c5344dc427413%3A7b22737461747573223a2022646f6e65222c2022746f6f6c5f70616e656c5f73656374696f6e5f6b6579223a202273656374696f6e5f63675f746f6f6c73222c2022746f6f6c5f736865645f7265706f7369746f72795f696473223a205b2266326462343165316661333331623365225d2c2022736865645f746f6f6c5f636f6e66223a20222e2f736865645f746f6f6c5f636f6e662e786d6c222c2022746f6f6c5f70617468223a20222e2e2f736865645f746f6f6c73222c20227265706f5f696e666f5f6469637473223a205b22363036383035643762333136306366383530326466326231303564663535313066363161323530633a376232323734363537333734363936653637356636333637363137343666366636633733323233613230356232323734363836393733323036393733323036313230373436353733373432303232326332303232363837343734373033613266326637343635373337343734366636663663373336383635363432653637333232653632373832653730373337353265363536343735326637323635373036663733326636323633373236313639366532643633366636643730366336353734363536373635366536663664363936333733326637343635373337343639366536373566363336373631373436663666366337333232326332303232333933383332363533313339363333323339363536333330323232633230323233323333323232633230323236323633373236313639366532643633366636643730366336353734363536373635366536663664363936333733323232633230366537353663366335643764225d2c2022696e7374616c6c5f746f6f6c5f646570656e64656e63696573223a2066616c73652c2022696e636c756465735f746f6f6c73223a20747275652c2022746f6f6c5f736865645f75726c223a2022687474703a2f2f74657374746f6f6c736865642e67322e62782e7073752e6564752f222c20226d657373616765223a2022222c2022696e636c756465735f746f6f6c5f646570656e64656e63696573223a2066616c73657d HTTP/1.1" 200 - "http://lx-galaxy-1:8080/admin_toolshed/prepare_for_install?tool_shed_url=ht…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" 172.22.128.28 - - [10/Sep/2012:11:47:31 -0700] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://lx-galaxy-1:8080/admin_toolshed/initiate_repository_installation?she…" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" galaxy.tools.data WARNING 2012-09-10 11:47:32,181 Cannot find index file 'tool-data/cg_crr_files.loc' for tool data table 'cg_crr_files' galaxy.tools.data DEBUG 2012-09-10 11:47:32,181 Loaded tool data table 'cg_crr_files' The 'cg_crr_files.loc' is not created and the tool_data_table.xml does not get updated with the information from the tool_data_table.xml.sample file in the repository. Is this a bug or has something changed that I'm not aware of with regards to the tool-data files and tool_data_table.xml? Thanks Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, Inc. (650) 428-6023 office | (408) 605-3938 mobile bcrain(a)completegenomics.com<mailto:bcrain@completegenomics.com> ________________________________ The contents of this e-mail and any attachments are confidential and only for use by the intended recipient. Any unauthorized use, distribution or copying of this message is strictly prohibited. If you are not the intended recipient please inform the sender immediately by reply e-mail and delete this message from your system. Thank you for your co-operation.

2 3

problem in Fastq tool
by Felix Homa 14 Sep '12

14 Sep '12

Hello, We noticed a problem with the tool Filter Fastq. It seemed to mishandle the paired-end checkbox. Here is the XML which seems to not work: <param name="paired_end" label="This is paired end data" type="boolean" truevalue="paired_end" falsevalue="single_end" checked="False"/> Concretely, $paired_end.value return True or False as value and not "paired_end" or "single_end", so in the python code execution, #if $paired_end.value == 'single_end': qual_scores_split = [ qual_scores ] #else: qual_scores_split = [ qual_scores[ 0:int( len( qual_scores ) / 2 ) ], qual_scores[ int( len( qual_scores ) / 2 ): ]] Always go in the else, always splitting reads even when they are single_end. It induces wrong results in our read filtering. I hope the explanation is clear enough Best regards, Felix

2 1