September 2011 - galaxy-dev - lists.galaxyproject.org

Galaxy egg fetching error? Mac OS X 10.7 (Lion)
by Whyte, Jeffrey 07 Sep '11

07 Sep '11

Hi, I've been having trouble running a local instance of Galaxy on a Mac Pro after upgrading to OS X 10.7 (Lion). My Python version is 2.7.1 and Mercurial is 1.9.1 for MacOS X 10.7. I don't have any MacPorts installed. The error I see after running the startup script is pasted at the end of this message. Thanks in advance for any help or advice. jjw ----- [~/galaxy-dist] myuserid 10:11 AM > ./run.sh Some eggs are out of date, attempting to fetch... Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 30, in <module> c.resolve() # Only fetch eggs required by the config File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: numpy==1.6.0 Fetch failed. [~/galaxy-dist] myuserid 10:12 AM >

3 10

Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion)
by Whyte, Jeffrey 07 Sep '11

07 Sep '11

That worked, Nate! Thank you very much. I really appreciate your efforts. Jeff ________________________________________ From: Nate Coraor [nate(a)bx.psu.edu] Sent: Wednesday, September 07, 2011 10:33 AM To: Whyte, Jeffrey Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion) Hi Jeff, Could you try: % python ./scripts/fetch_eggs.py >From the top-level galaxy-dist directory? I have just placed eggs for py2.7-macosx-10.6-intel-ucs2 on http://eggs.g2.bx.psu.edu/ --nate Whyte, Jeffrey wrote: > Thanks, Nate! Much appreciated. > Jeff > > > > On Sep 7, 2011, at 7:54 AM, Nate Coraor wrote: > > > Whyte, Jeffrey wrote: > >> Nate, > >> > >> For some reason, ~/galaxy-dist wasn't in my PATH. Here is the output when the script is run from the ~/galaxy-dist directory: > >> > >> [~/galaxy-dist] myuserid 03:50 PM > python ./scripts/get_platforms.py > >> macosx-10.6-intel-ucs2 > >> [~/galaxy-dist] myuserid 03:50 PM > > >> > >> Looks like it is set for Mac OS X 10.6 Snow Leopard instead of 10.7 Lion. (?) > > > > This is normal since OS X has binary compatibility with older releases, > > the Python distributors build for an older platform rather than each > > individual platform. > > > > I see there's now an Intel-only build for 10.6+, I'll work on this ASAP. > > > >> > >> Thanks, > >> Jeff > >> > >> > >> On Sep 6, 2011, at 10:18 AM, Nate Coraor wrote: > >> > >>> Whyte, Jeffrey wrote: > >>>> Thanks for the information, Nate. > >>>> > >>>> I installed Python as you described, and the error I get now is pasted below. I appreciate your help. > >>>> > >>>> Jeff > >>>> > >>>> [~/galaxy-dist] myuserid 03:06 PM > ./run.sh > >>>> Some eggs are out of date, attempting to fetch... > >>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched > >>>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched > >>>> pysam 0.4.2 couldn't be downloaded automatically. You can try > >>>> building it by hand with: > >>>> python scripts/scramble.py -e pysam > >>>> Fetch failed. > >>>> [~/galaxy-dist] myuserid 03:06 PM > > >>> > >>> Could you report back with the output of: > >>> > >>> % python ./scripts/get_platforms.py > >>> > >>> ? > >>> > >>> Thanks, > >>> --nate > >>> > >>>> ________________________________________ > >>>> From: Nate Coraor [nate(a)bx.psu.edu] > >>>> Sent: Friday, September 02, 2011 1:16 PM > >>>> To: Whyte, Jeffrey > >>>> Cc: galaxy-dev(a)lists.bx.psu.edu > >>>> Subject: Re: [galaxy-dev] Galaxy egg fetching error? Mac OS X 10.7 (Lion) > >>>> > >>>> Whyte, Jeffrey wrote: > >>>>> Hi, > >>>>> > >>>>> I've been having trouble running a local instance of Galaxy on a Mac Pro after upgrading to OS X 10.7 (Lion). My Python version is 2.7.1 and Mercurial is 1.9.1 for MacOS X 10.7. I don't have any MacPorts installed. > >>>>> > >>>>> The error I see after running the startup script is pasted at the end of this message. Thanks in advance for any help or advice. > >>>>> jjw > >>>> > >>>> Hi Jeffrey, > >>>> > >>>> We haven't yet gotten our dependencies up to speed on Lion. Could you > >>>> grab a copy of Python from python.org and use this? It'll install under > >>>> /Library/Frameworks/Python.framework, just add the correct bin/ > >>>> directory to the front of your $PATH and start Galaxy as normal. > >>>> > >>>> Sorry for the inconvenience, > >>>> --nate > >>>> > >>>>> > >>>>> ----- > >>>>> > >>>>> [~/galaxy-dist] myuserid 10:11 AM > ./run.sh > >>>>> Some eggs are out of date, attempting to fetch... > >>>>> Warning: MarkupSafe (a dependent egg of Mako) cannot be fetched > >>>>> Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched > >>>>> Warning: simplejson (a dependent egg of WebHelpers) cannot be fetched > >>>>> Traceback (most recent call last): > >>>>> File "./scripts/fetch_eggs.py", line 30, in <module> > >>>>> c.resolve() # Only fetch eggs required by the config > >>>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve > >>>>> egg.resolve() > >>>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve > >>>>> return self.version_conflict( e.args[0], e.args[1] ) > >>>>> File "/Users/myuserid/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict > >>>>> r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) > >>>>> File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 565, in resolve > >>>>> raise DistributionNotFound(req) # XXX put more info here > >>>>> pkg_resources.DistributionNotFound: numpy==1.6.0 > >>>>> Fetch failed. > >>>>> [~/galaxy-dist] myuserid 10:12 AM > > >>>>> > >>>>> > >>>>> > >>>>> ___________________________________________________________ > >>>>> Please keep all replies on the list by using "reply all" > >>>>> in your mail client. To manage your subscriptions to this > >>>>> and other Galaxy lists, please use the interface at: > >>>>> > >>>>> http://lists.bx.psu.edu/ > >>>> > >> > >> > >

1 0

Tool Shed problems with browse repository files?
by Peter Cock 07 Sep '11

07 Sep '11

Hi all, I've been having a few server errors and other problems browsing files for tool shed repositories, for example clicking to expand a folder sometimes gives a red exclamation icon with the text "Load error! (error)". Other times it does load but seems to merge several entries into single lines, e.g. clustalw -- rgclustal/ ---- README rgClustal_testout.fasta rgClustalw.xml ---- rgClustal_testin.fasta rgClustal_testout.log Or, clustalomega -- clustalomega/ ----- clustal-omega-1.0.2/ README test-data/ ----- hide_stderr.py rgClustalOmega.xml Could someone take a look at this please? Thanks, Peter

2 1

Path to tool designation in tool config xml
by Timothy Wu 07 Sep '11

07 Sep '11

Hi, I'm trying the add tool tutorial at http://wiki.g2.bx.psu.edu/Admin/Tools/Add%20Tool%20Tutorial . It works, but how do I designate the path of the tool if it's not dropped into the same directory as the tool config xml? Some executable may not be in the directory. Surely it couldn't possiblely be the case with external tools like BLAST, isn't it?! But looking at the BLAST config xmls I couldn't make out the difference and I didn't find any documentation on how to designate the path of the tool. Help appreciated, thank you. Timothy

3 4

value_label tool input param attribute doesn't work as documented
by Leandro Hermida 07 Sep '11

07 Sep '11

Hi, Looking at the example on the Tool Config Syntax wiki page: Example Assume that the tool includes an input parameter named "database" which is a select list ( e.g., assume the following inputs ): Toggle line numbers<http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#> 1 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <inputs> 2 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <param format="tabular" name="input" type="data" label="Input stuff"/> 3 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <param type="select" name="database" label="Database"> 4 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <option value="alignseq.loc">*Human (hg18)*</option> 5 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> <option value="faseq.loc">Fly (dm3)</option> 6 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> </param> 7 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-4197fe3d76b…> </inputs> Assume that the user selects the first option in the $database select list. Then the following will ensure that the tool produces a tabular data set whose associated history item has the label "Blat on *Human (hg18)*". Toggle line numbers<http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#> 1 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> <outputs> 2 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> <data format="input" name="output" label="Blat on ${database.value_label}" /> 3 <http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#CA-c1ccb05e35e…> </outputs> It shows that .value_label attribute should produce the string in between the <option> tags, not the actual value. Why does this not work? When I use it it produces the value, or am I doing something wrong? regards, Leandro

2 2

Providing Galaxy tool inputs as an (XML) file?
by Peter Cock 07 Sep '11

07 Sep '11

Hi all, Some of my medium/long term plans for Galaxy involve running one tool which then generates input parameters (and perhaps input files) for another tool. This would be particularly powerful in combination with workflows. One example is phylogenetics where you might have a model selection tool which selects assorted parameters for tree building (e.g. Gamma on or off). General model selection for classifiers might also benefit from such a flow - e.g. Model selection on a subset of the data assessed on a test data set, then running the best model on the full dataset. I can see how I might solve this on a case by case basis, where for each tool which I want to be able to call with a parameter file a wrapper script would be needed, plus the tool would have a top level parameter selection between values from a file, or values entered by the user in the normal way. At least in some niche fields such parameter files already exist - for example the Nexus file format (although many tools which support this as an input format do so in a simplistic way and only look at the alignment or trees, not the commands/settings). However, I'm envisioning a general mechanism for providing (some or all of) the inputs to a Galaxy tool via a file, which for extensibility and consistency with the rest of Galaxy would be in XML. Galaxy would need a nice viewer for this new datatype, which could appear in the history as just another dataset. However, the key new functionality would be on any tool dialogue (if a suitable tool parameter XML file was in the history) would be a new option/button to apply settings from this file to the current form. I'm imagining the new XML would include a list of tool IDs for which it can be used on (e.g. Maximum likelihood model selection might produce settings for both PhyML and RAxML, but these tools might have differently named parameters in their Galaxy tool XML files). Have the Galaxy team been thinking of anything like this already? Thanks, Peter

1 0

Re: [galaxy-dev] [galaxy-bugs] Error message when trying to dowload Manhattan plot using the web version
by Ross 06 Sep '11

06 Sep '11

Hi, Alison I've made some fixes to the Manhattan/QQ tool (galaxy-central revision #dacf97a6c663) so it now handles your data - thanks again for discovering these edge cases - the revised code is now available and working on http://test.g2.bx.psu.edu/ where your data produces plots. In particular X in your data, and for good measure Y and Mt are now translated into 23,24 and 25 respectively so the code can now cope with those inputs and they should appear in linear order in the Manhattan x axis The floppydisk link (download) problem remains to be fixed as it's more complex. We only update the main server every few weeks so the changes I just made will take a while to appear there. Meanwhile, please try the revised tool by making yourself an account and then uploading your data to http://test.g2.bx.psu.edu/ - please let me know how you get on? Of course, the (?FDR adjusted) p values in your data are a challenge - typically, the Manhattan and QQ plots are run with unadjusted p values which are expected to be distributed uniformly 0-1 under the null... FYI, when I use the grouping tool on your data, I see that chromosomes 20,21 and 22 are absent from the file - is this expected? Group with a count gives: 1 22910 10 9990 11 12764 12 13413 13 13068 14 11319 15 11175 16 8483 17 12017 18 10073 19 6927 2 18332 3 16173 4 15401 5 15259 6 15487 7 15217 8 12485 9 14560 X 3951 Thanks again for helping us improve Galaxy... On Mon, Jul 18, 2011 at 10:52 PM, Alison Harrill <AHarrill(a)thehamner.org> wrote: > The first time I uploaded the file, I loaded in the -logP values > instead. After that, I changed the file to raw p values and reuploaded. > Do you think it would help to upload the file again, but change the > filename? I just wonder if that original file didn't get cleared out. > > Alison > > -----Original Message----- > From: Ross [mailto:ross.lazarus@gmail.com] > Sent: Monday, July 18, 2011 8:51 AM > To: Alison Harrill > Subject: Re: [galaxy-bugs] Error message when trying to dowload > Manhattan plot using the web version > > ok - thanks. These are not raw pvalues and your data is definitely > causing the Manhattan plot to fail. > > This will take me a little time to figure out - I'll get back to you > in a day or two - thanks again for reporting this problem > > On Mon, Jul 18, 2011 at 10:16 PM, Alison Harrill > <AHarrill(a)thehamner.org> wrote: >> Good morning, >> >> Sorry for taking a while to get back to you (I was out of town for the >> weekend). I set permissions to share the dataset with you. Please let > me >> know if you see an obvious workaround. >> >> Thank you, >> Alison >> >> >> Alison Harrill, Ph.D. >> Research Investigator >> Institute for Drug Safety Sciences >> Hamner Institutes for Health Sciences >> phone: (919) 226-3142 >> fax: (919) 226-3150 >> >> >> >> -----Original Message----- >> From: Ross [mailto:ross.lazarus@gmail.com] >> Sent: Sunday, July 17, 2011 7:25 AM >> To: Alison Harrill >> Subject: Re: [galaxy-bugs] Error message when trying to dowload >> Manhattan plot using the web version >> >> Can you please share a history with me (use this email address) so I >> can take a look please? >> >> I'm talking about links on the results page where you (should!) see >> some graphs - when you click the history item display ('eye') icon - >> not the history item save ('floppy disk') icon >> >> On Sun, Jul 17, 2011 at 9:15 PM, Alison Harrill > <AHarrill(a)thehamner.org> >> wrote: >>> Hi, Ross, >>> >>> Thanks so much for kindly following up with me. I tried the > workaround >>> you suggested, but it appears to not be working for me. > Right-clicking >>> prompts me to save a file called display.htm and when I open the > file, >>> it just reads "Unable to remove temporary library download archive > and >>> directory". >>> >>> Best wishes, >>> Alison >>> >>> -----Original Message----- >>> From: Ross [mailto:ross.lazarus@gmail.com] >>> Sent: Sunday, July 17, 2011 4:09 AM >>> To: Alison Harrill >>> Subject: Re: [galaxy-bugs] Error message when trying to dowload >>> Manhattan plot using the web version >>> >>> Hi, Alison - I wanted to be sure you are able to get your images from >>> the tool - the proper fix is going to take a while so I hope those >>> links are working for you? >>> >>> cheers... >>> >>> >>> On Sat, Jul 16, 2011 at 9:39 AM, Ross <ross.lazarus(a)gmail.com> wrote: >>>> Hi, Alison, >>>> >>>> Thanks for reporting this error - I just confirmed that you have >> found >>>> a bug - the download function is currently not working correctly for >>>> composite datatypes in the history. >>>> >>>> Fortunately, the images themselves have download links under each > one >>>> and can be saved - eg on a PC or linux workstation, while hovering >>>> over the '(Click to download image Armitagep_manhattan.png)' press >> the >>>> right side mouse button and choose "save link as.." - you will be >> able >>>> to choose a location to save the image file on your local >> workstation. >>>> I hope this "work around" allows you to get on with your work - > sorry >>>> for the inconvenience and thanks again for finding and reporting > this >>>> error - we will fix it as soon as we can. >>>> >>>> >>>> >>>> On Sat, Jul 16, 2011 at 6:46 AM, Alison Harrill >>> <AHarrill(a)thehamner.org> wrote: >>>>> Greetings, >>>>> >>>>> I have a question about generating a Manhattan/QQ plot using the > web >>> tool. I >>>>> was able to successfully run the plot (the result box is green), > but >>> when I >>>>> try to download the results, I get a message that says "Unable to >>> remove >>>>> temporary library download archive and directory." Is there another >>> way to >>>>> retrieve the data? >>>>> >>>>> Thank you, >>>>> >

2 1

get data interface for public Galaxy instance
by Scott Emrich 06 Sep '11

06 Sep '11

Dear team, There are multiple 3rd party sources to import arthropod vector data (e.g. BioMart) from our NIH Bioinformatics Resource Center (BRC) VectorBase into Galaxy, but some are outdated and some of our users would like a direct source. Given this increase of Galaxy in our vector biology community, what is the policy for adding a new server to "Get Data" and is there information on formatting our results appropriately (i.e., using a REST interface)? Our NIH BRC server is www.vectorbase.org. I noticed our sister BRC EuPathDB has made their data available via http://main.g2.bx.psu.edu, and we have similar developer resources to make our server compatible; however, I couldn't find this information online besides addressing this list. Thanks in advance, and best regards, - S --- Scott Emrich, PhD Scientific Manager, VectorBase Assistant Professor, Computer Science and Engineering University of Notre Dame

3 4

Galaxy megablastdb indexer script proposal
by Jacob Israel Cervantes Luevano (LANGEBIO) 06 Sep '11

06 Sep '11

Hi Galaxy Dev, Nice to meet you. My name is Jacob and I am from México , Actually I'm working in LANGEBIO (Laboratorio Nacional de Genomica para la Biodiversidad) at Bioinformatics. I would like to contribute with galaxy dev team, I developed shell script to set up reference genomes by megablast. the linux shell script that I prepared uses your current prepdb.py script. the mini pipeline flow-work: 1. read input values: * fasta filename (nucleotide) * blast format: blast format: F->nucleotide, T->protein * database output name * fasta sequence IDs format (ncbi-gbk , other) where: ncbi-gbk means something like ">gi|gi-number|gb|accesión|locus" other means something like ">my own IDs" 2. if fasta sequence IDs are like ncbi format(ncbi-gk) then execute prepdb.py python script and generate new fasta file then go step 4 example: >gi|gi-number|gb|accesión|locus 3. if fasta sequence IDs has diferent (other) identifiers set to galaxy format first then execute prepdb.py python script then go step 4 >my own identifier 4. to format database using formatdb blast utility 5. setting up Megablast Galaxy Loc File 5. delete temp files 6. done Usage example: query help: ./set_galaxy_blastdb.sh -help format blast nucleotide dabatabase yeast, set galaxy loc file ./set_galaxy_blastdb.sh -fasta /media/xlinux/genomes/galaxy/palomero/megablast/yeast_nt.fasta -p F -n yeastdb -fastadesc ncbi-gk when script finished we have following results: $ls /media/xlinux/genomes/galaxy/palomero/megablast/ blastdb.loc new.yeast_nt.fasta yeastdb.nhr yeastdb.nin yeastdb.nsq yeast_nt.fasta How can I contribute with this shell script ? Thanks in advance -- Jacob http://www.langebio.cinvestav.mx/bioinformatica/jacob -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

2 1

TEMP variable in cluster install
by Shantanu Pavgi 06 Sep '11

06 Sep '11

We have configured galaxy to work with our SGE cluster using drmaa job runner interface. We are using 'unified method' for this install and both TEMP environment variable and new_file_path in universe_wsgi.ini file have been configured correctly. However, we are seeing some errors where local /tmp space on compute nodes is being referenced by the galaxy tools. Specifically we saw it mentioned in error messages from following tools: * bwa_wrapper and upload tools: 'No space left on device: /tmp...' * sam_to_bam tool: 'Error sorting alignments from /tmp/filename..' Shouldn't it be referencing TEMP environment variable or new_file_path configuration value? Is it getting overridden by TMP or TMPDIR variables in python wrapper code? Has anyone else experienced similar issue? -- Thanks, Shantanu.

5 6