July 2012 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Sample tracking data transfer hangs "in queue" forever
by TerAvest, Emily 27 Jul '12

27 Jul '12

Hi Leandro, I am also experiencing the same problem with the latest version of galaxy. I just attempted to connect our sequencer for the first time yesterday. I do not have an older version of galaxy to test and compare to see if it works in earlier versions. I am able to transfer data from the sequence to the import directory, however it is not moved to the data library. My data_transfer.log also has the same error. "The server could not comply with the request since it is either malformed or otherwise incorrect." Does anyone have a solution for this? Thanks Emily ------------------------------ Message: 17 Date: Wed, 28 Mar 2012 13:15:47 +0200 From: Leandro Hermida <softdev(a)leandrohermida.com> To: Luobin Yang <yangluob(a)isu.edu> Cc: Galaxy Dev <galaxy-dev(a)bx.psu.edu> Subject: Re: [galaxy-dev] Sample tracking data transfer hangs "in queue" forever Message-ID: <CAOhzmPJ67udRQBhfChgyxNrXsm3QcY1c7jna1kR0aJBNKEKJxA(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Dear Galaxy Dev, Ok I have gotten further found out what was going on, in my data_transfer.log I was getting HTTP 404 Not Authorized when the data transfer was trying to access Galaxy API URLs http://galaxyserver/api/... This is because we are using external user authentication as documented in http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy and this puts all of Galaxy behind this authentication. Since the Galaxy API uses API keys to essentially authenticate you have to change you <Location /> ... </Location> container to not match URLs starting with /api. To do this you have to change it to <LocationMatch "^/(?!api)"> .... </LocationMatch>. best, leandro On Mon, Mar 26, 2012 at 6:10 PM, Luobin Yang <yangluob(a)isu.edu> wrote: > I've got the same problem and when I looked at the data_transfer.log, I saw > the following message: > > 2012-03-01 15:12:27,338 - datatx_13870 - > (u'9c17d84742cd2acb63d88b5bd41d968f', > u'http://xxx.xxx.xxx.xxx/api/samples/2d9035b3fc152403', > {'sample_dataset_ids': ['a799d38679e985db', '33b43b4e7093c91f'], > 'error_msg': '', 'update_type': 'sample_dataset_transfer_status', > 'new_status': 'Adding to data library'}) > 2012-03-01 15:12:27,342 - datatx_13870 - Error. <!DOCTYPE HTML PUBLIC > "-//IETF//DTD HTML 2.0//EN"> > <html><head> > <title>405 Method Not Allowed</title> > </head><body> > <h1>Method Not Allowed</h1> > <p>The requested method PUT is not allowed for the URL > /api/samples/2d9035b3fc152403.</p> > <hr> > > Using Galaxy instead of Apache as the web server changed the sample status > from "in queue" to complete but didn't add the downloaded files to the data > library. > > Luobin > > On Mon, Mar 26, 2012 at 9:54 AM, Leandro Hermida > <softdev(a)leandrohermida.com> wrote: >> >> Dear Galaxy Dev, >> >> I've set up the Galaxy sample tracking system data transfer >> functionality exactly as specified here >> https://main.g2.bx.psu.edu/u/rkchak/p/data-transfer, but when I >> attempt to transfer datasets it puts them into the queue and then >> never seems to transfer anything. >> >> The galaxy_listener.log shows: >> >> 2012-03-26 17:16:24,515 - GalaxyAMQP - GALAXY LISTENER PID: 8738 - >> {'config_file': 'universe_wsgi.ini', 'http_server_section': >> 'server:main'} >> 2012-03-26 17:16:24,518 - GalaxyAMQP - {'exchange': 'galaxy_exchange', >> 'rabbitmqctl_path': >> >> '/gpfshpc/software/pmrdcdb/galaxy/rabbitmq/rabbitmq_server-2.8.1/sbin/rabbitmqctl', >> 'userid': 'galaxy_dev', 'routing_key': 'bar_code_scanner', 'queue': >> 'galaxy_queue', 'host': '127.0.0.1', 'virtual_host': >> 'galaxy_dev_messaging_engine', 'password': 'mypassword', 'port': >> '5672'} >> 2012-03-26 17:16:24,565 - GalaxyAMQP - Connected to rabbitmq server - >> 127.0.0.1:5672 >> 2012-03-26 17:31:23,393 - GalaxyAMQP - MESSAGE RECVD: data_transfer >> 2012-03-26 17:31:23,393 - GalaxyAMQP - DATA TRANSFER >> 2012-03-26 17:31:23,401 - GalaxyAMQP - Started process (25406): python >> >> "/gpfshpc/software/pmrdcdb/galaxy/galaxy_dist_dev/scripts/galaxy_messaging/server/data_transfer.py" >> " <data_transfer> >> <galaxy_host>rd-galaxydev.app.pmi</galaxy_host> >> <api_key>69bbd1a9b18eda2a888e9963820d8ef0</api_key> >> <data_host>chp723</data_host> >> <data_user>galaxy</data_user> >> <data_password>mypassword</data_password> >> <request_id>12</request_id> >> <sample_id>12</sample_id> >> <library_id>15</library_id> >> <folder_id>52</folder_id> ? ? ? ? ? ? ? ? ? ?<dataset> >> ?<dataset_id>10</dataset_id> >> <name>ttt_2.fastq.gz</name> >> <file>/gpfshpc/data/pmrdcdb/RNA-Seq/datasets/ttt-2.fastq.gz</file> >> ? ? ? ? ?</dataset><dataset> >> <dataset_id>9</dataset_id> >> <name>ttt_1.fastq.gz</name> >> <file>/gpfshpc/data/pmrdcdb/RNA-Seq/datasets/ttt-1.fastq.gz</file> >> ? ? ? ? ?</dataset> ? ? ? ? ? ? ? ?</data_transfer>" >> "universe_wsgi.ini" >> >> The FASTQ files are for testing and aren't very big (7.6 MB), what >> could be going wrong? >> >> regards, >> Leandro >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. ?To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> >> ?http://lists.bx.psu.edu/ > > ************ This email message and any electronic files transmitted with it relating to the official business of Codexis, Inc. and its affiliates are proprietary, strictly confidential, and may be legally privileged. This email message is intended solely for the use of the individual(s) to whom it is addressed. If you believe you have received this email in error, please delete this email from your system and notify the sender immediately. If you are not the intended recipient, you should not disclose, distribute, use or copy this email or take any action in reliance on its contents.

4 9

Announcement: Galaxy Reference Data
by Nate Coraor 27 Jul '12

27 Jul '12

Hi all, A frequent request we've had is to make the locally cached reference data used by the public Galaxy server (Main) available for download to local installations. To facilitate this, we've created an rsync server at datacache.g2.bx.psu.edu that serves the exact reference builds and associated indexes that we use on Galaxy Main. Fetching and installing such data is currently a manual process, so Dave Bouvier has been working on an administrative UI in Galaxy for automatically fetching references and building indexes. Keep your eyes on the development news briefs for an announcement once this feature is ready. --nate

1 0

for question
by huafang sheng 27 Jul '12

27 Jul '12

Hi, I am a master student in Southern Medical University of China. Thank you so much for LEFse development,which is a very important tool in microbial community data analysis. And I have try it on line,just found it failled for these two datasets. Woud you help me to check what is happend? Thank you for your appreciation and looking forward to hear from you soon. Best Wishes, Yours sincerely, Ms. Hua-Fang Sheng

1 0

Java applications in the toolshed
by Jim Johnson 26 Jul '12

26 Jul '12

Dan and Greg, SnpEff is an example of a java based tool in the tooshed that would be good to be able to deploy without any editing by the installer. Pablo has hardcoded the path to the jar and config files in the tool_config: /Users/pablocingolani/snpEff/ With directions for the installer to change that path. Would it be possible to supply the install path of a tool to the tool's cheetah command/config templating in a similar way that $__tool_data_path__ is provided? $__tool_path__ If a tool had dependencies could those be provided as a dict? $__tool_dependencies__['samtools'] Then snpEff command could be: <command>java -Xmx6G -jar $__tool_path__/snpEff.jar eff -c $__tool_path__/snpEff.config -i $inputFormat -o $outputFormat -upDownStreamLen $udLength $filterIn $filterHomHet -no $filterOut -stats $statsFile $genomeVersion $input > $output </command> This could allow for tool versioning and be used for for the common java packaging technique of an executable jar with META_DATA that references dependencies in a relative dir, usually ./lib/ Thanks, JJ Browse snpeff revision c052639fa666 (repository tip) Clone this repository:hg clonehttp://jjohnson@toolshed.g2.bx.psu.edu/repos/pcingola/snpeff Contents: snpeff <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff_2_1a/ <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff_2_1a/ <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> galaxy/ <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff_download.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpSift_annotate.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpSift_caseControl.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpSift_filter.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpSift_int.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> tool_conf.xml <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> scripts/ <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff.config <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> snpEff.jar <http://toolshed.g2.bx.psu.edu/repository/browse_repository?webapp=community…> <tool id="snpEff" name="SnpEff" version="1.0"> <description>Variant efefct and annotation</description>  <command>java -Xmx6G -jar /Users/pablocingolani/snpEff/snpEff.jar eff -c /Users/pablocingolani/snpEff/snpEff.config -i $inputFormat -o $outputFormat -upDownStreamLen $udLength $filterIn $filterHomHet -no $filterOut -stats $statsFile $genomeVersion $input > $output </command>

1 0

index file
by Unknown 26 Jul '12

26 Jul '12

Hi galaxy team, How can we make an index file for a presorted bam file using galaxy? Thanks, Deepthi

1 0

annoying bug when deleting datasets
by David Hoover 26 Jul '12

26 Jul '12

I recently switched from galaxy-dist to galaxy-central as the source of my local install (version 7427:cb9272041d95). When I delete a dataset from a history, all the delete icons disappear from the datasets. They come back if I refresh the page. Has anyone else noticed this? David Hoover Helix Systems Staff

3 2

help
by romain desprat 26 Jul '12

26 Jul '12

I am trying to work on 23 giga text file. My laptop being not powerful enough I have split it in 10 files and made some modification on them. Now I have made (linux and window) a double boot on my laptop and install galaxy ( on the linux partition). Unfortunately, galaxy doesn t have enough memory to work with this file. My question is : is it because I did not allocate enough memory to galaxy/linux partition when I did the double boot ( 5 giga f I recall right ) or is it because galaxy over 2 giga anyway will not be able to handle the situation. Thanks any advices are welcome, Romain romain.desprat(a)einstein.yu.edu

2 1

Help to run a C program
by Rodrigo Faccioli 26 Jul '12

26 Jul '12

Hi, I'm newbie in Galaxy. In fact, this is my first post. I'd like to create a web-interface to run a C program. It is necessary a config file which is used as input to this program. Therefore, the values of parameters are informed by user through this web-interface. Its output is some files. I searched in Galaxy wiki some examples or tutorials. However, I didn't find anything that helped me. I'm confused in what I can do. Could you help me showing where I can find specific materials? Best regards, -- Rodrigo Antonio Faccioli Ph.D Student in Electrical Engineering University of Sao Paulo - USP Engineering School of Sao Carlos - EESC Department of Electrical Engineering - SEL Intelligent System in Structural Bioinformatics http://laips.sel.eesc.usp.br Phone: 55 (16) 3373-8739 Curriculum Lattes - http://lattes.cnpq.br/1025157978990218 Public Profile - http://br.linkedin.com/pub/rodrigo-faccioli/7/589/a5 Personal Blogg - http://rodrigofaccioli.blogspot.com/

3 3

HMMER wrappers
by Peter Cock 26 Jul '12

26 Jul '12

Hello all, Has anyone done a wrapper for HMMER2 or HMMER3? http://hmmer.janelia.org/software I can't find anything in the Tool Shed http://toolshed.g2.bx.psu.edu/ so we are likely to look into doing this ourselves - starting with hmmerscan. Thanks, Peter

4 16

having a directory and all contents retained as the 'output'
by Brian Haas 26 Jul '12

26 Jul '12

Greetings, I have a process that is going to generate dozens of output files, each of which I want to retain as a data repository that will be leveraged by other downstream tools. I could just point the downstream tool at the directory, and it'll know what to look for, rather than to have to list each output file as a separate parameter. Is it possible to do this in Galaxy? And is there a best practice (or example) for it? thanks! -b -- -- Brian J. Haas Manager, Genome Annotation and Analysis, Research and Development The Broad Institute http://broad.mit.edu/~bhaas

2 2