September 2010 - galaxy-dev - lists.galaxyproject.org

Add a new data type
by Kaiyu Shen 21 Sep '10

21 Sep '10

Hi, folks: I am a newbie to galaxy and has no prior experience in python. I want to add a data format (tar file) to galaxy, could you please give me some detailed suggestions on how to implement? I have tried to upload a zip file, however, the galaxy only recognizes the first file in a zip package which does not suit my needs. Thank you very much. Kaiyu

2 1

Issue with purging deleted datasets
by Johannes Eichner 17 Sep '10

17 Sep '10

Dear Galaxy dev team, as my galaxy database folder ("galaxy-dev/database/files/000/") contains too many large files, which are not needed any more, I just tried to use the cleanup scripts to delete all stuff older than 10 days. I executed the corresponding shell scripts in the recommended order: delete_userless_histories.sh purge_histories.sh purge_libraries.sh purge_folders.sh purge_datasets.sh The problem is that "purge_datasets.sh" did not remove any file from the disk (although I used the "-r" option). I also tried to run "delete_datasets.sh" in advance. According to the respective log-file ("delete_datasets.log") some dataset instances were deleted: Examined 329 datasets, marked 329 as deleted and purged 329 dataset instances Total elapsed time: 101.456291914 However, if I try to run "purge_datasets.sh" subsequently, nothing happens: Purged 0 datasets Freed disk space: 0 Elapsed time: 0.834518909454 Could you please help me solving this issue? Kind regards, Johannes -- Dipl.-Inf. (Bioinf.) Johannes Eichner Univ. Tuebingen, WSI-RA, Sand 1, D-72076 Tuebingen, Germany Phone: (+49/0) 7071 / 29 70436 Fax: (+49/0) 7071 / 29 5091

2 1

reload display_application
by Davide Cittaro 17 Sep '10

17 Sep '10

Hi all, Will it be possible (in the future) to reload also display applications in the "reload tool configuration" menu under admin pages? d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

1 0

external display application styles
by Davide Cittaro 17 Sep '10

17 Sep '10

Hi all, I need the users to see the direct link for BAM files so that they can copy/paste it in their desktop visualizer (either IGV, IGB, whatever...). I could write the application xml definition and it works properly: <display id="igv_bam" version="1.0.0" name="view link for"> <link id="igv" name="IGV"> <url>$track.url</url> <param type="data" name="bam_file" url="galaxy_${DATASET_HASH}.bam" strip_https="True" /> <param type="data" name="bai_file" url="galaxy_${DATASET_HASH}.bam.bai" metadata="bam_index" strip_https="True" /> <param type="template" name="track" viewable="True" strip_https="True"> Select "File -> Load from URL..." in IGV, then copy the following link: ${bam_file.url} </param> </link> </display> I just would like now to have this page using galaxy css so that I can write nicer instruction (with colors and images) for the dummiest end-user. I see that the resulting page source encloses everything between <pre/> tags: <html> <body> <pre style="word-wrap: break-word; white-space: pre-wrap;"> " HERE IS THE CONTENT OF THE APPLICATION " </pre> </body> </html> How can I override this behavior? I mean, I can edit the template (which is the template?) but that would affect every other display application... Thanks d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

1 0

Workflow Fails with persistence issue?
by Jacob Biesinger 17 Sep '10

17 Sep '10

Hi! I have a pretty simple NGS workflow I've set up. I'm running Galaxy's MACS wrapper pushing the output to MEME, a motif finder, in hopes of finding overrepresented TF motifs. Running MACS and then MEME when the job is complete works just fine, but when I extract the workflow or create my own workflow and link the pieces together, I get a strange SQLAlchemy persistence error when MACS is run. MACS successfully completes and I can see the output just fine, but the jobs that follow in the workflow stay grayed out forever. I can run other jobs just fine, so I don't think the dispatcher died. Here is the error message: 69.234.135.61 - - [16/Sep/2010:22:08:44 -0700] "GET /workflow/list_for_run HTTP/1.1" 200 - "http://iron.ics.uci.edu :8080/root/tool_menu" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.4) Gecko/2008102920 Firefox/3 .0.4" 169.234.135.61 - - [16/Sep/2010:22:08:46 -0700] "GET /workflow/run?id=1cd8e2f6b131e891 HTTP/1.1" 200 - "http://iron. ics.uci.edu:8080/workflow/list_for_run" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.4) Gecko/20 08102920 Firefox/3.0.4" galaxy.jobs DEBUG 2010-09-16 22:09:09,483 dispatching job 33 to local runner galaxy.jobs INFO 2010-09-16 22:09:09,725 job 33 dispatched galaxy.jobs.runners.local DEBUG 2010-09-16 22:09:10,159 executing: python /home/galaxy/galaxy_dist/tools/peak_callin g/macs_wrapper.py /home/galaxy/galaxy_dist/database/job_working_directory/33/tmpSvTS8S /home/galaxy/galaxy_dist/data base/files/000/dataset_84.dat /home/galaxy/galaxy_dist/database/files/000/dataset_87.dat /home/galaxy/galaxy_dist/da tabase/job_working_directory/33/dataset_87_files galaxy.jobs ERROR 2010-09-16 22:09:11,379 failure running job 34 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 186, in __monitor_step self.sa_session.refresh( job ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 926, in refresh self._validate_persistent(state) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1236, in _validate_persistent mapperutil.state_str(state)) InvalidRequestError: Instance '<Job at 0x5e04f90>' is not persistent within this Session galaxy.jobs ERROR 2010-09-16 22:09:11,925 failure running job 35 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 186, in __monitor_step self.sa_session.refresh( job ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 926, in refresh self._validate_persistent(state) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1236, in _validate_persistent mapperutil.state_str(state)) InvalidRequestError: Instance '<Job at 0x5e57d10>' is not persistent within this Session galaxy.jobs ERROR 2010-09-16 22:09:12,666 failure running job 36 Traceback (most recent call last): File "/home/galaxy/galaxy_dist/lib/galaxy/jobs/__init__.py", line 186, in __monitor_step self.sa_session.refresh( job ) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 926, in refresh self._validate_persistent(state) File "/home/galaxy/galaxy_dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1236, in _validate_persistent mapperutil.state_str(state)) InvalidRequestError: Instance '<Job at 0x5e4b350>' is not persistent within this Session 169.234.135.61 - - [16/Sep/2010:22:09:08 -0700] "POST /workflow/run?id=1cd8e2f6b131e891 HTTP/1.1" 200 - "http://iron .ics.uci.edu:8080/workflow/run?id=1cd8e2f6b131e891" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0 .4) Gecko/2008102920 Firefox/3.0.4" ... a few job failures later galaxy.jobs.runners.local DEBUG 2010-09-16 22:09:15,495 execution finished: python /home/galaxy/galaxy_dist/tools/pe ak_calling/macs_wrapper.py /home/galaxy/galaxy_dist/database/job_working_directory/33/tmpSvTS8S /home/galaxy/galaxy_ dist/database/files/000/dataset_84.dat /home/galaxy/galaxy_dist/database/files/000/dataset_87.dat /home/galaxy/galax y_dist/database/job_working_directory/33/dataset_87_files galaxy.jobs DEBUG 2010-09-16 22:09:16,192 job 33 ended It seems that the subsequent jobs (34, 35, etc) aren't waiting for MACS to complete. FYI: Running fairly recent checkout of galaxy_dist (not more than a month old). Using Postgres 8.4.4 currently, but was on the default sqlite3 table schema, where I saw the same issue. Ubuntu 10.04, with BASH replacing the default DASH. Any thoughts or places I could look to figure out what's going on? -- Jake Biesinger Graduate Student Xie Lab, UC Irvine (949) 231-7587

1 1

Galaxy Roadmap?
by lentaing＠jimmy.harvard.edu 16 Sep '10

16 Sep '10

Hi, In addition to the "Development Cycle News Briefs" and "Features" page, it might be useful to have a "Galaxy Roadmap" page which will tell the developer community what the galaxy team is currently or will work on in the upcoming months or quarter. This will help those who deploy local galaxy instances to better plan when to update/merge their codebase. Len Cistrome Development Team

1 0

new display application (IGV)
by Davide Cittaro 16 Sep '10

16 Sep '10

Hi all, Before reinventing the wheel I would like to know if anybody here has tried to write a display_application to visualize a BAM (or a bed or a bedgraph...) into a local IGV. d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

1 0

KeyError when sharing workflow
by Branden Timm 15 Sep '10

15 Sep '10

Hello, I have a workflow that I've tried to share both with users and via link. Trying to use either of these methods to access the shared workflow results in a Server Error. paster.log reports: File '/home/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py', line 795 in visit_inputs callback( "", input, value[input.name] ) KeyError: 'referenceGenome' referenceGenome is a param from one of my tool's xml configuration file. Here is the relevant section: <param name="referenceGenome" type="select" label="Reference Genome"> <options from_file="soap_indices.loc"> <column name="name" index="1"/> <column name="value" index="2"/> </options> </param> soap_indices.loc only contains one line currently ecoliU00096 E. Coli U00096 /home/galaxy/galaxy_dist/soap_indices/ecoli_U00096.fasta.index I'm confused as to why I am getting this KeyError, but I suspect that perhaps I have set up my .loc file incorrectly? Any help would be greatly appreciated. Cheers -- Branden Timm Great Lakes Bioenergy Research Center btimm(a)glbrc.wisc.edu

2 4

transferring sample datasets stuck in 'In queue' status
by nsfox＠uwaterloo.ca 15 Sep '10

15 Sep '10

Hi, I set up the rabbitmq-server and created an exchange and queue with a routing key. My setup is allowing the dataset status to change to 'In queue' from 'Not started' without any error messages. However the dataset doesn't seem to actually be transferred. It is just left in the queue. Is there something else that I need to be doing? Any ideas to what the problem might be are appreciated. Thanks, Natalie

2 2

environmental variables (PATH)
by Erick Antezana 15 Sep '10

15 Sep '10

Hi, I was wondering whether there is a way to set (or more properly to update) the PATH environmental variable for my Galaxy instance? so that I could simply avoid using the absolute path of the scripts/tools that are defined in the XML(tool) files I created. thanks, Erick

3 4