April 2014 - galaxy-dev - lists.galaxyproject.org

Unable to remove old datasets
by Sanka, Ravi 07 Apr '14

07 Apr '14

Greetings, Despite being an admin, I am unable to remove old datasets from our Galaxy instance. I am following the procedure detailed in: https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%2… 1. delete_userless_histories.sh 2. purge_histories.sh 3. purge_libraries.sh 4. purge_folders.sh 5. delete_datasets.sh --> desired to remove datasets before their outer container had been deleted 6. purge_datasets.sh None of the scripts have been changed. They all call cleanup_datasets.py with –d at 10 and –r enabled. But it does not appear to have any effect. All datasets (both those older than 10 days and those more recent) in <galaxy root>/database/files are still present, despite the –r setting in each script. Is there some parameter or such that needs to be set in the universe config that will allow this process to work? ---------------------------------------------- Ravi Sanka ICS – Sr. Bioinformatics Engineer J. Craig Venter Institute 301-795-7743 ----------------------------------------------

4 15

Fwd: [galaxy-user] known name outputs shown in /database/files but not in galaxy windows
by ANGELICA GOMEZ ROIG 07 Apr '14

07 Apr '14

Hi Jen, I tried what you explain, but still doens't work. I did create a new datatype, gem. And I modified my code as you suggested. The tool i'm using always generates by default two archives . gem and .log. Those are the ones that I can't show in the galaxy interface, the output I get is empty, the dataset_54.dat and the dataset_55.dat. I can't handle to show me the .dat.gem and .dat.log ones. dataset_54.dat (my first declared output that it's empty) dataset_54.dat.gem (the output that throws the tool) dataset_54.dat.log (the other output file that throws the tool) dataset_55.dat (my second declared output that it's empty) Here it is the gem_indexer.xml <tool id="GEM Index" name="create GEM Indexer"> <command interpreter="python"> gem_indexer.py $input $output1 $output2</command> <inputs> <param name="input" type="data" format="fasta" label="Fichero de entrada(formato .fas)" /> </inputs> <stdio> <exit_code range="1:" level="fatal" description="Bad input dataset" /> </stdio> <outputs> <data format_source="gem" name="output1" label="${tool.name} on ${on_string}: salida" from_work_dir="gem_tools/salida.gem" /> <data format_source="tab" name="output2" label="${tool.name} on ${on_string}: salida" from_work_dir="gem_tools/salida.log" /> </outputs> </tool> And the gem_indexer.py import sys,os def __main__(): # Command to run GEM tool. Here the Gem indexer is getting invoked along with arguments to be passed. cmd="gem-indexer -i "+sys.argv[1]+" -o salida" os.system(cmd) if __name__ == "__main__" : __main__() 2014-04-04 6:05 GMT+02:00 Jennifer Jackson <jen(a)bx.psu.edu>: Reply posted: > > http://dev.list.galaxyproject.org/known-name-outputs-shown-in-database-file… > > Angelica, > Next time it would be great to post to only a single list. The > galaxy-dev(a)bx.psu.edu list is best for this type of question. > Take care, > Jen > Galaxy team > > > On 4/3/14 12:13 AM, ANGELICA GOMEZ ROIG wrote: > > I have made a wrapper that has a tool that giving it an input file, > gives you two output files with known extension: .gem and .log > I call the tool with a python file, and I pass it the input file and the > two output file names for galaxy to show them, but when I look into de > database/files/000/ i get four files: > > dataset_54.dat (my first declared output that it's empty) > dataset_54.dat.gem (the output that throws the tool) > dataset_54.dat.log (the other output file that throws the tool) > dataset_55.dat (my second declared output that it's empty) > > So,what I'm getting in the galaxy interface is _54.dat and _55.dat, both > empty, instead of _54.dat.gem and _54.dat.log that are the good ones. How > can I fix these? > Thanks > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > > > -- > Jennifer Hillman-Jacksonhttp://galaxyproject.org > >

3 3

The PBS job did not stop after the running dataset was deleted
by Jian-Long Huang 07 Apr '14

07 Apr '14

Hi, I used the torque/pbs system with Galaxy In the newer stable version (2014.2.10), the PBS job did not stop when I deleted the running dataset. It can be deleted with such API: http://127.0.0.1:8080/datasets/f2db41e1fa331b3e/delete_async Seems the js did not work? Regards, Jian-Long

2 1

Stopping Upload jobs
by Ganote, Carrie L 07 Apr '14

07 Apr '14

Hi List, I've noticed that the mechanisms for catching job termination signals don't seem to apply to uploading jobs. I was having trouble with *any* jobs being terminated until I pulled Nate's changeset 1298d3f from Galaxy central, but it still doesn't work for uploads. It will remove them from the history, but if I restart Galaxy, they are put back into the history (whether or not there is anything actually happening). If the upload.py script is actively running and I delete the job, the history item comes back up during the next watch cycle. I've been experimenting with staging files as part of a job and I notice that if I set the job state to upload during the staging, the job never gets marked as deleted or deleted_new if I try to remove it from the history with the X button - so the logic I have to stop the upload is never run. Thanks for any insight, Carrie Ganote

2 1

Access a tool inside another tool
by Janaki Rama Rao Gollapudi 07 Apr '14

07 Apr '14

Hi, For testing I have installed custom tool A and its repository dependency custom tool B. Actually as per the implementation there is no dependency among these two custom tools. But in general if we want to access a tool inside another tool implementation how should I do that. Can you please share resources with me if any. Thanks, JanakiRam

1 0

This tool was disabled before the job completed. Please contact your Galaxy administrator.
by Ravi Alla 04 Apr '14

04 Apr '14

Hi devs, I deleted a tool called upload_local_file and tried to restart galaxy server but I keep getting the following error. Any idea how to solve this? File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 39, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/app.py", line 130, in __init__ self.job_manager = manager.JobManager( self ) File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/jobs/manager.py", line 37, in __init__ self.job_handler.start() File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/jobs/handler.py", line 35, in start self.job_queue.start() File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/jobs/handler.py", line 78, in start self.__check_jobs_at_startup() File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/jobs/handler.py", line 109, in __check_jobs_at_startup JobWrapper( job, self ).fail( 'This tool was disabled before the job completed. Please contact your Galaxy administrator.' ) File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/jobs/__init__.py", line 761, in fail self.app.object_store.update_from_file(dataset.dataset, create=True) File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 349, in update_from_file self.create(obj, **kwargs) File "/srv/www/galaxy/source/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 298, in create open(path, 'w').close() # Should be rb? IOError: [Errno 2] No such file or directory: '/global/scratch/galaxy/webapp/files/000/dataset_194.dat' Thanks Ravi

1 2

Galaxy packages for Bio-Linux - an update + Apache proxying
by Tim Booth 04 Apr '14

04 Apr '14

Hi All, As previously mentioned, I'm back working on packaging the Galaxy server as DEB packages for Bio-Linux (ie. Ubuntu 12.04 LTS) and ultimately pushing towards something that could be Debian compliant. There's a way to go in that regard, but I do now have an updated package for Bio-Linux in final testing and it also has a new trick: doing "apt-get install galaxy-server-apache-proxy" will set up just that with no further configuration needed. The galaxy server appears at http://localhost/galaxy and users log in with their regular system username and password. Uploads are enabled via regular SFTP so no special FTP server configuration is needed. It's a little hacky in parts but I'm generally pleased with the result. If anyone want to take a look I'd welcome comments. It's not in the main BL repo yet but can be found here: https://launchpad.net/~nebc/+archive/galaxy/+sourcepub/3711751/+listing-arc… Cheers, TIM -- Tim Booth <tbooth(a)ceh.ac.uk> NERC Environmental Bioinformatics Centre Centre for Ecology and Hydrology Maclean Bldg, Benson Lane Crowmarsh Gifford Wallingford, England OX10 8BB http://nebc.nerc.ac.uk +44 1491 69 2705

4 7

Depth of Coverage (local galaxy __ CloudMap)
by Wang, Xiaofei 04 Apr '14

04 Apr '14

Dear there, I got this error when I run CloudMap EMS Variant Density Mapping workflow for Depth of Coverage on the local Galaxy instance on Mac. In fact, it works fine for Depth of Coverage when I run another workflow (Unmapped Mutant workflow) locally. I really appreciate you guys for helping me to figure out so many problems! Thanks a lot! Best, Xiaofei Traceback (most recent call last): File "/Users/xiaofeiwang/softwares/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 152, in prepare_job job_wrapper.prepare() File "/Users/xiaofeiwang/softwares/galaxy-dist/lib/galaxy/jobs/__init__.py", line 652, in prepare if job.user is None and job.galaxy_session is None: File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/attributes.py", line 168, in __get__ return self.impl.get(instance_state(instance),dict_) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/attributes.py", line 453, in get value = self.callable_(state, passive) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/strategies.py", line 508, in _load_for_state return self._emit_lazyload(session, state, ident_key) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/strategies.py", line 552, in _emit_lazyload return q._load_on_ident(ident_key) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/query.py", line 2512, in _load_on_ident return q.one() File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/query.py", line 2184, in one ret = list(self) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/query.py", line 2227, in __iter__ return self._execute_and_instances(context) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/orm/query.py", line 2242, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "/Users/xiaofeiwang/softwares/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-macosx-10.6-intel-ucs2.egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) database is locked u'SELECT galaxy_user.id AS galaxy_user_id, galaxy_user.create_time AS galaxy_user_create_time, galaxy_user.update_time AS galaxy_user_update_time, galaxy_user.email AS galaxy_user_email, galaxy_user.username AS galaxy_user_username, galaxy_user.password AS galaxy_user_password, galaxy_user.external AS galaxy_user_external, galaxy_user.form_values_id AS galaxy_user_form_values_id, galaxy_user.deleted AS galaxy_user_deleted, galaxy_user.purged AS galaxy_user_purged, galaxy_user.disk_usage AS galaxy_user_disk_usage, galaxy_user.active AS galaxy_user_active, galaxy_user.activation_token AS galaxy_user_activation_token \nFROM galaxy_user \nWHERE galaxy_user.id = ?' (1,)

2 1

version of SAM-to-BAM (CloudMap)
by Wang, Xiaofei 04 Apr '14

04 Apr '14

Dear there, When I run CloudMap EMS Variant Density Mapping workflow (takes VCF of heterozygous and homozygous variants to subtract) (imported from uploaded file) on local Galaxy instance, I got this [X] , when I wanted to edit the workflow. Also, I got this The following tools are beinge executed with a different version from what was available when this workflow was last saved because the previous version is no longer available for use on this galaxy instance. To upgrade your workflow and dismiss this message simply edit the workflow and re-save it to update the stored tool version. * toolshed.g2.bx.psu.edu/repos/devteam/sam_to_bam/sam_to_bam/1.1.2: using version '1.1.2' instead of version '1.1.3' indicated in this workflow. * ,when I wanted to run the workflow. I know it is the problem with version of SAM-to-BAM. But, when I searched the Tool sheds > Search and browse tool sheds, there is only version 1.1.4 for SAM-to-BAM. When I tried to edit the workflow from Workflow > ...> Edit, I did not find where could I change the version of SAM-to-BAM. When I clicked on SAM-to-BAM on the Workflow Canvas, it showed the version is 1.1.3 for it. Could you tell me which version do I need exactly, and how to figure out this? Thanks a lot! Best, Xiaofei

2 2

Download workflow from main Galaxy
by Wang, Xiaofei 04 Apr '14

04 Apr '14

Dear there, I tried to download workflow and import it into another my local Galaxy instance. But, when I downloaded and exported it from main Galaxy by "Download to File", it showed as below. Not Found The resource could not be found. No route for /u/w/imported-cloudmap-ems-variant-density-mapping-workflow-takes-vcf-of-heterozygous-and-homozygous-variants-to-subtract/json-download Then, I tried "URL for Importing to Another Galaxy Use this URL to import the workflow directly into another Galaxy server: https://usegalaxy.org/u//w/imported-cloudmap-ems-variant-density-mapping-wo… (Copy this URL into the box titled 'Workflow URL' in the Import Workflow page.)" , it showed Failed to open URL: https://usegalaxy.org/u//w/imported-cloudmap-ems-variant-density-mapping-wo… Exception: HTTP Error 404: Not Found What is the reason for this? Many thanks for helping me! Best, Xiaofei

2 2