January 2013 - galaxy-dev - lists.galaxyproject.org

Job output not returned from Cluster
by Sascha Kastens 17 Jan '13

17 Jan '13

Hi all, I have exactly the same problem as described here: http://gmod.827538.n3.nabble.com/Job-output-not-returned-from-cluster-td879… All the SGE stuff is administrated in our company and unfortunately they know nothing about Galaxy and so it?s my task to get it running. Everything works fine except the problem mentioned above. I?ve contacted Erick but haven?t received a response yet. If anybody can give me some hints where I can look at to solve the problem or at least what I can tell the admins, I would be really grateful. Thanks in advance. If you need further information let me know! Cheers, Sascha

4 8

Tool Error or Bug?
by Elizabeth Clare 17 Jan '13

17 Jan '13

Hi, I cannot determine if I have a tool error or a bug and I am getting errors when I try to report the problem so I am not sure where to send a request for help. I hope i have directed this to the correct place for assistance. I have used Galaxy's main web tools many times successfully so I am familiar with the formats etc. I have now installed Galaxy locally. I successfully uploaded a Fastq file, converted it's format etc. However I cannot get the Fasta manipulation tools to work. When I attempted to use Fasta Width formatter I got the message "dataset_2.dat.Z: No such file or directory /bin/sh: fasta_formatter: command not found" I can't interpret whether that is the file it can't find or the command... it sounds like both? I've had similar errors on the other tools. When I tried to use the bug report to request help I got the error "Mail is not configured for this galaxy instance" so I can't report the problem. I suspect there is something wrong with my local install (are tool installed separately?) but I'd appreciate any insight into the problem or the best place to direct my question. The exact details are as follows: -using new local install (Mac OS 10.8, Python newest version, Galaxy downloaded today) -don't know what an hg pull number is -bug occurred around Thursday at noon (UK time zone) -exact steps, get data to upload a fastq file, convert to fasta, attempt to use fasta width formatter -I've tried to re-upload the file -I've tried the main galaxy server "get data" but it's been stuck at "job waiting to run" for hours so perhaps traffic is heavy. Thanks Beth -- -------------------------------- Dr. Elizabeth L. Clare School of Biological and Chemical Sciences Queen Mary, University of London Mile End Road, London E1 4NS Room 1.02, G. E. Fogg Building e.clare(a)qmul.ac.uk +44 (0)207 882 ext. 5687 School of Biological Sciences University of Bristol, Bristol, UK. Phone: +44 117 954 5909 Website: http://seis.bris.ac.uk/~bzelc/ "If you say yes more often than you say no, you will do interesting things in your life"

1 0

Fwd: Number of SNPs
by Babanejad 16 Jan '13

16 Jan '13

Dear Galaxy Team We recently bagan to use galaxy in our lab. I uploaded the whole SNPs several times but the number of them showed difference in every time . Would you please do me a favor and let me know why the number of SNPs differ in every times and what is the reason of this? Best regards, M. Babanejad

1 0

running parallel workflow fails
by Charles Girardot 16 Jan '13

16 Jan '13

Hi, I have a workflow that I can successfully launch on a single FASTQ (the only input of the workflow). I am now trying to start the workflow on 8 different FASTQ files (after demultiplexing) but systematically get this error (local galaxy install): Thx for your help Charles URL: http://manni/galaxy/workflow/run Module paste.exceptions.errormiddleware:143 in __call__ >> app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ >> environ, self.app) Module paste.wsgilib:539 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:91 in __call__ >> return self.app( environ, start_response ) Module paste.httpexceptions:632 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ >> body = method( trans, **kwargs ) TypeError: run() takes at least 3 arguments (2 given) ###### below is a copy-paste of all variables (if useful): CONTENT_LENGTH '0' HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING 'gzip, deflate' HTTP_ACCEPT_LANGUAGE 'en-us' HTTP_AUTHORIZATION 'Basic Z2lyYXJkb3Q6Y3QwMmhlODg=' HTTP_CONNECTION 'Keep-Alive' HTTP_COOKIE 'galaxysession=7df64c2bf0628c5b57afd358adee6e7b2cd6161345bb6b47388e497cbd26f407876c2bed128d9226;wikidb_ngs_Token=1f38fe6b5eb5863596cad189f608835c; wikidb_ngs_UserName=Girardot; wikidb_ngs_UserID=15; wikidb_ngs__session=d9n3qq68n0nl6p6j9t96k5q6m3' HTTP_HOST 'manni' HTTP_REFERER 'http://manni/galaxy/workflow/run?id=52d6640a458dfabe' HTTP_REMOTE_USER 'girardot(a)embl.de' HTTP_USER_AGENT 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2' PATH_INFO '/workflow/run' REMOTE_ADDR '10.11.72.108' REQUEST_METHOD 'GET' SCRIPT_NAME '/galaxy' SERVER_NAME 'manni' SERVER_PORT '8080' SERVER_PROTOCOL 'HTTP/1.1' Configuration __file__ '/g/funcgen/galaxy/universe_wsgi.combined.ini' admin_users 'su@embl.de,sauer@embl.de,girardot@embl.de,galaxy@embl.de' allow_library_path_paste 'True' allow_user_dataset_purge 'True' allow_user_impersonation 'True' api_allow_run_as 'galaxy(a)embl.de' cleanup_job 'always' cluster_files_directory '/g/galaxy/galaxy_data/pbs' collect_outputs_from 'new_file_path,job_working_directory' cookie_path '/galaxy' database_connection 'postgres://galaxy:galaxy@manni/galaxy_db' database_engine_option_max_overflow '20' database_engine_option_pool_size '10' database_engine_option_server_side_cursors 'True' database_engine_option_strategy 'threadlocal' debug 'True' default_cluster_job_runner 'drmaa://-q gbcs_q -N galaxy_stdjob -l ncpus=1,mem=4gb/' drmaa_external_killjob_script 'scripts/drmaa_external_killer.py' drmaa_external_runjob_script 'scripts/drmaa_external_runner.py' enable_api 'True' enable_job_recovery 'True' enable_quotas 'True' enable_tracks 'False' environment_setup_file '/g/funcgen/galaxy/env_setup' error_email_to 'gbcs(a)embl.de' external_chown_script 'scripts/external_chown_script.py' file_path '/g/galaxy/galaxy_data/files' here '/g/funcgen/galaxy' id_secret 'all_your_base_are_belong_to_us' job_handlers 'runner0' job_manager 'runner0' job_working_directory '/g/galaxy/galaxy_data/job_working_directory' library_import_dir '/home/galaxy/' log_level 'DEBUG' logo_url 'http://manni/ngswiki/index.php/Galaxy' new_file_path '/g/galaxy/galaxy_data/tmp' nglims_config_file 'tool-data/nglims.yaml' outputs_to_working_directory 'True' qa_url 'http://slyfox.bx.psu.edu:8080/' remote_user_maildomain 'embl.de' require_login 'True' retry_job_output_collection '10' sanitize_all_html 'False' set_metadata_externally 'True' smtp_server 'smtp.embl.de' start_job_runners 'drmaa' static_cache_time '360' static_dir '/g/funcgen/galaxy/static/' static_enabled 'True' static_favicon_dir '/g/funcgen/galaxy/static/favicon.ico' static_images_dir '/g/funcgen/galaxy/static/images' static_robots_txt '/g/funcgen/galaxy/static/robots.txt' static_scripts_dir '/g/funcgen/galaxy/static/scripts/' static_style_dir '/g/funcgen/galaxy/static/june_2007_style/blue' tool_config_file 'tool_conf.xml,shed_tool_conf.xml' tool_dependency_dir '/g/funcgen/galaxy/dependencies' tool_path 'tools' track_jobs_in_database 'True' use_interactive 'False' use_nglims 'False' use_remote_user 'True' WSGI Variables application <paste.debug.prints.PrintDebugMiddleware object at 0x22dbdf90> paste.cookies (<SimpleCookie: galaxysession='7df64c2bf0628c5b57afd358adee6e7b2cd6161345bb6b47388e497cbd26f407876c2bed128d9226'wikidb_ngs_Token='1f38fe6b5eb5863596cad189f608835c' wikidb_ngs_UserID='15' wikidb_ngs_UserName='Girardot' wikidb_ngs__session='d9n3qq68n0nl6p6j9t96k5q6m3'>, 'galaxysession=7df64c2bf0628c5b57afd358adee6e7b2cd6161345bb6b47388e497cbd26f407876c2bed128d9226;wikidb_ngs_Token=1f38fe6b5eb5863596cad189f608835c; wikidb_ngs_UserName=Girardot; wikidb_ngs_UserID=15; wikidb_ngs__session=d9n3qq68n0nl6p6j9t96k5q6m3') paste.expected_exceptions [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions <paste.httpexceptions.HTTPExceptionHandler object at 0x22928b10> paste.httpserver.thread_pool <paste.httpserver.ThreadPool object at 0x1ddf9950> paste.printdebug_listeners [<cStringIO.StringO object at 0x28c57618>, <paste.script.serve.LazyWriter object at 0x1dd5a6d0>] paste.recursive.forward <paste.recursive.Forwarder from /galaxy> paste.recursive.include <paste.recursive.Includer from /galaxy> paste.recursive.include_app_iter <paste.recursive.IncluderAppIter from /galaxy> paste.recursive.script_name '/galaxy' paste.remove_printdebug <function remove_printdebug at 0x2aaaac032aa0> paste.throw_errors True webob._parsed_query_vars (MultiDict([]), '') wsgi process 'Multithreaded' ===================================== Charles Girardot European Molecular Biology Laboratory GBCS / Furlong Group Tel: +49 6221 387 -8585(V205) or -8433 (V320) Fax: +49-(0)6221-387-8166 Email: charles.girardot(a)embl.de Room V205 (GBCS)/V320 (Furlong Group) Meyerhofstraße 1, 69117 Heidelberg, Germany =====================================

1 1

envirtual and import numpy problem
by Huayan Gao 16 Jan '13

16 Jan '13

Hi, First it started with the error message: self.check_version_conflict(). So I use the virtualenv as instructed in wiki. But I got a new error: ImportError: No module named rpy/numpy....... I will not have this importerror if I dont use the virtualenv. How to solve this problem? Thanks, Huayan

1 0

local galaxy tool shed install
by mark.rose＠syngenta.com 16 Jan '13

16 Jan '13

Having followed the directions in "HostingALocalToolShed" I ran run_community.sh with the default configurations in community_wsgi.ini. It executed without error but failed to start the tool shed as http://127.0.0.1:9009 returns "Unable to Connect" error Thanks for your help Mark This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited.

2 1

Synchronize Data Library with Directory
by Swanson, Erik R 15 Jan '13

15 Jan '13

I installed a local Galaxy server with Apache proxy, kerberos authentication and submitting to a cluster with DRMAA. Everything works great, so far. Sequence data is NFS mounted on the Galaxy server and all cluster nodes at /shared/ngs/<labname>. I would like to configure one Data Library for each lab, add the data from the lab's ngs directory to their library, and synchronize the library and directory. I used "allow_library_path_paste = True" and "Upload files from filesystem paths" to add data to a library from an ngs lab directory. This works great the first time, but if you do it a second time the data is duplicated in the library. 1) Is there a way to synchronize a data library and directory? 2) If so, is there a way to automate the synchronization, perhaps scripted with cron? Thanks for your help! -Erik

1 0

Can't see history settings, reload etc. after recent update
by Anthonius deBoer 15 Jan '13

15 Jan '13

Hi, I just updated to the latest galaxy-dist, but now my history pane is not showing the options and no reload etc. Attached a picture, not sure it will come through... Any ideas? Now my galaxy is totally broken...I was hoping not to have to worry about it as much by switching to the galaxy-dist,from galaxy-central..... Hope I can revert the changes.... Thon

2 2

DRMAA runner weirdness
by Kyle Ellrott 15 Jan '13

15 Jan '13

I'm running a test Galaxy system on a cluster (merged galaxy-dist on Janurary 4th). And I've noticed some odd behavior from the DRMAA job runner. I'm running a multithread system, one web server, one job_manager, and three job_handlers. DRMAA is the default job runner (the command for tophat2 is drmaa://-V -l mem_total=7G -pe smp 2/), with SGE 6.2u5 being the engine underneath. My test involves trying to run three different Tophat2 jobs. The first two seem to start up (and get put on the SGE queue), but the third stays grey, with the job manager listing it in state 'new' with command line 'None'. It doesn't seem to leave this state. Both of the jobs that actually got onto the queue die (reasons unknown, but much to early, probably some tophat/bowtie problem), but one job is listed in error state with stderr as 'Job output not returned from cluster', while the other job (which is no longer in the SGE queue) is still listed as running. Any ideas? Kyle

3 6

FTP and data library uploads
by Shantanu Pavgi (Campus) 15 Jan '13

15 Jan '13

I am trying to understand how FTP and data library upload options are working in Galaxy. When a non-binary file is uploaded through FTP option, it goes through three move operations: 1. First it is copied to a temporary namespace line-by-line converting newlines 2. Then the temporary file is moved back to the FTP directory with the same name [1] 3. Later the newline sanitized FTP file is moved to datasets directory These move operations in Python are carried as copy and delete tasks. I don't see the same approach being taken with data libraries or other file-system import/upload options. I looked at library_common code, but I couldn't follow it [2]. I was wondering if someone could help in understanding how file upload is implemented for different upload mechanisms and datatypes. Also, can FTP upload option reduce the number of move operations? For example, can the original FTP file or temporary file copied/moved directly to the datasets directory? This will be helpful in supporting FTP-type upload where 'galaxy' user isn't the primary owner of user's files (move operations perform chmod and it requires primary ownership). -- Thanks, Shantanu 1. https://bitbucket.org/galaxy/galaxy-central/src/530fb4f8204f2106e11f419c381… 2. https://bitbucket.org/galaxy/galaxy-central/src/530fb4f8204f/lib/galaxy/web…

2 3