- galaxy-dev - lists.galaxyproject.org

Setup Galaxy with Son of Grid
by Vipul Patel 18 Feb '15

18 Feb '15

Hi, I try to setup a new galaxy env by following the wiki (used nginx, uwsgi) Everything is actually working but if I try to add the grid engine, my jobs will not be fired... I already found this post and try to adept it.. but well it did not helped. http://dev.list.galaxyproject.org/quot-This-job-is-waiting-to-run-quot-td46… The grid engine it self runs. Is there any way to observe on the fly what actually happens. All I see are just that the galaxy server started. Currently I have no clue how I can solve this problem. Here is my job_conf <job_conf> <plugins workers="2"> <plugin id="gridengine" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"> <param id="drmaa_library_path">/opt/sge/lib/lx-amd64/libdrmaa.so</param> </plugin>  </plugins> <handlers default="handlers"> <handler id="handler0" tags="handlers"/> <handler id="handler1" tags="handlers"/> </handlers> <destinations default="cluster"> <destination id="cluster" runner="gridengine"/>  </destinations> <limits> </limits> </job_conf> best vipul

1 0

apache proxy
by Michael Thon 18 Feb '15

18 Feb '15

If I use apache to proxy connections to the galaxy run.sh process, what is the best way to launch galaxy at boot time? Can apache do it?

2 1

fastq format recognition
by Michael Thon 18 Feb '15

18 Feb '15

I have a new galaxy installation on my server. I uploaded two fastq files and now I'm trying to run trimmomatic and fastqc. In both cases I am presented with a drop down list of fastq files to select from but the lists are empty. I double checked the fastq file attributes and they are both set to fastq. Any ideas what's wrong? Another question: I have fastq files already on the same server as the galaxy installation. How can I get them into galaxy? I have not set up ftp yet. Can I move them into a data directory somewhere? Thanks!

4 7

LD_LIBRARY_PATH not exporting to galaxy
by Nikhil Joshi 18 Feb '15

18 Feb '15

Hi all, So we have forked the stable branch of galaxy and are making our own modifications. We are running galaxy on Ubuntu 14.04.1 LTS and we're using modules to load the underlying software, i.e. set environment variables. This works just fine for PATH, JAVA_JAR_PATH, PYTHONPATH, etc., however, LD_LIBRARY_PATH is not being passed to the tools. When a tool (such as cuffdiff) tries to access LD_LIBRARY_PATH, it is empty. We've checked to make sure that loading the cufflinks module correctly updates the LD_LIBRARY_PATH, however, when it is used in galaxy, all the other environment variables are set properly, but not LD_LIBRARY_PATH. Does anyone have any ideas on why this would happen and how to fix it? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

4 10

Starting uWSGI
by Nicholas Kline 17 Feb '15

17 Feb '15

Hi Nate, So I went ahead and set `static_enabled = False` in universe_wsgi.ini, restarted uWSGI, and browsed to localhost. I'm including the results of each step below: After restarting uWSGI, the last few lines of uwsgi.log looked like this: WSGI app 0 (mountpoint='') ready in 11 seconds on interpreter 0xa934c0 pid: 3561 (default app) *** uWSGI is running in multiple interpreter mode *** spawned uWSGI master process (pid: 3561) spawned uWSGI worker 1 (pid: 3575, cores: 4) spawned uWSGI worker 2 (pid: 3576, cores: 4) spawned uWSGI worker 3 (pid: 3583, cores: 4) spawned uWSGI worker 4 (pid: 3587, cores: 4) spawned uWSGI worker 5 (pid: 3591, cores: 4) spawned uWSGI worker 6 (pid: 3595, cores: 4) spawned uWSGI worker 7 (pid: 3599, cores: 4) spawned uWSGI worker 8 (pid: 3603, cores: 4) spawned uWSGI worker 9 (pid: 3607, cores: 4) spawned uWSGI worker 10 (pid: 3611, cores: 4) *** Stats server enabled on 127.0.0.1:9191 fd: 30 *** ...so no traceback this time! Then, browsing to localhost redirects to https://localhost and finally returns: "Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team." That request was logged to uwsgi.log: 127.0.0.1 - - [03/Feb/2015:14:51:01 -0400] "GET / HTTP/1.1" 500 - "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.KeyError'>: 'SCRIPT_NAME' URL: https://localhost/ File 'lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File 'lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File 'lib/galaxy/web/framework/base.py', line 159 in handle_request trans = self.transaction_factory( environ ) File 'lib/galaxy/web/framework/webapp.py', line 71 in <lambda> self.set_transaction_factory( lambda e: self.transaction_chooser( e, galaxy_app, session_cookie ) ) File 'lib/galaxy/web/framework/webapp.py', line 102 in transaction_chooser return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie ) File 'lib/galaxy/web/framework/webapp.py', line 207 in __init__ self._ensure_logged_in_user( environ, session_cookie ) File 'lib/galaxy/web/framework/webapp.py', line 441 in _ensure_logged_in_user if self.request.path.startswith( external_display_path ): File 'lib/galaxy/web/framework/base.py', line 258 in __get__ value = self.func( obj ) File 'lib/galaxy/web/framework/base.py', line 341 in path return self.environ['SCRIPT_NAME'] + self.environ['PATH_INFO'] KeyError: 'SCRIPT_NAME' CGI Variables ------------- DOCUMENT_ROOT: '/usr/share/nginx/html' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_HOST: 'localhost' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/' REMOTE_ADDR: '127.0.0.1' REMOTE_PORT: '50064' REQUEST_METHOD: 'GET' REQUEST_URI: '/' SERVER_NAME: 'localhost' SERVER_PORT: '443' SERVER_PROTOCOL: 'HTTP/1.1' UWSGI_SCHEME: 'https' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f2ec2f32290> is_api_request: False paste.cookies: (<SimpleCookie: >, '') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f2ec2f32210> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fadecee2abdd11e4bfb614feb5e02dd6' uwsgi.core: 0 uwsgi.node: 'i changed this string to hide the server name' uwsgi.version: '2.0.9' wsgi process: 'Multi process AND threads (?)' wsgi.file_wrapper: <built-in function uwsgi_sendfile> ------------------------------------------------------------ [pid: 3607|app: 0|req: 1/1] 127.0.0.1 () {40 vars in 593 bytes} [Tue Feb 3 14:51:01 2015] GET / => generated 777 bytes in 230 msecs (HTTP/1.1 500) 1 headers in 63 bytes (1 switches on core 0) I thought I would include some pieces of universe_wsgi.ini to help with troubleshooting: # ---- HTTP Server ---------------------------------------------------------- # Configuration of the internal HTTP server. [server:main] # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http # The port on which to listen. #port = 8080 # The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. host = 0.0.0.0 # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 10 # Set the number of seconds a thread can work before you should kill it (assuming it will never finish) to 3 hours. threadpool_kill_thread_limit = 10800 # ---- uwsgi ---------------------------------------------------------- [uwsgi] processes = 10 stats = 127.0.0.1:9191 socket = 127.0.0.1:4001 pythonpath = lib threads = 4 logto = /home/galaxy/galaxy-dist/uwsgi.log master = True Also, we are using a self-signed certificate as a temporary solution, so not sure if that matters. Thanks!

2 3

Re: [galaxy-dev] Galaxy CloudMan instance - missing link 'display at UCSC' for bed files
by Anne Pajon 16 Feb '15

16 Feb '15

Many thanks for helping. Here is what I found in the universe_wsgi.ini ucsc_display_sites = main,test,archaea,ucla Where else should I look? Any other suggestions… many thanks in advance. Best regards, Anne. On 16 Feb 2015, at 12:30, Hans-Rudolf Hotz <hrh(a)fmi.ch> wrote: > Hi Anne > > Have you also checked 'universe_wsgi.ini' to make sure it contains: > > ucsc_display_sites = main > > Though, this might me stored in config/galaxy.ini in the current Galaxy CloudMan code > > > Hans-Rudolf > > > > On 02/16/2015 12:29 PM, Anne Pajon wrote: >> Hi Hans, >> >> Many thanks for your suggestion. I’ve looked at ucsc_build_sites.txt and >> its content is: >> >> #Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn >> mainhttp://genome.ucsc.edu/cgi-bin/hgTracks?priPac1,danRer4,mm9,mm8,droAna1… >> #Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn >> archaeahttp://archaea.ucsc.edu/cgi-bin/hgTracks?alkaEhrl_MLHE_1,shewW318,id… > eot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_T > ULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDN > IH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00 >> #Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn >> testhttp://genome-test.cse.ucsc.edu/cgi-bin/hgTracks?anoCar1,ce4,ce3,ce2,ce… > ,cb2,cb3,cb1,borEut13,droSec1,felCat3,strPur1,strPur2,otoGar1,catArr1,anoGam1,triCas2 >> uclahttp://epigenomics.mcdb.ucla.edu/cgi-bin/hgTracks?araTha1 >> >> It does contain hg19, the genome of the bed file I am trying to get >> displayed at UCSC but I still do not have the link to do so… any other >> ideas where I should look? Many thanks in advance. >> >> >> Best regards, >> Anne. >> >> On 16 Feb 2015, at 08:01, Hans-Rudolf Hotz <hrh(a)fmi.ch >> <mailto:hrh@fmi.ch>> wrote: >> >>> Hi Anne >>> >>> Have you also checked: ~/tool-data/shared/ucsc/ucsc_build_sites.txt >>> to make sure the the 'genome' of your bed file is actually listed ? >>> >>> >>> Regards, Hans-Rudolf >>> >>> >>> >>> >>> On 02/13/2015 07:05 PM, Anne Pajon wrote: >>>> Dear All, >>>> >>>> I’ve installed a Galaxy CloudMan instance on amazon for a training >>>> course we will be giving at the end of the month, as a re-run of a >>>> course we did last September. I am facing a problem as I am unable to >>>> get the link 'display at UCSC’ for our bed files, and it was not the >>>> case back in September. >>>> >>>> So far, I’ve edited /mnt/galaxy/galaxy-app/datatypes_conf.xml in our >>>> cloud instance and uncommented <display >>>> file="ucsc/interval_as_bed.xml" /> but it did not change anything >>>> after a re-start. Any ideas what could have changed? Or how I could >>>> turn this back on? Any help would be greatly appreciated otherwise I >>>> will have to rewrite part of the training materials… Many thanks in >>>> advance. >>>> >>>> Best wishes, >>>> Anne. >>>> -- >>>> Dr Anne Pajon - Bioinformatics Core >>>> Cancer Research UK - Cambridge Institute >>>> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >>>> anne.pajon(a)cruk.cam.ac.uk <mailto:anne.pajon@cruk.cam.ac.uk> | +44 >>>> (0)1223 769 631 >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >>>> >> >> -- >> Dr Anne Pajon - Bioinformatics Core >> Cancer Research UK - Cambridge Institute >> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >> anne.pajon(a)cruk.cam.ac.uk <mailto:anne.pajon@cruk.cam.ac.uk> | +44 >> (0)1223 769 631 -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 769 631

3 3

Getting empty files when upload
by Erwan DELAGE 16 Feb '15

16 Feb '15

Hello dear galaxy developpers, I'm facing a weird bug when trying to upload files in Galaxy. When I want to upload, let's say for example a csv file, the dataset turns green but the file is empty. I checked in the galaxy datasets directory, the file is here but empty. And this happens only when configuring Galaxy to work with SGE on a cluster, not when running "locally". I did take a look at the script upload.py and I found that if I replace the call to shutil.move( dataset.path, output_path ) by calling shutil.copy(dataset.path, output_path) at the end of the add_file function, then it works. Both Galaxy temp and dataset directories are on the same partition, and when I try from a cluster execution node to manually do a shutil.move from the temp directory to the dataset directory with the galaxy user, it works well. Any idea of what could go wrong here ? Thanks Erwan

1 0

Galaxy CloudMan instance - missing link 'display at UCSC' for bed files
by Anne Pajon 16 Feb '15

16 Feb '15

Dear All, I’ve installed a Galaxy CloudMan instance on amazon for a training course we will be giving at the end of the month, as a re-run of a course we did last September. I am facing a problem as I am unable to get the link 'display at UCSC’ for our bed files, and it was not the case back in September. So far, I’ve edited /mnt/galaxy/galaxy-app/datatypes_conf.xml in our cloud instance and uncommented <display file="ucsc/interval_as_bed.xml" /> but it did not change anything after a re-start. Any ideas what could have changed? Or how I could turn this back on? Any help would be greatly appreciated otherwise I will have to rewrite part of the training materials… Many thanks in advance. Best wishes, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 769 631

2 3

R lib dependencies in the toolshed
by Nikos Sidiropoulos 14 Feb '15

14 Feb '15

Hi all, I have a package repository that installs an R package and another repository with a set of tools that use that package repository. Tools that use an Rscript command interpreter load that module successfully, e.g.: "normalize.xml" <requirements> <requirement type="package" version="3.1.1">R_3_1_1</requirement> <requirement type="R-module">RNAprobR</requirement> <requirement type="package" version="1.0.0">RNAprobR</requirement> <requirement type="set_environment">RNA_PROBING_SCRIPT_PATH</requirement> </requirements> <command interpreter="Rscript"> normalize.R .... </command> "normalize.R" suppressMessages(require(RNAprobR)) ... But there is a tool which executes an Rscript indirectly, meaning: "summarize_unique_barcodes.sh" <requirements> <requirement type="package" version="4.1.0">gnu_awk</requirement> <requirement type="package" version="0.1.19">samtools</requirement> <requirement type="package" version="3.1.1">R_3_1_1</requirement> <requirement type="R-module">RNAprobR</requirement> <requirement type="package" version="1.0.0">RNAprobR</requirement> <requirement type="set_environment">RNA_PROBING_SCRIPT_PATH</requirement> </requirements> <command interpreter="bash"> summarize_unique_barcodes.sh ... -r \$RNA_PROBING_SCRIPT_PATH </command> "summarize_unique_barcodes.sh" .... #Produce k2n file if [ "$k2n" == "True" ]; then Rscript $R_SCRIPT_PATH/k2n.R merged_temp.gz $output_dir/unique_barcodes.txt $output_dir/k2n.txt fi "k2n.R" suppressMessages(require(RNAprobR)) I always get the following error: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, : there is no package called 'RNAprobR' A solution to the problem could be to pass the R library path of the RNAprobR module or the installation folder of package "RNAprobR" as a parameter to the bash wrapper and then the k2n.R script but I can't seem to be able to do it. Do you have any suggestions? Regards, Nikos

2 2

Re: [galaxy-dev] Tool test failure
by Dooley, Damion 13 Feb '15

13 Feb '15

I tried using the default config/job_conf.xml but that didn't make a difference. I'll just try testing on a different solo galaxy instance, and will report back. I'm guessing it will work elsewhere. Message: 3 ... Subject: Re: [galaxy-dev] Tool test failure - I could use some help... ... Thanks for the tips, Peter. Hmm. No amount of playing with my galaxy-dist/tool_conf.xml.sample file is enabling the tool to be found by run_tests.sh . In the run_functional_tests.html I see:

1 0