February 2015 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] Galaxy server and ssl
by Dannon Baker 19 Feb '15

19 Feb '15

You may have done this already, but if not -- can you try editing config/tool_sheds_conf.xml and change the two url entries to https? If this file doesn't exist, create it from the .sample and then make the modifications. -Dannon On Thu Feb 19 2015 at 8:17:32 AM Scott Szakonyi <Scott.B.Szakonyi.1(a)nd.edu> wrote: > Hi Dannon, > > There are two things I'm seeing, which seem to be independent problems. > > 1. Https/ssl issue with URLs generated in util.py. It seems the > detection of Https is not working properly, therefore the links underlying > the toolshed functionality are coming up as http rather than https. The > browser then refuses to following them because they are "mixed mode". I've > gone around this for the moment by hacking the code in util.py, but it's > possible (as I'm very new to server administration) that I'm missing some > necessary configuration item that I just don't know about. > 2. Toolshed installation is not working properly when Galaxy is run > with multiple processes. New categories are not created, and the installed > tool does not show up in user menus until the Galaxy server is restarted. > I've worked around this by running the Galaxy server in single process mode > while installing tools. > > Thanks for the link... I'll look into that and see what I can glean from > it. > > Best regards, > > Scott > > On Wed, Feb 18, 2015 at 7:14 PM, Dannon Baker <dannon.baker(a)gmail.com> > wrote: > >> Hey Scott, >> >> What weird behavior are you seeing? Paste itself isn't going to serve >> https, though it's definitely possible to configure an upstream proxy to do >> so. Are you using a setup similar to https://wiki.galaxyproject. >> org/Admin/Config/nginxProxy#SSL ? >> >> -Dannon >> >> On Wed Feb 18 2015 at 4:34:51 PM Scott Szakonyi < >> Scott.B.Szakonyi.1(a)nd.edu> wrote: >> >>> Hello all, >>> >>> Just curious if anyone has successfully implemented Galaxy with ssl and >>> standalone Paste-based processes. We're attempting to do so, and we're >>> getting some weird behavior, particularly with regards to the Toolshed. >>> Just want to know if it can be done successfully. >>> >>> Thanks! >>> >>> -- >>> Scott B. Szakonyi >>> Research Programmer >>> >>> *Center for Research Computing* >>> 107 Information Technology Center >>> Notre Dame, IN 46556 >>> http://crc.nd.edu >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> https://lists.galaxyproject.org/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >> >> > > > -- > Scott B. Szakonyi > Research Programmer > > *Center for Research Computing* > 107 Information Technology Center > Notre Dame, IN 46556 > http://crc.nd.edu >

2 1

Re: [galaxy-dev] Galaxy server and ssl
by Dannon Baker 19 Feb '15

19 Feb '15

Hey Scott, What weird behavior are you seeing? Paste itself isn't going to serve https, though it's definitely possible to configure an upstream proxy to do so. Are you using a setup similar to https://wiki.galaxyproject.org/Admin/Config/nginxProxy#SSL ? -Dannon On Wed Feb 18 2015 at 4:34:51 PM Scott Szakonyi <Scott.B.Szakonyi.1(a)nd.edu> wrote: > Hello all, > > Just curious if anyone has successfully implemented Galaxy with ssl and > standalone Paste-based processes. We're attempting to do so, and we're > getting some weird behavior, particularly with regards to the Toolshed. > Just want to know if it can be done successfully. > > Thanks! > > -- > Scott B. Szakonyi > Research Programmer > > *Center for Research Computing* > 107 Information Technology Center > Notre Dame, IN 46556 > http://crc.nd.edu > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/

2 1

Galaxy server and ssl
by Scott Szakonyi 18 Feb '15

18 Feb '15

Hello all, Just curious if anyone has successfully implemented Galaxy with ssl and standalone Paste-based processes. We're attempting to do so, and we're getting some weird behavior, particularly with regards to the Toolshed. Just want to know if it can be done successfully. Thanks! -- Scott B. Szakonyi Research Programmer *Center for Research Computing* 107 Information Technology Center Notre Dame, IN 46556 http://crc.nd.edu

1 0

Setup Galaxy with Son of Grid
by Vipul Patel 18 Feb '15

18 Feb '15

Hi, I try to setup a new galaxy env by following the wiki (used nginx, uwsgi) Everything is actually working but if I try to add the grid engine, my jobs will not be fired... I already found this post and try to adept it.. but well it did not helped. http://dev.list.galaxyproject.org/quot-This-job-is-waiting-to-run-quot-td46… The grid engine it self runs. Is there any way to observe on the fly what actually happens. All I see are just that the galaxy server started. Currently I have no clue how I can solve this problem. Here is my job_conf <job_conf> <plugins workers="2"> <plugin id="gridengine" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"> <param id="drmaa_library_path">/opt/sge/lib/lx-amd64/libdrmaa.so</param> </plugin>  </plugins> <handlers default="handlers"> <handler id="handler0" tags="handlers"/> <handler id="handler1" tags="handlers"/> </handlers> <destinations default="cluster"> <destination id="cluster" runner="gridengine"/>  </destinations> <limits> </limits> </job_conf> best vipul

1 0

apache proxy
by Michael Thon 18 Feb '15

18 Feb '15

If I use apache to proxy connections to the galaxy run.sh process, what is the best way to launch galaxy at boot time? Can apache do it?

2 1

fastq format recognition
by Michael Thon 18 Feb '15

18 Feb '15

I have a new galaxy installation on my server. I uploaded two fastq files and now I'm trying to run trimmomatic and fastqc. In both cases I am presented with a drop down list of fastq files to select from but the lists are empty. I double checked the fastq file attributes and they are both set to fastq. Any ideas what's wrong? Another question: I have fastq files already on the same server as the galaxy installation. How can I get them into galaxy? I have not set up ftp yet. Can I move them into a data directory somewhere? Thanks!

4 7

LD_LIBRARY_PATH not exporting to galaxy
by Nikhil Joshi 17 Feb '15

17 Feb '15

Hi all, So we have forked the stable branch of galaxy and are making our own modifications. We are running galaxy on Ubuntu 14.04.1 LTS and we're using modules to load the underlying software, i.e. set environment variables. This works just fine for PATH, JAVA_JAR_PATH, PYTHONPATH, etc., however, LD_LIBRARY_PATH is not being passed to the tools. When a tool (such as cuffdiff) tries to access LD_LIBRARY_PATH, it is empty. We've checked to make sure that loading the cufflinks module correctly updates the LD_LIBRARY_PATH, however, when it is used in galaxy, all the other environment variables are set properly, but not LD_LIBRARY_PATH. Does anyone have any ideas on why this would happen and how to fix it? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)

4 10

Starting uWSGI
by Nicholas Kline 17 Feb '15

17 Feb '15

Hi Nate, So I went ahead and set `static_enabled = False` in universe_wsgi.ini, restarted uWSGI, and browsed to localhost. I'm including the results of each step below: After restarting uWSGI, the last few lines of uwsgi.log looked like this: WSGI app 0 (mountpoint='') ready in 11 seconds on interpreter 0xa934c0 pid: 3561 (default app) *** uWSGI is running in multiple interpreter mode *** spawned uWSGI master process (pid: 3561) spawned uWSGI worker 1 (pid: 3575, cores: 4) spawned uWSGI worker 2 (pid: 3576, cores: 4) spawned uWSGI worker 3 (pid: 3583, cores: 4) spawned uWSGI worker 4 (pid: 3587, cores: 4) spawned uWSGI worker 5 (pid: 3591, cores: 4) spawned uWSGI worker 6 (pid: 3595, cores: 4) spawned uWSGI worker 7 (pid: 3599, cores: 4) spawned uWSGI worker 8 (pid: 3603, cores: 4) spawned uWSGI worker 9 (pid: 3607, cores: 4) spawned uWSGI worker 10 (pid: 3611, cores: 4) *** Stats server enabled on 127.0.0.1:9191 fd: 30 *** ...so no traceback this time! Then, browsing to localhost redirects to https://localhost and finally returns: "Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team." That request was logged to uwsgi.log: 127.0.0.1 - - [03/Feb/2015:14:51:01 -0400] "GET / HTTP/1.1" 500 - "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.KeyError'>: 'SCRIPT_NAME' URL: https://localhost/ File 'lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File 'lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File 'lib/galaxy/web/framework/base.py', line 159 in handle_request trans = self.transaction_factory( environ ) File 'lib/galaxy/web/framework/webapp.py', line 71 in <lambda> self.set_transaction_factory( lambda e: self.transaction_chooser( e, galaxy_app, session_cookie ) ) File 'lib/galaxy/web/framework/webapp.py', line 102 in transaction_chooser return GalaxyWebTransaction( environ, galaxy_app, self, session_cookie ) File 'lib/galaxy/web/framework/webapp.py', line 207 in __init__ self._ensure_logged_in_user( environ, session_cookie ) File 'lib/galaxy/web/framework/webapp.py', line 441 in _ensure_logged_in_user if self.request.path.startswith( external_display_path ): File 'lib/galaxy/web/framework/base.py', line 258 in __get__ value = self.func( obj ) File 'lib/galaxy/web/framework/base.py', line 341 in path return self.environ['SCRIPT_NAME'] + self.environ['PATH_INFO'] KeyError: 'SCRIPT_NAME' CGI Variables ------------- DOCUMENT_ROOT: '/usr/share/nginx/html' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_HOST: 'localhost' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/' REMOTE_ADDR: '127.0.0.1' REMOTE_PORT: '50064' REQUEST_METHOD: 'GET' REQUEST_URI: '/' SERVER_NAME: 'localhost' SERVER_PORT: '443' SERVER_PROTOCOL: 'HTTP/1.1' UWSGI_SCHEME: 'https' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f2ec2f32290> is_api_request: False paste.cookies: (<SimpleCookie: >, '') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f2ec2f32210> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fadecee2abdd11e4bfb614feb5e02dd6' uwsgi.core: 0 uwsgi.node: 'i changed this string to hide the server name' uwsgi.version: '2.0.9' wsgi process: 'Multi process AND threads (?)' wsgi.file_wrapper: <built-in function uwsgi_sendfile> ------------------------------------------------------------ [pid: 3607|app: 0|req: 1/1] 127.0.0.1 () {40 vars in 593 bytes} [Tue Feb 3 14:51:01 2015] GET / => generated 777 bytes in 230 msecs (HTTP/1.1 500) 1 headers in 63 bytes (1 switches on core 0) I thought I would include some pieces of universe_wsgi.ini to help with troubleshooting: # ---- HTTP Server ---------------------------------------------------------- # Configuration of the internal HTTP server. [server:main] # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http # The port on which to listen. #port = 8080 # The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. host = 0.0.0.0 # Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True # Number of threads in the web server thread pool. threadpool_workers = 10 # Set the number of seconds a thread can work before you should kill it (assuming it will never finish) to 3 hours. threadpool_kill_thread_limit = 10800 # ---- uwsgi ---------------------------------------------------------- [uwsgi] processes = 10 stats = 127.0.0.1:9191 socket = 127.0.0.1:4001 pythonpath = lib threads = 4 logto = /home/galaxy/galaxy-dist/uwsgi.log master = True Also, we are using a self-signed certificate as a temporary solution, so not sure if that matters. Thanks!

2 3

Re: [galaxy-dev] Galaxy CloudMan instance - missing link 'display at UCSC' for bed files
by Anne Pajon 16 Feb '15

16 Feb '15

Many thanks for helping. Here is what I found in the universe_wsgi.ini ucsc_display_sites = main,test,archaea,ucla Where else should I look? Any other suggestions… many thanks in advance. Best regards, Anne. On 16 Feb 2015, at 12:30, Hans-Rudolf Hotz <hrh(a)fmi.ch> wrote: > Hi Anne > > Have you also checked 'universe_wsgi.ini' to make sure it contains: > > ucsc_display_sites = main > > Though, this might me stored in config/galaxy.ini in the current Galaxy CloudMan code > > > Hans-Rudolf > > > > On 02/16/2015 12:29 PM, Anne Pajon wrote: >> Hi Hans, >> >> Many thanks for your suggestion. I’ve looked at ucsc_build_sites.txt and >> its content is: >> >> #Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn >> mainhttp://genome.ucsc.edu/cgi-bin/hgTracks?priPac1,danRer4,mm9,mm8,droAna1… >> #Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn >> archaeahttp://archaea.ucsc.edu/cgi-bin/hgTracks?alkaEhrl_MLHE_1,shewW318,id… > eot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_T > ULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDN > IH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00 >> #Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn >> testhttp://genome-test.cse.ucsc.edu/cgi-bin/hgTracks?anoCar1,ce4,ce3,ce2,ce… > ,cb2,cb3,cb1,borEut13,droSec1,felCat3,strPur1,strPur2,otoGar1,catArr1,anoGam1,triCas2 >> uclahttp://epigenomics.mcdb.ucla.edu/cgi-bin/hgTracks?araTha1 >> >> It does contain hg19, the genome of the bed file I am trying to get >> displayed at UCSC but I still do not have the link to do so… any other >> ideas where I should look? Many thanks in advance. >> >> >> Best regards, >> Anne. >> >> On 16 Feb 2015, at 08:01, Hans-Rudolf Hotz <hrh(a)fmi.ch >> <mailto:hrh@fmi.ch>> wrote: >> >>> Hi Anne >>> >>> Have you also checked: ~/tool-data/shared/ucsc/ucsc_build_sites.txt >>> to make sure the the 'genome' of your bed file is actually listed ? >>> >>> >>> Regards, Hans-Rudolf >>> >>> >>> >>> >>> On 02/13/2015 07:05 PM, Anne Pajon wrote: >>>> Dear All, >>>> >>>> I’ve installed a Galaxy CloudMan instance on amazon for a training >>>> course we will be giving at the end of the month, as a re-run of a >>>> course we did last September. I am facing a problem as I am unable to >>>> get the link 'display at UCSC’ for our bed files, and it was not the >>>> case back in September. >>>> >>>> So far, I’ve edited /mnt/galaxy/galaxy-app/datatypes_conf.xml in our >>>> cloud instance and uncommented <display >>>> file="ucsc/interval_as_bed.xml" /> but it did not change anything >>>> after a re-start. Any ideas what could have changed? Or how I could >>>> turn this back on? Any help would be greatly appreciated otherwise I >>>> will have to rewrite part of the training materials… Many thanks in >>>> advance. >>>> >>>> Best wishes, >>>> Anne. >>>> -- >>>> Dr Anne Pajon - Bioinformatics Core >>>> Cancer Research UK - Cambridge Institute >>>> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >>>> anne.pajon(a)cruk.cam.ac.uk <mailto:anne.pajon@cruk.cam.ac.uk> | +44 >>>> (0)1223 769 631 >>>> >>>> ___________________________________________________________ >>>> Please keep all replies on the list by using "reply all" >>>> in your mail client. To manage your subscriptions to this >>>> and other Galaxy lists, please use the interface at: >>>> https://lists.galaxyproject.org/ >>>> >>>> To search Galaxy mailing lists use the unified search at: >>>> http://galaxyproject.org/search/mailinglists/ >>>> >> >> -- >> Dr Anne Pajon - Bioinformatics Core >> Cancer Research UK - Cambridge Institute >> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >> anne.pajon(a)cruk.cam.ac.uk <mailto:anne.pajon@cruk.cam.ac.uk> | +44 >> (0)1223 769 631 -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cruk.cam.ac.uk | +44 (0)1223 769 631

3 3

Getting empty files when upload
by Erwan DELAGE 16 Feb '15

16 Feb '15

Hello dear galaxy developpers, I'm facing a weird bug when trying to upload files in Galaxy. When I want to upload, let's say for example a csv file, the dataset turns green but the file is empty. I checked in the galaxy datasets directory, the file is here but empty. And this happens only when configuring Galaxy to work with SGE on a cluster, not when running "locally". I did take a look at the script upload.py and I found that if I replace the call to shutil.move( dataset.path, output_path ) by calling shutil.copy(dataset.path, output_path) at the end of the add_file function, then it works. Both Galaxy temp and dataset directories are on the same partition, and when I try from a cluster execution node to manually do a shutil.move from the temp directory to the dataset directory with the galaxy user, it works well. Any idea of what could go wrong here ? Thanks Erwan

1 0