April 2013 - galaxy-dev - lists.galaxyproject.org

Knowing who is currently logged into your system
by Jennifer Jackson 20 Apr '13

20 Apr '13

Hi Akshay, I am going to post your question over to the galaxy-dev(a)bx.psu.edu mailing list, to give it better viability. You could probably run an sql query against a database table to find out this information, but there may be a better way. Let's see if someone has a method worked out they want to share. Another alternative is to look in our documentation, ReadTheDocs. I did a search on "login" and a few potentials popped up: http://galaxy-dist.readthedocs.org/en/latest/ Going forward, the galaxy-dev list is the list you will want to post to - and consider subscribing to - if you are running a local instance and want to join/discuss issues with the community of other users doing the same. http://wiki.galaxyproject.org/MailingLists http://wiki.galaxyproject.org/Support#Mailing_Lists Thanks! Jen Galaxy team On 3/12/13 5:46 PM, Akshay Vivek Choche wrote: > Hello All, > > I was wondering if there is a way to know all the users logged into your local galaxy server? > > Thanks, > -Akshay Choche > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ -- Jennifer Hillman-Jackson Galaxy Support and Training http://galaxyproject.org

4 5

Error in universe_wsgi.ini regarding gzip
by Perez, Ricardo 19 Apr '13

19 Apr '13

Dear all, I have encountered a problem where my browser does not show the history sidebar in galaxy. In my configuration I had the following line in universe_wsgi.ini uncommented, which seem to be the source of the problem. #filter-with = gzip Commenting that line will fix my browser and see the history bar again. Thanks, --Ricardo Perez

1 0

Errors when starting galaxy
by Perez, Ricardo 19 Apr '13

19 Apr '13

Dear all, I get the following errors when starting galaxy, is there any way to fix this. Error loading display application (./display_applications/genetrack.xml): [Errno 2] No such file or directory: './display_applications/genetrack.xml' Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 498, in load_display_applications display_app = DisplayApplication.from_file( config_path, self ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/application.py", line 163, in from_file return cls.from_elem( parse_xml( filename ).getroot(), datatypes_registry ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", line 143, in parse_xml tree = ElementTree.parse(fname) File "/usr/local/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/usr/local/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") Error loading display application (./display_applications/genetrack.xml): [Errno 2] No such file or directory: './display_applications/genetrack.xml' Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 498, in load_display_applications display_app = DisplayApplication.from_file( config_path, self ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/display_applications/application.py", line 163, in from_file return cls.from_elem( parse_xml( filename ).getroot(), datatypes_registry ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", line 143, in parse_xml tree = ElementTree.parse(fname) File "/usr/local/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/usr/local/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") Best, --Ricardo Perez

1 0

ToolShed functional tests bug
by Peter Cock 19 Apr '13

19 Apr '13

Hi all, I've been looking at the new functional test results now being shown on the tool shed, and found a bug/glitch: 1. Goto http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp 2. Click on top right menu "Repository Actions", "View functional test results" menu item 3. Notice two failed tests for htseq_count (which is not in this repo) This was for 12:6753a9261390 of this repo. Peter

3 3

TestToolShed out of date? (not running unit tests)
by Peter Cock 19 Apr '13

19 Apr '13

Hi all, I'd like to be able to experiment with the new automated testing recently added to the main Tool Shed http://toolshed.g2.bx.psu.edu/ by first trying my updates on the Test Tool Shed (to make sure my tests will work before making the tool update public). It appears that the testing features are not available on the Test Tool Shed, http://testtoolshed.g2.bx.psu.edu/ Is this correct? Is this intentional? Thanks, Peter P.S. This would be less urgent if I could solve this issue which is blocking my local testing: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014123.html

3 11

Re: [galaxy-dev] [galaxy-user] Question about scripting
by Nate Coraor 19 Apr '13

19 Apr '13

Hello, I've moved this over to the galaxy-dev list since it concerns a local installation. You'll want to use the 'condor' job running plugin, which is provided with the Galaxy distribution. Have a look at the job_conf.xml.sample_advanced file to see how this would be configured. A very basic job_conf.xml that should submit all jobs to HTCondor: <?xml version="1.0"?> <job_conf> <plugins workers="4"> <plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner" /> </plugins> <destinations default="condor"> <destination id="condor" runner="condor"/> </destinations> </job_conf> --nate On Apr 19, 2013, at 8:32 AM, leconte wrote: > hello, > > My subject is at the border between Galaxy and Condor > > I'found method to use Galaxy with condor cluster solution. > > In fact I have wrote a script when Galaxy call clustalw2 , it calls in fact a script called clustalw2. > > All run exactly as I want excepting ... but Galaxy get in error and said there was a problem > > > I turn my script to debug mode and it puts followings output > > + prog=/usr/bin/clustalw2 > + err=0 > + infile=-infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat > + outfile=-outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat > + outorder=-OUTORDER=ALIGNED > + seqnos=-SEQNOS=OFF > + typeseq=-TYPE=DNA > + var6= > + var7= > + var8= > + var9= > + echo > > ++ echo -infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat > ++ sed -e s/-infile=// > + infile2=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat > ++ sed -e 's/galaxy_dataset_$.*$\.dat//' > ++ sed -e s/-outfile=// > ++ echo -outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat > + WorkingDir=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/ > + mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/ > + mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/ > + mkdir -p /home/galaxy/galaxy-dist/database/job_working_directory/000/46/ > + [[ -n /usr/bin/clustalw2==clustalw2 ]] > + echo 'universe = vanilla' > + echo 'executable = /usr/bin/mpirun ' > + echo 'arguments = "-np 10 -hostfile /home/galaxy/hostfile /usr/bin/clustalw-mpi -infile=/home/galaxy/galaxy-dist/database/files/000/dataset_11.dat -outfile=/home/galaxy/galaxy-dist/database/job_working_directory/000/46/galaxy_dataset_80.dat -OUTORDER=ALIGNED -SEQNOS=OFF -TYPE=DNA "' > + echo 'Getenv = True > machine_count = 1 > transfer_input_files = /home/galaxy/hostfile,/usr/bin/clustalw-mpi, /home/galaxy/galaxy-dist/database/files/000/dataset_11.dat > should_transfer_files = yes > Log = /home/galaxy/logs/normal.$(cluster).$(process).log > Output = /home/galaxy/logs/normal.$(cluster).$(process).out > Error = /home/galaxy/logs/normal.$(cluster).$(process).error > notification = never > queue 1' > + /usr/bin/condor_submit /home/galaxy/submit > Submitting job(s). > 1 job(s) submitted to cluster 179. > > > > > I suspect the method I use, doesn't return properly information to galaxy ... but I don't know what I can do ... > > Any Idea ... > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/

1 0

Set upload from EBI SRA tool in a local instance of Galaxy
by Sarah Maman 19 Apr '13

19 Apr '13

Hello, When we search some sequences on NCBI thanks to the EBI SRA tool in a local instance of Galaxy, you can click on "galaxy" to upload data found. Except that our file is sent on Galaxy main and not in our local instance. Could you please tell me how to set my local instance of Galaxy ? Thank you in advance, Sarah -- --*-- Sarah Maman INRA - LGC - SIGENAE http://www.sigenae.org/ Chemin de Borde-Rouge - Auzeville - BP 52627 31326 Castanet-Tolosan cedex - FRANCE Tel: +33(0)5.61.28.57.08 Fax: +33(0)5.61.28.57.53 --*--

1 0

user names with authentication
by Jillian Rowe 19 Apr '13

19 Apr '13

Hello, I have a galaxy instance authentication using LDAP. The authentication works fine, and I would like to enable the user upload option. I would like to have: user_library_import_dir = /a/path and for galaxy to look for users with just the username. So if I log in with auser(a)example.com I want for galaxy to look in /a/path/auser instead of /a/path/auser(a)example.com Is there anywhere I can set this? Best, Jillian

3 4

External Display Applications
by John Chilton 18 Apr '13

18 Apr '13

This message is a continuation of a conversation from IRC, but it is too long for that medium. This is probably only directly meant for 3 or 4 people, but I sending this out wide and attaching the IRC conversation from yesterday because setting up and maintaining third party display applications is very confusing and difficult (JJ and I have spent way more time than we would care to admit trying to get these things to work and keep them working) and so the more search results that come up when you Google these things the better. My follow ups: 1) When that remote_user check was being skipped yesterday, my setup should have looked like one without external auth and it did not work. Makes me think something like IGV probably doesn't work against central in the non-remote user case right now. Just conjecture but you may want to test it. 2) Based on dblank's comments I am starting to understand the difference between /dataset/display_application and /display_application. It is still confusing that they are so close. I would recommend changing one or the other routes for clarity. 3) I think it is important to have different URLs for when the URL is meant for consumption by the web browser (when the expectation is that the user will have session information) and for when it will be consumed by a Java application (IGV) or third party web app (ProtVis, etc...) which will not have the users session and the displayonly REMOTE_USER hack must be employed. It would cleanup proxy configuration and give admins a clearer sense of what traffic is coming from where. Combining 2 + 3) maybe keep dataset/display_application as is but change /display_application to /prettified_browser_access /prettified_third_party_access 4) The displayonly REMOTE_USER hack is no longer on the wiki but it is still needed I think (right?). Why was it removed? 5) A long term goal, maybe should be getting away from the displayonly REMOTE_USER hack altogether. For /prettified_third_party_access routes Galaxy could setup a dummy user internally or just not require a user be set at all. Thanks, -John The IRC conversation: (09:04:08 AM) jmchilton1: Ummm... are you guys no longer redirecting to /display_application URLs, are these all going to /dataset? Any ideas how is that going to work with REMOTE_USER? (09:04:11 AM) dannon: Ahh, ok, so I did understand the error. This request won't (always) have a session. (09:13:15 AM) dannon: Ok, will have a fix out for this in a second. (09:22:55 AM) guerler left the room (quit: Quit: guerler). (09:23:49 AM) dannon: And, I'm not informed on display_application urls, Dan or Carl would be the best bet there. (09:23:52 AM) jmchilton1: Looking through my logs I guess that last question doesn't make sense. (09:24:06 AM) dannon: Made enough sense for me to say "NOT IT!" :) (09:24:36 AM) dannon: Just testing my fix to make sure it doesn't break anything else. (09:24:55 AM) jmchilton1: :) I think the problem is related to the session stuff though. display_application/blah -> 302 to /dataset/blah use to work and now redirects the same but the display application doesn't have access to the data. (09:25:25 AM) dannon: Ahh, that could be the case of the dataset api is relying on session being available. (09:25:46 AM) jmchilton1: It wasn't the API though, it was the normal controller. (09:26:27 AM) dannon: Ok, so how do I test this? (09:26:38 AM) jmchilton1: I have no clue. (09:26:52 AM) jmchilton1: I wish I understood how it was working in the first place (09:27:18 AM) dannon: Me too, I've never touched it. (09:29:07 AM) jmchilton1: It is all complicated greatly by the fact that nginx/apache need to be hacked for the display applications. Hmmm... I will keep poking at this and let you know if I discover anything. (09:29:27 AM) dannon: Ok, sounds good. In the meanwhile, do you have a script that just uploads a simple file via library api? (09:29:43 AM) dannon: Trying to test it and I remember how much of a pain this was now. (09:30:15 AM) dannon: Actually, nm, I have one. wtach folder will do it (09:30:20 AM) jmchilton1: I do not have a script. (09:30:24 AM) jmchilton1: Great. (09:40:50 AM) dannon: k, seems to work for me now, let me know if that doesn't fix it. (10:20:13 AM) guerler [~user(a)wireless-aca-ndb-a.nat.emory.edu] entered the room. (10:58:56 AM) clements [~clements(a)71-220-230-96.eugn.qwest.net] entered the room. (11:08:23 AM) botton [~willie(a)router.isis.poly.edu] entered the room. (11:08:23 AM) botton left the room. (11:26:13 AM) ceberhard [~carleberh(a)107.194.88.162] entered the room. (12:12:41 PM) jmchilton1: http://pastebin.com/07EikX6q (12:12:43 PM) mrscribe: Title: AttributeError: 'NoneType' object has no attribute 'deleted' - Pastebin.com (at pastebin.com) (12:13:41 PM) jmchilton1: dannon: It looks like you broke my display application :). (12:13:47 PM) dannon: Aww. (12:14:34 PM) jmchilton1: Do I just add another check that session has user before checking if user is deleted? (12:15:14 PM) dannon: I need to see how display applications work and why session isn't being associated. (12:15:17 PM) dannon: But maybe. (12:15:34 PM) jmchilton1: The display action doesn't act as the user, it doesn't have that users session information (12:15:54 PM) jmchilton1: That might also be nonsense (12:16:25 PM) jmchilton1: There is this hack you need to do though to set a display only user at the proxy level for display application accesses. (12:16:44 PM) jmchilton1: I will just try that hack for now and let you know how it goes. (12:16:45 PM) dannon: Ahh (12:16:58 PM) dannon: So it was a hack to get them to work before, and I broke the hack? (12:17:30 PM) dannon: If so, I feel less bad now, but we should still be able to get this to work in a non-hacky way. (12:17:34 PM) jmchilton1: I don't think MSI is the only one doing this. (12:17:48 PM) jmchilton1: Good luck with that :) (12:18:20 PM) jmchilton1: It is kind of a tough problem really, any way you slice it I think there is some complexity between the proxy and the display applications that is going to seem hacky. (12:18:42 PM) jmchilton1: Would love to be proven wrong though, this has really been a large headache for us over the years :). (12:19:20 PM) dannon: If you can explain how it worked before, I can probably at least translate that, just let me know what I can do. (12:19:39 PM) jmchilton1: Here, let me see if I can find the wiki page about it. (12:21:09 PM) jmchilton1: Maybe this is an MSI hack, I am not seeing anything about it on the wiki page. (12:21:59 PM) jmchilton1: I will keep thinking about this and send you documentation if I can find some. (12:22:13 PM) dannon: Ok, awesome. If it's broken, we can fix it, so just let me know. (12:22:20 PM) dannon: We have the technology! (12:27:42 PM) natefoo: pretty sure i had that in the apache page. (12:28:05 PM) jmchilton1: http://gmod.827538.n3.nabble.com/galaxy-dev-ldap-integration-td839409.html (12:28:27 PM) jmchilton1: Ry4an descirbe the displayonly hack somewhere in that thread. (12:32:31 PM) jmchilton1: Also I am noticing a bunch of new histories are being created automatically now. Oh how I wish I would have gotten that last merge right, I am tired of tracking central in production :(. (12:33:10 PM) dannon: Under what circumstances are new histories being created? (12:35:21 PM) jmchilton1: I don't know, it is related to the display application viewing though I think. (12:36:26 PM) dannon: Argh, ok. Sounds like it's time for me to learn how display applications should really work. (12:39:15 PM) jmchilton1: Wait, the histories was probably my mistake. (12:39:45 PM) jmchilton1: dannon: I wouldn't spend to much time on this until we have ruled out John's incompetence. (12:40:05 PM) dannon: Haha, ok. I'm going to play with sqlalchemy for a little bit, then. (01:04:22 PM) jmchilton1: Okay I made some misjudgments with the history change and stuff. But I am 43% confident there was a change related to this stuff that broke display applications and it is not just me. (01:04:24 PM) jmchilton1: _ensure_logged_in_user is now redirecting /display_application/blah.... to /root, I don't think it could have been doing that before. (01:05:11 PM) jmchilton1: I am confident those requests are being redicted, it is an assumption that _ensure_logged_in_user is the method redirecting (01:06:00 PM) dannon: Ok, let me look again at what might have changed that redirect. (01:07:04 PM) jmchilton1: Seems like according to blame, nothing in that function seems like it changed, I don't think _ensure_logged_in_user was called for these requests previously (01:07:14 PM) jmchilton1: I could be wrong again, that is just my first impression. (01:14:03 PM) dannon: Except, if I understand the displayonly user hack, it does create a galaxy_session and would have galaxy_session.user, so wouldn't redirect. (01:14:35 PM) dannon: So the change is that somehow, now, it isn't creating/associating that user and/or creating a valid session. (01:14:55 PM) jmchilton1: Is there a debug statement I can add to figure out which it is? (01:16:11 PM) jmchilton1: I don't understand how the REMOTE_USER is still set after the next request. (01:16:26 PM) dannon: Sure, just log the value of self.environ['HTTP_REMOTE_USER'], self.galaxy_session and galaxy_session.user in _ensure_logged_in_user (01:16:27 PM) jmchilton1: That is irrespecitve of these changes though. (01:19:21 PM) dannon: This would be easier to work with if I had a good test setup with remote user here, working on that now. (01:24:43 PM) jmchilton1: There has to be a second non-dannon bug conflating this, because the display application is attempting to download a /dataset/display_application URL not a /display_application url. When I manually attempt to download the /display_application url though I get a remote user in env and session, but user is None. (01:27:46 PM) dannon: I don't know if I feel better or worse about that. (01:27:59 PM) jmchilton1: Also, it is the same controller (01:28:21 PM) jmchilton1: The distinction must just be so that you can set remote_user on the /display_application version (01:28:43 PM) jmchilton1: I can see why that might change accidentally, it is not too obvious (01:42:02 PM) jmchilton1: Damnit! I am coming back around to thinking this is my own incompetence again. dannon, you should probably just pretend like you are not around for the rest of the day and see I have figured this out by tomorrow morning. (01:43:04 PM) dannon: I can do that, but then if it was my fault I'm going to feel bad. (01:43:25 PM) jmchilton1: Better than you wasting your day when it was my fault. (01:44:58 PM) jmchilton1: I should say more of your day :). (01:46:14 PM) dannon: Heh, ok, works for me. I'm going to do other things for a while. (02:27:32 PM) acu_ [~acu(a)24-159-215-150.static.roch.mn.charter.com] entered the room. (02:33:17 PM) natefoo left the room (quit: Read error: Operation timed out). (02:33:34 PM) natefoo [~nate(a)victory.bx.psu.edu] entered the room. (02:43:01 PM) acu_: can tool panel be customized differently for each user , that is each user would see different set of tools? (03:07:31 PM) bag [~bag(a)HSI-KBW-046-005-177-036.hsi8.kabel-badenwuerttemberg.de] entered the room. (03:12:03 PM) acu_ left the room (quit: Quit: Leaving). (03:12:16 PM) acu_ [~acu(a)24-159-215-150.static.roch.mn.charter.com] entered the room. (03:55:33 PM) jmchilton1: Is it possible that the url_for on line 43 of datatypes/display_applications/application.py use to generate a /display_application url but the change to mapper.explicit means it now generates a dataset/display_application url? The magic /display_application url is on line 57 of buildapp.py? (03:56:39 PM) jmchilton1: If so is there a clear way to force the /display_application version of the URL? (03:59:36 PM) ceberhard: jmchilton1: there are two styles of display applications. The newer version (AFAIK) is the only one of the two that produce the shorter, cleaner urls. (03:59:54 PM) jmchilton1: Is this not the newer style? (04:00:22 PM) ceberhard: Checking it out now. (04:06:22 PM) ceberhard: You're seeing the longer, verbose urls on these links? (04:06:51 PM) jmchilton1: Yes, I think so. (04:07:31 PM) jmchilton1: I have an interesting setup, so I worry it might be a problem on my end. But the goal is for this code to produce the shorter link? (04:09:04 PM) ceberhard: Yes. It successfully does this in web.base.controller (for the api) by removing the absolute slash before 'dataset'. It may be that a better fix would be to change the mapper to accept the absolute url instead. (04:09:10 PM) ceberhard: Let me try that. (04:11:24 PM) dannon: The leading slash disables route memory (which (04:11:34 PM) dannon: is the default now) (04:11:55 PM) dannon: Well. I should say, it's the only option now, with the removal of mapper.explicit == False (04:12:39 PM) ceberhard: So the better fix would be to remove the redundant slash in application.py, then. (04:12:59 PM) dannon: Unless there's hidden magic somewhere I'm not aware of and it's actually doing something. Doesn't *hurt* anything, though. (04:13:28 PM) ceberhard: Seems to still work in the version used in the api. (04:13:44 PM) dannon: So using the slash (or not) changes the url? (04:13:47 PM) dannon: And, how so? (04:14:13 PM) dannon: Or are you just saying that the display application links are working in the API? (04:14:37 PM) ceberhard: (AFAIK) There's a mapper for display applications in buildapp that seems to create a cleaner url. (04:14:49 PM) ceberhard: Yes. The api versions work. (04:15:27 PM) ceberhard: The line 57 in buildapp John mentioned. (04:15:51 PM) ceberhard: Not a mapper - a route. (04:17:08 PM) ceberhard: I found in the API that when it's called with a leading slash on the controller it isn't found and generates a longer url. (04:18:19 PM) dannon: Ok, so route memory does still work. (04:18:48 PM) dannon: The slash disables it, so it doesn't find the pretty route, and goes oldschool with controller/action/etc (04:19:00 PM) ceberhard: Makes sense. (04:19:13 PM) dannon: So now the question is, should that longer link ever be used? (04:20:07 PM) acu_ left the room (quit: Quit: Leaving). (04:20:16 PM) dannon: Because, with the leading slash, it should be doing the same exact thing as it was before. (04:20:36 PM) ceberhard: dblank: that may be a better question for you. (04:21:22 PM) dblank: fwiw: I had to fix the stderr/stdout link generation recently due to the disabling of route memory https://bitbucket.org/galaxy/galaxy-central/commits/2cfc5c8223ef (04:21:24 PM) mrscribe: Title: galaxy / galaxy-central / commit / 2cfc5c8223ef Bitbucket (at bitbucket.org) (04:21:55 PM) dannon: dblank: Yep, that was expected, and why this went in after the -dist :) (04:22:06 PM) dannon: We're going to find things like that here and there. (04:23:23 PM) dblank: indeed, just mentioning it since disable it did, have an effect. the leading slash was used previously to disable it, if its not helping now, what does removing it do? (04:23:51 PM) dannon: Well, I think carl was saying it did still change things? (04:24:35 PM) ceberhard: Haven't tested the difference in a while. I removed it to get the new-style links to clean up a while back. (04:24:36 PM) dannon: In any event, previously that display application url_for, starting with /dataset, was intentionally disabling route memory to get /dataset/display_application/blahblah urls, right? (04:25:39 PM) dblank: urls used to not have the /dataset/ part, just start with /display_application (04:26:35 PM) dannon: So is get_display_url in DisplayApplication not what's generating these? It hasn't been touched since 6714. (04:27:08 PM) jmchilton1: I actually have ruled that line out as where the link is coming from, I cannot find where the link for my display application is being generated. (04:27:18 PM) ceberhard: Just checked: adding in the API the slash produces the verbose url. (04:27:35 PM) ceberhard: The api uses a different version of that call. (04:27:48 PM) ceberhard: It was added to get around the two transactions/mappers. (04:28:13 PM) ceberhard: Are you seeing the link in the history panel, John, or somewhere else? (04:28:44 PM) jmchilton1: The remote display application is getting the link starting with /dataset/..., my hunch is this use to not happen (04:28:50 PM) dannon: ceberhard: Where's the other version of url_for? (04:29:05 PM) ceberhard: line 401, base.controller (04:29:25 PM) dblank: the link in the history panel looks ok to me, but now when the user is actually sent to the external app, its using the verbose /dataset/display_app... link (04:29:43 PM) dblank: jmchilton1: true, did not used to happen (04:29:50 PM) jmchilton1: :) (04:30:06 PM) jmchilton1: Progress? (04:31:40 PM) jmchilton1: I have to go catch a bus, but thanks for all the help guys. (04:32:26 PM) dannon: Anytime. So when you say "now when the user is actually sent to the external app", where does that happen? (04:32:57 PM) dblank: we'll have to figure this out though, since a lot of displays will be broken with the 'verbose' form, since links won't be generated to look like filenames with extensions, etc (04:33:41 PM) dblank: dannon: you click on the link in the history panel, then it goes off and generates the additional info, like viewport, format conversions etc (04:34:25 PM) dblank: when the app is ready, the user is redirected to the final destination (04:34:39 PM) dblank: the link in the history is ok, but the links afterwards are incorrect. (04:34:40 PM) dannon: Got it. So the url generated for the redirect is broken. (04:35:20 PM) dannon: ceberhard: You should be able to get rid of the extra url_for now, I think. It should do to exact same thing as routes.util.url_for used elsewhere. (04:35:54 PM) ceberhard: Yep. Understood. (04:36:21 PM) dblank: not just in the redirect, but also in the content that is generated (04:37:22 PM) dblank: so e.g you have http://localhost:8080/display_application/a43779105051f6e9/ucsc_vcf/main/ which has http://localhost:8080/display_application/a43779105051f6e9/ucsc_vcf/main/No…, which creates a track with content: track type="vcfTabix" name="BAM Coverage on data 1" bigDataUrl="http://localhost:8080/dataset/display_application?app_action=data&user_id=6… (04:38:21 PM) guerler left the room (quit: Ping timeout: 245 seconds). (04:38:22 PM) dblank: the /track link that is provided to ucsc via redirect in this case is being sent 'verbose', and the content inside of the track is also verbose (04:40:17 PM) the_cull left the room (quit: Read error: Operation timed out). (04:40:48 PM) ceberhard: Removing the slash before 'dataset' in application.py, line 43 seems to provide the redirect url more cleanly. Can you confirm, dblank? (04:43:25 PM) dannon: Right, that allows it to match the defined route instead of :controller/:action, but how did this possibly work before, since that hasn't changed forever?

1 1

Improving Administrative Data Clean Up (pgcleanup.py vs cleanup_datasets.py)
by Lance Parsons 18 Apr '13

18 Apr '13

I have been running a Galaxy server for our sequencing researchers for a while now and it's become increasingly successful. The biggest resource challenge for us has been, and continues to be disk space. As such, I'd like to implement some additional cleanup scripts. I thought I run a few questions by this list before I got too far into things. In general, I'm wondering how to implement updates/additions to the cleanup system that will be in line with the direction that the Galaxy project is headed. The pgcleanup.py script is the newest piece of code in this area (and even adds cleanup of exported histories, which are absent from the older cleanup scripts). Also, the pgcleanup.py script uses a "cleanup_event" table that I don't believe is used by the older cleanup_datasets.py script. However, the new pgcleanup.py script only works for Postgres, and worse, only for version 9.1+. I run my system on RedHat (CentOS) and thus we use version 8.4 of Postgres. Are there plans to support other databases or older versions of Postgres? I'd like to implement a script to delete (set the deleted flag) for certain datasets (e.g. raw data imported from our archive, for old, inactive users, etc.). I'm wondering if it would make sense to try and extend pgcleanup.py or cleanup_datasets.py. Or perhaps it would be best to just implement a separate script, though that seems like I'd have to re-implement a lot of boilerplate code for configuration reading, connections, logging, etc. Any tips on generally acceptable (supported) procedures for marking a dataset as deleted? Of course, I'll make any of the enhancements available (and would be happy to submit pull requests if there is interest). -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

2 4