October 2012 - galaxy-dev - lists.galaxyproject.org

High CPU usage of paster.py python process (daemon)
by Sebasian Sosnik 24 Oct '12

24 Oct '12

Hello, Recently I started 2 local galaxy servers on Linux PC for internal use. I noticed that the main WSGI process uses a lot of CPU time - see the output of ps command below: $ ps aux | head -1 & ps aux | grep wsgi [1] 5293 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND galaxy 26685 59.6 0.3 457332 125224 ? Sl Sep14 34447:36 python ./scripts/paster.py serve universe_wsgi.ini --daemon galaxy 19346 38.9 1.9 1098388 634772 ? Sl Oct17 4451:59 python ./scripts/paster.py serve universe_wsgi.ini --daemon >From this output, the first galaxy process used 59% of the CPU time in the recent month, it is 34447 minutes! The second process used 39% in the recent days. In the same month a tomcat web application used 10 minutes of CPU time, which is 0.0%. I have to say, that galaxy was accessed very, very seldom, a few times per week. Also the top command returns similar results: both galaxy server use more than 90% of CPU time, even if nobody is accessing galaxy. Is this normal? Do you have the same %CPU in ps command? Can this be configured/minimized somehow? Why is the CPU usage so high, if nobody is accessing galaxy? Is this a "feature" of WSGI? Sebastian ****************************************** Sebastian Sośnik FQS Poland 30-538 Krakow, ul. Parkowa 11 URL: <http://www.fqs.pl> http://www.fqs.pl Tel. (+48 12) 429 43 45 Mobile (+48) 608500161 Fax. (+48 12) 429 61 24 ******************************************

1 0

High CPU usage of paster.py python process (daemon)
by Sebasian Sosnik 24 Oct '12

24 Oct '12

Hi all, Recently I started 2 local galaxy servers on an internal Linux PC. I noticed that the main WSGI process uses a lot of CPU time - see the output of ps command below: $ ps aux | head -1 & ps aux | grep wsgi [1] 5293 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND galaxy 26685 59.6 0.3 457332 125224 ? Sl Sep14 34447:36 python ./scripts/paster.py serve universe_wsgi.ini --daemon galaxy 19346 38.9 1.9 1098388 634772 ? Sl Oct17 4451:59 python ./scripts/paster.py serve universe_wsgi.ini --daemon >From this output, the first galaxy process used 59% of the CPU time in the recent month, it is 34447 minutes! The second process used 39% in the recent days. Also the top command returns similar results: both galaxy server use more than 90% of CPU time, even if nobody is accessing galaxy. Is this normal? Do you have the same %CPU in ps command? Can this be configured/minimized somehow? Why is the CPU usage so high, if nothing happens? Is this a feature of WSGI? Sebastian ****************************************** Sebastian Sośnik FQS Poland 30-538 Krakow, ul. Parkowa 11 URL: http://www.fqs.pl Tel. (+48 12) 429 43 45 Mobile (+48) 608500161 Fax. (+48 12) 429 61 24 ******************************************

1 0

Enhancement: selecting input
by Joachim Jacob 24 Oct '12

24 Oct '12

Hi all, What if the input dataset selection for tools could be selected via clicking a dataset (or even more) directly in the history, instead of selecting from that good ol' dropdown box? It is just that I have to concatenate the contents of 24 files... I agree, in bash the command is typed below two seconds. But we're not (and my data is not) in bash, we have this wonderful Galaxy interface. Anybody triggered to develop this enhancement ? ;-) Cheers, Joachim. PS: I would have posted on Trello, but I am not allowed to do so. I understand that this is the way to propose enhancements. -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

3 4

Galaxy October 23, 2012 Distribution & News Brief
by Jennifer Jackson 24 Oct '12

24 Oct '12

*Galaxy Oct 23, 2012 Distribution & News Brief */http://wiki.g2.bx.psu.edu/2012_10_23_DistributionNewsBrief/* * * * *Highlights: * /http://wiki.g2.bx.psu.edu/News/ * *Important Security Fix <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_23#Compute_Tool_Security_Fix>* - upgrade *NOW*. * *New Trello Issue Board <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_23#New_Trello_Issue_Board>* - Learn how to submit, vote, and comment on tickets. * *http://wiki.g2.bx.psu.edu/Tool Shed <http://wiki.g2.bx.psu.edu/Tool%20Shed>* direct updates and changes in Galaxy <http://usegalaxy.org> improve metadata updating, repository management, and more control over tool config. * Updates to the *Visualization <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_23#Visualization_framework>* tool set cover *Trackster*, *Circster*, and new *Scatter Plot* tool. That's an example plot below. scatterplot-partialscreen <http://usegalaxy.org> * New *API <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_23#API>* fixes, tuneups, and brand-new methods to control libraries. * Multiple *Admin <http://wiki.g2.bx.psu.edu/DevNewsBriefs/2012_10_23#Source>* changes enable better access to user data to solve problems. *http://getgalaxy.org* *http://bitbucket.org/galaxy/galaxy-dist* new: $ hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist upgrade: $ hg pull -u -r 12fcd068b12e *Thanks for using Galaxy!* Jennifer Jackson <http://wiki.g2.bx.psu.edu/JenniferJackson> & the Galaxy Team <http://wiki.g2.bx.psu.edu/Galaxy%20Team>

1 0

Validation / validator questions
by Lukasse, Pieter 23 Oct '12

23 Oct '12

Hi, Is it possible to do validation of input fields depending on the value of OTHER input fields? Example 1 If field A is filled, then field B also should be filled. Example 2 Field B should be > field A etc Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

2 1

Trackster error when loading a gff file
by Yec'han Laizet 23 Oct '12

23 Oct '12

Hello, when I try to load a gff file in trackster on a galaxy instance (ubuntu server), I get the following error: Traceback (most recent call last): File "/home/pgtgal/galaxy-dist/lib/galaxy/datatypes/converters/interval_to_fli.py", line 91, in main() File "/home/pgtgal/galaxy-dist/lib/galaxy/datatypes/converters/interval_to_fli.py", line 30, in main for feature in read_unordered_gtf( open( in_fname, 'r' ) ): File "/home/pgtgal/galaxy-dist/lib/galaxy/datatypes/util/gff_util.py", line 375, in read_unordered_gtf transcript_id = line_attrs[ 'transcript_id' ] KeyError: 'transcript_id' I guess it's because not all the lines in the file have a transcript_id. Is it a specification in gff file format? I made it work by changing the line 65 of /home/pgtgal/galaxy-dist/lib/galaxy/datatypes/util/gff_util.py by this: transcript_id = line_attrs.get('transcript_id' , 0) You may want to change the default 0 value by something more pertinent... if it is really a bug and not a mistake from myself. Best regards Yec'han ================================================ Yec'han LAIZET Ingenieur Plateforme Genome Transcriptome Tel: 05 57 12 27 75 _________________________________ INRA-UMR BIOGECO 1202 Equipe Genetique 69 route d'Arcachon 33612 CESTAS ================================================

2 1

Compute on one row gives error
by Joachim Jacob 23 Oct '12

23 Oct '12

Hi all, I have the result of the count tool in one file, as just one number, in a tabular history item: 2160525 I want to divide that number by 3137161264: so I use 'Text Manipulation' to compute 'c1/3137161264'. (the naming of that section could be better IMHO). But I get an error: An error occurred running this job: /Missing or invalid 'columns' metadata value, click the pencil icon in the history item and select the Auto-detect option to correct it. This tool can only be used with tab-delimited data./ Is there an other way to compute this? Analysing this bug a bit further: -- adding more rows still reports this bug -- adding additional columns makes the bug disappear. Cheers, Joachim -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

1 0

Query regarding "tool_conf.xml" file editing
by Kumardeep Chaudhary 23 Oct '12

23 Oct '12

Hi all, I have installed galaxy on my local mac system and its been installed successfully. But when I am adding some additional links in the tool_conf.xml file (which is responsible for left menu); I am not getting it properly updated on my server (localhost:8080). Means whenever I make changes, those get changed in left menu; but those get refreshed after a long time (say half an hour); which makes it more difficult to trace and work on local system. PS: I do stop run.sh and re run again for webpage refreshing, even that does not properly refresh the web page interface of galaxy on my local. Is there some other file which needs to be manipulated for instant refreshing of the changes made in tool_conf.xml file ? Kindly help in this regard. Regards Kumardeep Research Fellow IMTECH Chandigarh India

2 1

Source code documentation
by Alex.Khassapov＠csiro.au 22 Oct '12

22 Oct '12

Hi, I wonder if there's some kind of documentation (reference) for the Galaxy source? At the moment I have a couple of questions for example. 1. How can I get the dataset object (in my Python wrapper) given the dataset name? 2. How can I access the job parameters (enered in the UI or 'hidden') in the Python code? In general, when I have this kind of questions, where do I look? -Alex

2 2

determination of errors
by David Hoover 22 Oct '12

22 Oct '12

How does Galaxy determine that a job has failed? Does it simply see if the STDERR is empty? What happens if an application normally outputs to STDERR? This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing. David Hoover Helix Systems Staff

4 3