- galaxy-commits - lists.galaxyproject.org

[galaxyproject/usegalaxy-playbook] 91a2ad: update Test
by GitHub 19 Dec '14

19 Dec '14

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 91a2ad0daa89339e718578cf80dc8740bd4a53ca https://github.com/galaxyproject/usegalaxy-playbook/commit/91a2ad0daa89339e… Author: martenson <cech.marten(a)gmail.com> Date: 2014-12-19 (Fri, 19 Dec 2014) Changed paths: M stage/group_vars/all.yml Log Message: ----------- update Test

1 0

commit/galaxy-central: 6 new changesets
by commits-noreply＠bitbucket.org 19 Dec '14

19 Dec '14

6 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/10661dcd25e0/ Changeset: 10661dcd25e0 Branch: tmcg/this-pull-request-is-to-extend-the-get-a-1418417648750 User: tmcg Date: 2014-12-12 20:54:33+00:00 Summary: This pull request is to extend the GET /api/jobs request/response. 1.) If the submitted api key is an admin's key AND the request has an included data_range_min or date_range_max value, the response will include jobs from all galaxy users. 2.) The response for (1) is extended to include a) external_id b) galaxy job id c) user email 3.) All users can submit a date range filter. 4.) Existing calls are unaffected by the change: https://galaxy-ng.msi.umn.edu/api/jobs?key=xxxxxxxxxxxxxxxxxxxxxxx [ { "create_time": "2014-12-01T19:23:37.270768", "exit_code": 0, "id": "cffdbbf7257b6526", "model_class": "Job", "state": "ok", "tool_id": "upload1", "update_time": "2014-12-01T19:24:24.608105" } ] 5.) Extended admin request: https://galaxy-ng.msi.umn.edu/api/jobs?date_range_min=2014-11-01?&key=xxxxx… ... { "create_time": "2014-11-11T15:59:19.909902", "exit_code": 0, "external_id": "31325.nokomis0015.msi.umn.edu", "id": "2003774561413b68", "job_id": 3733, "model_class": "Job", "state": "ok", "tool_id": "testtoolshed.g2.bx.psu.edu/repos/tmcgowan/fastqc/fastqc/0.60", "update_time": "2014-11-11T16:00:10.872804", "user_email": "mcgo0092+msistaff(a)msi.umn.edu" }, { "create_time": "2014-11-10T15:15:31.801047", "exit_code": 0, "external_id": "30299.nokomis0015.msi.umn.edu", "id": "e330bc56cb5095a7", "job_id": 3732, "model_class": "Job", "state": "ok", "tool_id": "testtoolshed.g2.bx.psu.edu/repos/jjohnson/snpeff/snpEff/4.0.1", "update_time": "2014-11-10T15:16:31.996148", "user_email": "jj+support(a)msi.umn.edu" } ... Affected #: 1 file diff -r efd0286e7629289c4c583898ebecb32646834c64 -r 10661dcd25e0888739a83f42b8424c0532efcaa7 lib/galaxy/webapps/galaxy/api/jobs.py --- a/lib/galaxy/webapps/galaxy/api/jobs.py +++ b/lib/galaxy/webapps/galaxy/api/jobs.py @@ -24,9 +24,12 @@ @expose_api def index( self, trans, **kwd ): """ - index( trans, state=None, tool_id=None, history_id=None ) + index( trans, state=None, tool_id=None, history_id=None, date_range_min=None, date_range_max=None ) * GET /api/jobs: return jobs for current user + + !! if user is admin and date_range_[min|max], then + return jobs for all galaxy users based on filtering - this is an extended service :type state: string or list :param state: limit listing of jobs to those that match one of the included states. If none, all are returned. @@ -36,6 +39,12 @@ :type tool_id: string or list :param tool_id: limit listing of jobs to those that match one of the included tool_ids. If none, all are returned. + :type date_range_min: string '2014-01-01' + :param date_range_min: limit the listing of jobs to those updated on or after requested date + + :type date_range_max: string '2014-12-31' + :param date_range_max: limit the listing of jobs to those updated on or before requested date + :type history_id: string :param history_id: limit listing of jobs to those that match the history_id. If none, all are returned. @@ -44,9 +53,15 @@ """ state = kwd.get( 'state', None ) - query = trans.sa_session.query( trans.app.model.Job ).filter( - trans.app.model.Job.user == trans.user - ) + if trans.user_is_admin() and (kwd.get('date_range_min', None) or kwd.get('date_range_max', None)): + is_extended_service = True + else: + is_extended_service = False + + if is_extended_service: + query = trans.sa_session.query( trans.app.model.Job ) + else: + query = trans.sa_session.query( trans.app.model.Job ).filter(trans.app.model.Job.user == trans.user) def build_and_apply_filters( query, objects, filter_func ): if objects is not None: @@ -63,6 +78,9 @@ query = build_and_apply_filters( query, kwd.get( 'tool_id', None ), lambda t: trans.app.model.Job.tool_id == t ) query = build_and_apply_filters( query, kwd.get( 'tool_id_like', None ), lambda t: trans.app.model.Job.tool_id.like(t) ) + + query = build_and_apply_filters( query, kwd.get( 'date_range_min', None ), lambda dmin: trans.app.model.Job.table.c.update_time >= dmin ) + query = build_and_apply_filters( query, kwd.get( 'date_range_max', None ), lambda dmax: trans.app.model.Job.table.c.update_time <= dmax ) history_id = kwd.get( 'history_id', None ) if history_id is not None: @@ -78,7 +96,24 @@ else: order_by = trans.app.model.Job.update_time.desc() for job in query.order_by( order_by ).all(): - out.append( self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) ) + j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) + + if is_extended_service: + def get_email_dict(trans): + user_email_dict = {} + user_query = trans.sa_session.query( trans.app.model.User ) + for user in user_query.all(): + u = user.to_dict() + user_email_dict[u['id']] = u['email'] + return user_email_dict + + user_emails = get_email_dict(trans) + j['external_id'] = job.job_runner_external_id + j['job_id'] = job.id + if job.user_id in user_emails: + j['user_email'] = user_emails[job.user_id] + + out.append(j) return out @expose_api https://bitbucket.org/galaxy/galaxy-central/commits/f09646961129/ Changeset: f09646961129 Branch: tmcg/this-pull-request-is-to-extend-the-get-a-1418417648750 User: tmcg Date: 2014-12-15 16:21:36+00:00 Summary: Edits based on reviewer comments. Removed redundant email query, removed job_id from response. Added user_details as param. Affected #: 1 file diff -r 10661dcd25e0888739a83f42b8424c0532efcaa7 -r f09646961129434a993983b81df3d7937b8ed72b lib/galaxy/webapps/galaxy/api/jobs.py --- a/lib/galaxy/webapps/galaxy/api/jobs.py +++ b/lib/galaxy/webapps/galaxy/api/jobs.py @@ -24,11 +24,11 @@ @expose_api def index( self, trans, **kwd ): """ - index( trans, state=None, tool_id=None, history_id=None, date_range_min=None, date_range_max=None ) + index( trans, state=None, tool_id=None, history_id=None, date_range_min=None, date_range_max=None, user_details=False ) * GET /api/jobs: return jobs for current user - !! if user is admin and date_range_[min|max], then + !! if user is admin and user_details is True, then return jobs for all galaxy users based on filtering - this is an extended service :type state: string or list @@ -39,6 +39,9 @@ :type tool_id: string or list :param tool_id: limit listing of jobs to those that match one of the included tool_ids. If none, all are returned. + :type user_details: boolean + :param user_details: if true, and requestor is an admin, will return external job id and user email. + :type date_range_min: string '2014-01-01' :param date_range_min: limit the listing of jobs to those updated on or after requested date @@ -51,9 +54,8 @@ :rtype: list :returns: list of dictionaries containing summary job information """ - state = kwd.get( 'state', None ) - if trans.user_is_admin() and (kwd.get('date_range_min', None) or kwd.get('date_range_max', None)): + if trans.user_is_admin() and kwd.get('user_details', False): is_extended_service = True else: is_extended_service = False @@ -96,24 +98,12 @@ else: order_by = trans.app.model.Job.update_time.desc() for job in query.order_by( order_by ).all(): - j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) - + j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) if is_extended_service: - def get_email_dict(trans): - user_email_dict = {} - user_query = trans.sa_session.query( trans.app.model.User ) - for user in user_query.all(): - u = user.to_dict() - user_email_dict[u['id']] = u['email'] - return user_email_dict - - user_emails = get_email_dict(trans) j['external_id'] = job.job_runner_external_id - j['job_id'] = job.id - if job.user_id in user_emails: - j['user_email'] = user_emails[job.user_id] - + j['user_email'] = job.user.email out.append(j) + return out @expose_api https://bitbucket.org/galaxy/galaxy-central/commits/f8331619132e/ Changeset: f8331619132e User: jmchilton Date: 2014-12-19 02:49:03+00:00 Summary: Merge pull request #610. Affected #: 1 file diff -r 114b0d3764700a69e87c39f07a1c925a76814261 -r f8331619132ead73b700c337f2fa2610ba92993f lib/galaxy/webapps/galaxy/api/jobs.py --- a/lib/galaxy/webapps/galaxy/api/jobs.py +++ b/lib/galaxy/webapps/galaxy/api/jobs.py @@ -24,9 +24,12 @@ @expose_api def index( self, trans, **kwd ): """ - index( trans, state=None, tool_id=None, history_id=None ) + index( trans, state=None, tool_id=None, history_id=None, date_range_min=None, date_range_max=None, user_details=False ) * GET /api/jobs: return jobs for current user + + !! if user is admin and user_details is True, then + return jobs for all galaxy users based on filtering - this is an extended service :type state: string or list :param state: limit listing of jobs to those that match one of the included states. If none, all are returned. @@ -36,17 +39,31 @@ :type tool_id: string or list :param tool_id: limit listing of jobs to those that match one of the included tool_ids. If none, all are returned. + :type user_details: boolean + :param user_details: if true, and requestor is an admin, will return external job id and user email. + + :type date_range_min: string '2014-01-01' + :param date_range_min: limit the listing of jobs to those updated on or after requested date + + :type date_range_max: string '2014-12-31' + :param date_range_max: limit the listing of jobs to those updated on or before requested date + :type history_id: string :param history_id: limit listing of jobs to those that match the history_id. If none, all are returned. :rtype: list :returns: list of dictionaries containing summary job information """ - state = kwd.get( 'state', None ) - query = trans.sa_session.query( trans.app.model.Job ).filter( - trans.app.model.Job.user == trans.user - ) + if trans.user_is_admin() and kwd.get('user_details', False): + is_extended_service = True + else: + is_extended_service = False + + if is_extended_service: + query = trans.sa_session.query( trans.app.model.Job ) + else: + query = trans.sa_session.query( trans.app.model.Job ).filter(trans.app.model.Job.user == trans.user) def build_and_apply_filters( query, objects, filter_func ): if objects is not None: @@ -63,6 +80,9 @@ query = build_and_apply_filters( query, kwd.get( 'tool_id', None ), lambda t: trans.app.model.Job.tool_id == t ) query = build_and_apply_filters( query, kwd.get( 'tool_id_like', None ), lambda t: trans.app.model.Job.tool_id.like(t) ) + + query = build_and_apply_filters( query, kwd.get( 'date_range_min', None ), lambda dmin: trans.app.model.Job.table.c.update_time >= dmin ) + query = build_and_apply_filters( query, kwd.get( 'date_range_max', None ), lambda dmax: trans.app.model.Job.table.c.update_time <= dmax ) history_id = kwd.get( 'history_id', None ) if history_id is not None: @@ -78,7 +98,12 @@ else: order_by = trans.app.model.Job.update_time.desc() for job in query.order_by( order_by ).all(): - out.append( self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) ) + j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) + if is_extended_service: + j['external_id'] = job.job_runner_external_id + j['user_email'] = job.user.email + out.append(j) + return out @expose_api https://bitbucket.org/galaxy/galaxy-central/commits/8d37d708a96c/ Changeset: 8d37d708a96c Branch: tmcg/this-pull-request-is-to-extend-the-get-a-1418417648750 User: jmchilton Date: 2014-12-19 02:49:33+00:00 Summary: Close branch. Affected #: 0 files https://bitbucket.org/galaxy/galaxy-central/commits/943bc5a4c60a/ Changeset: 943bc5a4c60a User: jmchilton Date: 2014-12-19 02:50:15+00:00 Summary: PEP-8 fix for api/jobs.py. Affected #: 1 file diff -r f8331619132ead73b700c337f2fa2610ba92993f -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 lib/galaxy/webapps/galaxy/api/jobs.py --- a/lib/galaxy/webapps/galaxy/api/jobs.py +++ b/lib/galaxy/webapps/galaxy/api/jobs.py @@ -27,7 +27,7 @@ index( trans, state=None, tool_id=None, history_id=None, date_range_min=None, date_range_max=None, user_details=False ) * GET /api/jobs: return jobs for current user - + !! if user is admin and user_details is True, then return jobs for all galaxy users based on filtering - this is an extended service @@ -41,10 +41,10 @@ :type user_details: boolean :param user_details: if true, and requestor is an admin, will return external job id and user email. - + :type date_range_min: string '2014-01-01' - :param date_range_min: limit the listing of jobs to those updated on or after requested date - + :param date_range_min: limit the listing of jobs to those updated on or after requested date + :type date_range_max: string '2014-12-31' :param date_range_max: limit the listing of jobs to those updated on or before requested date @@ -59,7 +59,7 @@ is_extended_service = True else: is_extended_service = False - + if is_extended_service: query = trans.sa_session.query( trans.app.model.Job ) else: @@ -80,7 +80,7 @@ query = build_and_apply_filters( query, kwd.get( 'tool_id', None ), lambda t: trans.app.model.Job.tool_id == t ) query = build_and_apply_filters( query, kwd.get( 'tool_id_like', None ), lambda t: trans.app.model.Job.tool_id.like(t) ) - + query = build_and_apply_filters( query, kwd.get( 'date_range_min', None ), lambda dmin: trans.app.model.Job.table.c.update_time >= dmin ) query = build_and_apply_filters( query, kwd.get( 'date_range_max', None ), lambda dmax: trans.app.model.Job.table.c.update_time <= dmax ) @@ -98,12 +98,12 @@ else: order_by = trans.app.model.Job.update_time.desc() for job in query.order_by( order_by ).all(): - j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) + j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) if is_extended_service: j['external_id'] = job.job_runner_external_id j['user_email'] = job.user.email out.append(j) - + return out @expose_api https://bitbucket.org/galaxy/galaxy-central/commits/815d38c48a56/ Changeset: 815d38c48a56 User: jmchilton Date: 2014-12-19 02:50:15+00:00 Summary: Increase consistency between job index and show. Add tests for new date range and history filtering as well as to ensure only admins get to see external_id and command_line and that users cannot see each other's jobs. Always allow admins to views all jobs on index (instead of only when user_details is specified) and show. Add user_email and external_id to show (for admins) to bring it inline with index and add command_line to index (for admins) to bring it in line with show. Show still allow more details including job standard error and output as well as job metrics. Affected #: 6 files diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py +++ b/lib/galaxy/model/__init__.py @@ -520,9 +520,15 @@ dataset.blurb = 'deleted' dataset.peek = 'Job deleted' dataset.info = 'Job output deleted by user before job completed' - def to_dict( self, view='collection' ): + + def to_dict( self, view='collection', system_details=False ): rval = super( Job, self ).to_dict( view=view ) rval['tool_id'] = self.tool_id + if system_details: + # System level details that only admins should have. + rval['external_id'] = self.job_runner_external_id + rval['command_line'] = self.command_line + if view == 'element': param_dict = dict( [ ( p.name, p.value ) for p in self.parameters ] ) rval['params'] = param_dict diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 lib/galaxy/web/security/__init__.py --- a/lib/galaxy/web/security/__init__.py +++ b/lib/galaxy/web/security/__init__.py @@ -73,7 +73,7 @@ if not isinstance( rval, dict ): return rval for k, v in rval.items(): - if ( k == 'id' or k.endswith( '_id' ) ) and v is not None and k not in [ 'tool_id' ]: + if ( k == 'id' or k.endswith( '_id' ) ) and v is not None and k not in [ 'tool_id', 'external_id' ]: try: rval[ k ] = self.encode_id( v ) except Exception: diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 lib/galaxy/webapps/galaxy/api/jobs.py --- a/lib/galaxy/webapps/galaxy/api/jobs.py +++ b/lib/galaxy/webapps/galaxy/api/jobs.py @@ -55,12 +55,10 @@ :returns: list of dictionaries containing summary job information """ state = kwd.get( 'state', None ) - if trans.user_is_admin() and kwd.get('user_details', False): - is_extended_service = True - else: - is_extended_service = False + is_admin = trans.user_is_admin() + user_details = kwd.get('user_details', False) - if is_extended_service: + if is_admin: query = trans.sa_session.query( trans.app.model.Job ) else: query = trans.sa_session.query( trans.app.model.Job ).filter(trans.app.model.Job.user == trans.user) @@ -98,9 +96,9 @@ else: order_by = trans.app.model.Job.update_time.desc() for job in query.order_by( order_by ).all(): - j = self.encode_all_ids( trans, job.to_dict( 'collection' ), True ) - if is_extended_service: - j['external_id'] = job.job_runner_external_id + job_dict = job.to_dict( 'collection', system_details=is_admin ) + j = self.encode_all_ids( trans, job_dict, True ) + if user_details: j['user_email'] = job.user.email out.append(j) @@ -123,12 +121,13 @@ :returns: dictionary containing full description of job data """ job = self.__get_job( trans, id ) - job_dict = self.encode_all_ids( trans, job.to_dict( 'element' ), True ) + is_admin = trans.user_is_admin() + job_dict = self.encode_all_ids( trans, job.to_dict( 'element', system_details=is_admin ), True ) full_output = util.asbool( kwd.get( 'full', 'false' ) ) if full_output: job_dict.update( dict( stderr=job.stderr, stdout=job.stdout ) ) - if trans.user_is_admin(): - job_dict['command_line'] = job.command_line + if is_admin: + job_dict['user_email'] = job.user.email def metric_to_dict(metric): metric_name = metric.metric_name @@ -199,10 +198,16 @@ decoded_job_id = trans.security.decode_id( id ) except Exception: raise exceptions.MalformedId() - query = trans.sa_session.query( trans.app.model.Job ).filter( - trans.app.model.Job.user == trans.user, - trans.app.model.Job.id == decoded_job_id - ) + query = trans.sa_session.query( trans.app.model.Job ) + if trans.user_is_admin(): + query = query.filter( + trans.app.model.Job.id == decoded_job_id + ) + else: + query = query.filter( + trans.app.model.Job.user == trans.user, + trans.app.model.Job.id == decoded_job_id + ) job = query.first() if job is None: raise exceptions.ObjectNotFound() diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 test/api/test_jobs.py --- a/test/api/test_jobs.py +++ b/test/api/test_jobs.py @@ -1,4 +1,6 @@ +import datetime import json +import time from operator import itemgetter from base import api @@ -11,13 +13,18 @@ def test_index( self ): # Create HDA to ensure at least one job exists... self.__history_with_new_dataset() - jobs_response = self._get( "jobs" ) + jobs = self.__jobs_index() + assert "upload1" in map( itemgetter( "tool_id" ), jobs ) - self._assert_status_code_is( jobs_response, 200 ) + def test_system_details_admin_only( self ): + self.__history_with_new_dataset() + jobs = self.__jobs_index( admin=False ) + job = jobs[0] + self._assert_not_has_keys( job, "command_line", "external_id" ) - jobs = jobs_response.json() - assert isinstance( jobs, list ) - assert "upload1" in map( itemgetter( "tool_id" ), jobs ) + jobs = self.__jobs_index( admin=True ) + job = jobs[0] + self._assert_has_keys( job, "command_line", "external_id" ) def test_index_state_filter( self ): # Initial number of ok jobs @@ -31,6 +38,33 @@ new_count = len( self.__uploads_with_state( "ok" ) ) assert original_count < new_count + def test_index_date_filter( self ): + self.__history_with_new_dataset() + two_weeks_ago = (datetime.datetime.utcnow() - datetime.timedelta(7)).isoformat() + last_week = (datetime.datetime.utcnow() - datetime.timedelta(7)).isoformat() + next_week = (datetime.datetime.utcnow() + datetime.timedelta(7)).isoformat() + today = datetime.datetime.utcnow().isoformat() + tomorrow = (datetime.datetime.utcnow() + datetime.timedelta(1)).isoformat() + + jobs = self.__jobs_index( data={"date_range_min": today[0:10], "date_range_max": tomorrow[0:10]} ) + assert len( jobs ) > 0 + today_job_id = jobs[0]["id"] + + jobs = self.__jobs_index( data={"date_range_min": two_weeks_ago, "date_range_max": last_week} ) + assert today_job_id not in map(itemgetter("id"), jobs) + + jobs = self.__jobs_index( data={"date_range_min": last_week, "date_range_max": next_week} ) + assert today_job_id in map(itemgetter("id"), jobs) + + def test_index_history( self ): + history_id, _ = self.__history_with_new_dataset() + jobs = self.__jobs_index( data={"history_id": history_id} ) + assert len( jobs ) > 0 + + history_id = self._new_history() + jobs = self.__jobs_index( data={"history_id": history_id} ) + assert len( jobs ) == 0 + def test_index_multiple_states_filter( self ): # Initial number of ok jobs original_count = len( self.__uploads_with_state( "ok", "new" ) ) @@ -58,6 +92,22 @@ job_details = show_jobs_response.json() self._assert_has_key( job_details, 'id', 'state', 'exit_code', 'update_time', 'create_time' ) + def test_show_security( self ): + history_id, _ = self.__history_with_new_dataset() + jobs_response = self._get( "jobs", data={"history_id": history_id} ) + job = jobs_response.json()[ 0 ] + job_id = job[ "id" ] + + show_jobs_response = self._get( "jobs/%s" % job_id, admin=False ) + self._assert_not_has_keys( show_jobs_response.json(), "command_line", "external_id" ) + + with self._different_user(): + show_jobs_response = self._get( "jobs/%s" % job_id, admin=False ) + self._assert_status_code_is( show_jobs_response, 404 ) + + show_jobs_response = self._get( "jobs/%s" % job_id, admin=True ) + self._assert_has_keys( show_jobs_response.json(), "command_line", "external_id" ) + def test_search( self ): history_id, dataset_id = self.__history_with_ok_dataset() @@ -133,3 +183,10 @@ history_id, dataset_id = self.__history_with_new_dataset() self._wait_for_history( history_id, assert_ok=True ) return history_id, dataset_id + + def __jobs_index( self, **kwds ): + jobs_response = self._get( "jobs", **kwds ) + self._assert_status_code_is( jobs_response, 200 ) + jobs = jobs_response.json() + assert isinstance( jobs, list ) + return jobs diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 test/base/api.py --- a/test/base/api.py +++ b/test/base/api.py @@ -11,6 +11,7 @@ from .api_asserts import ( assert_status_code_is, assert_has_keys, + assert_not_has_keys, assert_error_code_is, ) @@ -83,6 +84,9 @@ def _assert_has_keys( self, response, *keys ): assert_has_keys( response, *keys ) + def _assert_not_has_keys( self, response, *keys ): + assert_not_has_keys( response, *keys ) + def _assert_error_code_is( self, response, error_code ): assert_error_code_is( response, error_code ) diff -r 943bc5a4c60a0706633660fad41fe9c71fdd8ca3 -r 815d38c48a5639d47eed92806ef4c196406f58f1 test/base/api_asserts.py --- a/test/base/api_asserts.py +++ b/test/base/api_asserts.py @@ -20,6 +20,11 @@ assert key in response, "Response [%s] does not contain key [%s]" % ( response, key ) +def assert_not_has_keys( response, *keys ): + for key in keys: + assert key not in response, "Response [%s] contains invalid key [%s]" % ( response, key ) + + def assert_error_code_is( response, error_code ): if hasattr( response, "json" ): response = response.json() Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Fix bug in 8d5089f2ba8183439a866026f3583c377f8bfa74.
by commits-noreply＠bitbucket.org 18 Dec '14

18 Dec '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/114b0d376470/ Changeset: 114b0d376470 User: jmchilton Date: 2014-12-18 15:21:42+00:00 Summary: Fix bug in 8d5089f2ba8183439a866026f3583c377f8bfa74. Affected #: 1 file diff -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 -r 114b0d3764700a69e87c39f07a1c925a76814261 lib/galaxy/workflow/run_request.py --- a/lib/galaxy/workflow/run_request.py +++ b/lib/galaxy/workflow/run_request.py @@ -56,13 +56,13 @@ possible_input_keys = [] for inputs_by_el in inputs_by.split("|"): if inputs_by_el == "step_id": - possible_input_keys = [str( step.id )] + possible_input_keys.append(str( step.id )) elif inputs_by_el == "step_index": - possible_input_keys = [str( step.order_index )] + possible_input_keys.append(str( step.order_index )) elif inputs_by_el == "step_uuid": - possible_input_keys = [str( step.uuid )] + possible_input_keys.append(str( step.uuid )) elif inputs_by_el == "name": - possible_input_keys = [step.tool_inputs.get( 'name', None )] + possible_input_keys.append(step.tool_inputs.get( 'name', None )) else: message = "Workflow cannot be run because unexpected inputs_by value specified." raise exceptions.MessageException( message ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Merged in jmchilton/galaxy-central-fork-1 (pull request #617)
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/5e5c0cf930bb/ Changeset: 5e5c0cf930bb User: jmchilton Date: 2014-12-17 20:47:54+00:00 Summary: Merged in jmchilton/galaxy-central-fork-1 (pull request #617) Allow tool tests to assert properties about command, standard output, and standard error. Affected #: 6 files diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/parser/xml.py --- a/lib/galaxy/tools/parser/xml.py +++ b/lib/galaxy/tools/parser/xml.py @@ -186,6 +186,11 @@ rval = dict( outputs=__parse_output_elems(test_elem), inputs=__parse_input_elems(test_elem, i), + command=__parse_assert_list_from_elem( test_elem.find("assert_command") ), + stdout=__parse_assert_list_from_elem( test_elem.find("assert_stdout") ), + stderr=__parse_assert_list_from_elem( test_elem.find("assert_stderr") ), + expect_exit_code=test_elem.get("expect_exit_code"), + expect_failure=string_as_bool(test_elem.get("expect_failure", False)), ) _copy_to_dict_if_present(test_elem, rval, ["interactor", "num_outputs"]) return rval @@ -222,6 +227,11 @@ return name, file, attributes +def __parse_command_elem( test_elem ): + assert_elem = test_elem.find("command") + return __parse_assert_list_from_elem( assert_elem ) + + def __parse_test_attributes( output_elem, attrib ): assert_list = __parse_assert_list( output_elem ) file = attrib.pop( 'file', None ) @@ -252,6 +262,10 @@ def __parse_assert_list( output_elem ): assert_elem = output_elem.find("assert_contents") + return __parse_assert_list_from_elem( assert_elem ) + + +def __parse_assert_list_from_elem( assert_elem ): assert_list = None def convert_elem(elem): diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/parser/yaml.py --- a/lib/galaxy/tools/parser/yaml.py +++ b/lib/galaxy/tools/parser/yaml.py @@ -152,27 +152,47 @@ attributes["metadata"] = {} # TODO assert_list = [] - for key, assertion in attributes.get("asserts", {}).iteritems(): - # TODO: not handling nested assertions correctly, - # not sure these are used though. - children = [] - if "children" in assertion: - children = assertion["children"] - del assertion["children"] - assert_dict = dict( - tag=key, - attributes=assertion, - children=children, - ) - assert_list.append(assert_dict) + assert_list = __to_test_assert_list( attributes.get("asserts", [] ) ) attributes["assert_list"] = assert_list - _ensure_has(attributes, defaults) test_dict["outputs"] = new_outputs + test_dict["command"] = __to_test_assert_list( test_dict.get( "command", [] ) ) + test_dict["stdout"] = __to_test_assert_list( test_dict.get( "stdout", [] ) ) + test_dict["stderr"] = __to_test_assert_list( test_dict.get( "stderr", [] ) ) + test_dict["expect_exit_code"] = test_dict.get( "expect_exit_code", None ) + test_dict["expect_failure"] = test_dict.get( "expect_exit_code", False ) return test_dict +def __to_test_assert_list(assertions): + def expand_dict_form(item): + key, value = item + new_value = value.copy() + new_value["that"] = key + return new_value + + if isinstance( assertions, dict ): + assertions = map(expand_dict_form, assertions.items() ) + + assert_list = [] + for assertion in assertions: + # TODO: not handling nested assertions correctly, + # not sure these are used though. + children = [] + if "children" in assertion: + children = assertion["children"] + del assertion["children"] + assert_dict = dict( + tag=assertion["that"], + attributes=assertion, + children=children, + ) + assert_list.append(assert_dict) + + return assert_list or None # XML variant is None if no assertions made + + class YamlPageSource(PageSource): def __init__(self, inputs_list): diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -127,6 +127,11 @@ if num_outputs: num_outputs = int( num_outputs ) self.num_outputs = num_outputs + self.command_line = test_dict.get("command", None) + self.stdout = test_dict.get("stdout", None) + self.stderr = test_dict.get("stderr", None) + self.expect_exit_code = test_dict.get("expect_exit_code", None) + self.expect_failure = test_dict.get("expect_failure", False) except Exception, e: self.error = True self.exception = e diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/test_toolbox.py --- a/test/functional/test_toolbox.py +++ b/test/functional/test_toolbox.py @@ -1,6 +1,7 @@ import new import sys from base.twilltestcase import TwillTestCase +from base.asserts import verify_assertions from base.interactor import build_interactor, stage_data_in_history, RunToolException from base.instrument import register_job_data from galaxy.tools import DataManagerTool @@ -92,6 +93,9 @@ raise Exception( "Test parse failure" ) def _verify_outputs( self, testdef, history, jobs, shed_tool_id, data_list, galaxy_interactor ): + assert len(jobs) == 1, "Test framework logic error, somehow tool test resulted in more than one job." + job = jobs[ 0 ] + maxseconds = testdef.maxseconds if testdef.num_outputs is not None: expected = testdef.num_outputs @@ -102,7 +106,33 @@ raise Exception( message ) found_exceptions = [] - job_stdio = None + if testdef.expect_failure: + if testdef.outputs: + raise Exception("Cannot specify outputs in a test expecting failure.") + + # Wait for the job to complete and register expections if the final + # status was not what test was expecting. + job_failed = False + try: + galaxy_interactor.wait_for_job( job[ 'id' ], history, maxseconds ) + except Exception as e: + job_failed = True + if not testdef.expect_failure: + found_exceptions.append(e) + + if not job_failed and testdef.expect_failure: + error = AssertionError("Expected job to fail but Galaxy indicated the job successfully completed.") + found_exceptions.append(error) + + job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) + + expect_exit_code = testdef.expect_exit_code + if expect_exit_code is not None: + exit_code = job_stdio["exit_code"] + if str(expect_exit_code) != str(exit_code): + error = AssertionError("Expected job to complete with exit code %s, found %s" % (expect_exit_code, exit_code)) + found_exceptions.append(error) + for output_index, output_tuple in enumerate(testdef.outputs): # Get the correct hid name, outfile, attributes = output_tuple @@ -123,14 +153,25 @@ except Exception as e: if not found_exceptions: # Only print this stuff out once. - for job in jobs: - job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) - for stream in ['stdout', 'stderr']: - if stream in job_stdio: - print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) + for stream in ['stdout', 'stderr']: + if stream in job_stdio: + print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) found_exceptions.append(e) - if job_stdio is None: - job_stdio = galaxy_interactor.get_job_stdio( jobs[0][ 'id' ] ) + + other_checks = { + "command_line": "Command produced by the job", + "stdout": "Standard output of the job", + "stderr": "Standard error of the job", + } + for what, description in other_checks.items(): + if getattr( testdef, what, None ) is not None: + try: + data = job_stdio[what] + verify_assertions( data, getattr( testdef, what ) ) + except AssertionError, err: + errmsg = '%s different than expected\n' % description + errmsg += str( err ) + found_exceptions.append( AssertionError( errmsg ) ) if found_exceptions: raise JobOutputsError(found_exceptions, job_stdio) diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/tools/job_properties.xml --- /dev/null +++ b/test/functional/tools/job_properties.xml @@ -0,0 +1,62 @@ +<tool id="job_properties" name="Test Job Properties"> + <command> + #if $thebool + echo "The bool is true"; + echo "The bool is really true" 1>&2; + echo "This is a line of text." > $out_file1 + #else + echo "The bool is not true"; + echo "The bool is very not true" 1>&2; + echo "This is a different line of text." > $out_file1; + sh -c "exit 2" + #end if + #if $failbool + ; sh -c "exit 127" + #end if + + </command> + <inputs> + <param name="thebool" type="boolean" label="The boolean property" /> + <param name="failbool" type="boolean" label="The failure property" checked="false" /> + </inputs> + <outputs> + <data name="out_file1" /> + </outputs> + <stdio> + <exit_code range="127" level="fatal" description="Failing exit code." /> + </stdio> + <tests> + <test expect_exit_code="0"> + <param name="thebool" value="true" /> + <output name="out_file1" file="simple_line.txt" /> + <assert_command> + <has_text text="really" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is really true" /> + </assert_stderr> + </test> + <test expect_exit_code="2"> + <param name="thebool" value="false" /> + <output name="out_file1" file="simple_line_alternative.txt" /> + <assert_command> + <has_text text="very not" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is not true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is very not true" /> + </assert_stderr> + </test> + <test expect_exit_coded="127" expect_failure="true"> + <param name="thebool" value="true" /> + <param name="failbool" value="true" /> + </test> + </tests> + <help> + </help> +</tool> diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/tools/samples_tool_conf.xml --- a/test/functional/tools/samples_tool_conf.xml +++ b/test/functional/tools/samples_tool_conf.xml @@ -13,6 +13,7 @@ <tool file="multi_output_assign_primary.xml" /><tool file="composite_output.xml" /><tool file="metadata.xml" /> + <tool file="job_properties.xml" /><tool file="gzipped_inputs.xml" /><tool file="output_order.xml" /><tool file="output_format.xml" /> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/322c44f7519f/ Changeset: 322c44f7519f User: jmchilton Date: 2014-12-17 17:34:39+00:00 Summary: Allow tool tests to assert properties about command, standard output, and standard error. See test/functional/tools/job_properties.xml for example. Provides full access to assertion based XML tags - tabular and XML based assertions probably not so interesting so these include - has_text, has_line, not_has_text, has_text_matching, has_line_matching. Affected #: 6 files diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 lib/galaxy/tools/parser/xml.py --- a/lib/galaxy/tools/parser/xml.py +++ b/lib/galaxy/tools/parser/xml.py @@ -186,6 +186,9 @@ rval = dict( outputs=__parse_output_elems(test_elem), inputs=__parse_input_elems(test_elem, i), + command=__parse_assert_list_from_elem( test_elem.find("assert_command") ), + stdout=__parse_assert_list_from_elem( test_elem.find("assert_stdout") ), + stderr=__parse_assert_list_from_elem( test_elem.find("assert_stderr") ), ) _copy_to_dict_if_present(test_elem, rval, ["interactor", "num_outputs"]) return rval @@ -222,6 +225,11 @@ return name, file, attributes +def __parse_command_elem( test_elem ): + assert_elem = test_elem.find("command") + return __parse_assert_list_from_elem( assert_elem ) + + def __parse_test_attributes( output_elem, attrib ): assert_list = __parse_assert_list( output_elem ) file = attrib.pop( 'file', None ) @@ -252,6 +260,10 @@ def __parse_assert_list( output_elem ): assert_elem = output_elem.find("assert_contents") + return __parse_assert_list_from_elem( assert_elem ) + + +def __parse_assert_list_from_elem( assert_elem ): assert_list = None def convert_elem(elem): diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 lib/galaxy/tools/parser/yaml.py --- a/lib/galaxy/tools/parser/yaml.py +++ b/lib/galaxy/tools/parser/yaml.py @@ -152,27 +152,45 @@ attributes["metadata"] = {} # TODO assert_list = [] - for key, assertion in attributes.get("asserts", {}).iteritems(): - # TODO: not handling nested assertions correctly, - # not sure these are used though. - children = [] - if "children" in assertion: - children = assertion["children"] - del assertion["children"] - assert_dict = dict( - tag=key, - attributes=assertion, - children=children, - ) - assert_list.append(assert_dict) + assert_list = __to_test_assert_list( attributes.get("asserts", [] ) ) attributes["assert_list"] = assert_list - _ensure_has(attributes, defaults) test_dict["outputs"] = new_outputs + test_dict["command"] = __to_test_assert_list( attributes.get( "command", [] ) ) + test_dict["stdout"] = __to_test_assert_list( attributes.get( "stdout", [] ) ) + test_dict["stderr"] = __to_test_assert_list( attributes.get( "stderr", [] ) ) return test_dict +def __to_test_assert_list(assertions): + def expand_dict_form(item): + key, value = item + new_value = value.copy() + new_value["that"] = key + return new_value + + if isinstance( assertions, dict ): + assertions = map(expand_dict_form, assertions.items() ) + + assert_list = [] + for assertion in assertions: + # TODO: not handling nested assertions correctly, + # not sure these are used though. + children = [] + if "children" in assertion: + children = assertion["children"] + del assertion["children"] + assert_dict = dict( + tag=assertion["that"], + attributes=assertion, + children=children, + ) + assert_list.append(assert_dict) + + return assert_list or None # XML variant is None if no assertions made + + class YamlPageSource(PageSource): def __init__(self, inputs_list): diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -127,6 +127,9 @@ if num_outputs: num_outputs = int( num_outputs ) self.num_outputs = num_outputs + self.command_line = test_dict.get("command", None) + self.stdout = test_dict.get("stdout", None) + self.stderr = test_dict.get("stderr", None) except Exception, e: self.error = True self.exception = e diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 test/functional/test_toolbox.py --- a/test/functional/test_toolbox.py +++ b/test/functional/test_toolbox.py @@ -1,6 +1,7 @@ import new import sys from base.twilltestcase import TwillTestCase +from base.asserts import verify_assertions from base.interactor import build_interactor, stage_data_in_history, RunToolException from base.instrument import register_job_data from galaxy.tools import DataManagerTool @@ -132,6 +133,21 @@ if job_stdio is None: job_stdio = galaxy_interactor.get_job_stdio( jobs[0][ 'id' ] ) + other_checks = { + "command_line": "Command produced by the job", + "stdout": "Standard output of the job", + "stderr": "Standard error of the job", + } + for what, description in other_checks.items(): + if getattr( testdef, what, None ) is not None: + try: + data = job_stdio[what] + verify_assertions( data, getattr( testdef, what ) ) + except AssertionError, err: + errmsg = '%s different than expected\n' % description + errmsg += str( err ) + found_exceptions.append( AssertionError( errmsg ) ) + if found_exceptions: raise JobOutputsError(found_exceptions, job_stdio) else: diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 test/functional/tools/job_properties.xml --- /dev/null +++ b/test/functional/tools/job_properties.xml @@ -0,0 +1,53 @@ +<tool id="job_properties" name="Test Job Properties"> + <command> + #if $thebool + echo "The bool is true"; + echo "The bool is really true" 1>&2; + echo "This is a line of text." > $out_file1 + #else + echo "The bool is not true"; + echo "The bool is very not true" 1>&2; + echo "This is a different line of text." > $out_file1 + #end if + + </command> + <inputs> + <param name="thebool" type="boolean" label="The boolean property" /> + </inputs> + <outputs> + <data name="out_file1" /> + </outputs> + <stdio> + <exit_code range="127" level="fatal" description="Failing exit code." /> + </stdio> + <tests> + <test> + <param name="thebool" value="true" /> + <output name="out_file1" file="simple_line.txt" /> + <assert_command> + <has_text text="really" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is really true" /> + </assert_stderr> + </test> + <test> + <param name="thebool" value="false" /> + <output name="out_file1" file="simple_line_alternative.txt" /> + <assert_command> + <has_text text="very not" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is not true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is very not true" /> + </assert_stderr> + </test> + </tests> + <help> + </help> +</tool> diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 322c44f7519fcbc57108a03402fdfdd211932266 test/functional/tools/samples_tool_conf.xml --- a/test/functional/tools/samples_tool_conf.xml +++ b/test/functional/tools/samples_tool_conf.xml @@ -13,6 +13,7 @@ <tool file="multi_output_assign_primary.xml" /><tool file="composite_output.xml" /><tool file="metadata.xml" /> + <tool file="job_properties.xml" /><tool file="gzipped_inputs.xml" /><tool file="output_order.xml" /><tool file="output_format.xml" /> https://bitbucket.org/galaxy/galaxy-central/commits/6a166c8b58e1/ Changeset: 6a166c8b58e1 User: jmchilton Date: 2014-12-17 19:22:08+00:00 Summary: Implement expect_failure and expect_exit_code on test tag of tools. Affected #: 5 files diff -r 322c44f7519fcbc57108a03402fdfdd211932266 -r 6a166c8b58e13920e93ec22bffbbd87321d04520 lib/galaxy/tools/parser/xml.py --- a/lib/galaxy/tools/parser/xml.py +++ b/lib/galaxy/tools/parser/xml.py @@ -189,6 +189,8 @@ command=__parse_assert_list_from_elem( test_elem.find("assert_command") ), stdout=__parse_assert_list_from_elem( test_elem.find("assert_stdout") ), stderr=__parse_assert_list_from_elem( test_elem.find("assert_stderr") ), + expect_exit_code=test_elem.get("expect_exit_code"), + expect_failure=string_as_bool(test_elem.get("expect_failure", False)), ) _copy_to_dict_if_present(test_elem, rval, ["interactor", "num_outputs"]) return rval diff -r 322c44f7519fcbc57108a03402fdfdd211932266 -r 6a166c8b58e13920e93ec22bffbbd87321d04520 lib/galaxy/tools/parser/yaml.py --- a/lib/galaxy/tools/parser/yaml.py +++ b/lib/galaxy/tools/parser/yaml.py @@ -157,9 +157,11 @@ _ensure_has(attributes, defaults) test_dict["outputs"] = new_outputs - test_dict["command"] = __to_test_assert_list( attributes.get( "command", [] ) ) - test_dict["stdout"] = __to_test_assert_list( attributes.get( "stdout", [] ) ) - test_dict["stderr"] = __to_test_assert_list( attributes.get( "stderr", [] ) ) + test_dict["command"] = __to_test_assert_list( test_dict.get( "command", [] ) ) + test_dict["stdout"] = __to_test_assert_list( test_dict.get( "stdout", [] ) ) + test_dict["stderr"] = __to_test_assert_list( test_dict.get( "stderr", [] ) ) + test_dict["expect_exit_code"] = test_dict.get( "expect_exit_code", None ) + test_dict["expect_failure"] = test_dict.get( "expect_exit_code", False ) return test_dict diff -r 322c44f7519fcbc57108a03402fdfdd211932266 -r 6a166c8b58e13920e93ec22bffbbd87321d04520 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -130,6 +130,8 @@ self.command_line = test_dict.get("command", None) self.stdout = test_dict.get("stdout", None) self.stderr = test_dict.get("stderr", None) + self.expect_exit_code = test_dict.get("expect_exit_code", None) + self.expect_failure = test_dict.get("expect_failure", False) except Exception, e: self.error = True self.exception = e diff -r 322c44f7519fcbc57108a03402fdfdd211932266 -r 6a166c8b58e13920e93ec22bffbbd87321d04520 test/functional/test_toolbox.py --- a/test/functional/test_toolbox.py +++ b/test/functional/test_toolbox.py @@ -93,6 +93,9 @@ raise Exception( "Test parse failure" ) def _verify_outputs( self, testdef, history, jobs, shed_tool_id, data_list, galaxy_interactor ): + assert len(jobs) == 1, "Test framework logic error, somehow tool test resulted in more than one job." + job = jobs[ 0 ] + maxseconds = testdef.maxseconds if testdef.num_outputs is not None: expected = testdef.num_outputs @@ -103,7 +106,33 @@ raise Exception( message ) found_exceptions = [] - job_stdio = None + if testdef.expect_failure: + if testdef.outputs: + raise Exception("Cannot specify outputs in a test expecting failure.") + + # Wait for the job to complete and register expections if the final + # status was not what test was expecting. + job_failed = False + try: + galaxy_interactor.wait_for_job( job[ 'id' ], history, maxseconds ) + except Exception as e: + job_failed = True + if not testdef.expect_failure: + found_exceptions.append(e) + + if not job_failed and testdef.expect_failure: + error = AssertionError("Expected job to fail but Galaxy indicated the job successfully completed.") + found_exceptions.append(error) + + job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) + + expect_exit_code = testdef.expect_exit_code + if expect_exit_code is not None: + exit_code = job_stdio["exit_code"] + if str(expect_exit_code) != str(exit_code): + error = AssertionError("Expected job to complete with exit code %s, found %s" % (expect_exit_code, exit_code)) + found_exceptions.append(error) + for output_index, output_tuple in enumerate(testdef.outputs): # Get the correct hid name, outfile, attributes = output_tuple @@ -124,14 +153,10 @@ except Exception as e: if not found_exceptions: # Only print this stuff out once. - for job in jobs: - job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) - for stream in ['stdout', 'stderr']: - if stream in job_stdio: - print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) + for stream in ['stdout', 'stderr']: + if stream in job_stdio: + print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) found_exceptions.append(e) - if job_stdio is None: - job_stdio = galaxy_interactor.get_job_stdio( jobs[0][ 'id' ] ) other_checks = { "command_line": "Command produced by the job", diff -r 322c44f7519fcbc57108a03402fdfdd211932266 -r 6a166c8b58e13920e93ec22bffbbd87321d04520 test/functional/tools/job_properties.xml --- a/test/functional/tools/job_properties.xml +++ b/test/functional/tools/job_properties.xml @@ -7,12 +7,17 @@ #else echo "The bool is not true"; echo "The bool is very not true" 1>&2; - echo "This is a different line of text." > $out_file1 + echo "This is a different line of text." > $out_file1; + sh -c "exit 2" + #end if + #if $failbool + ; sh -c "exit 127" #end if </command><inputs><param name="thebool" type="boolean" label="The boolean property" /> + <param name="failbool" type="boolean" label="The failure property" checked="false" /></inputs><outputs><data name="out_file1" /> @@ -21,7 +26,7 @@ <exit_code range="127" level="fatal" description="Failing exit code." /></stdio><tests> - <test> + <test expect_exit_code="0"><param name="thebool" value="true" /><output name="out_file1" file="simple_line.txt" /><assert_command> @@ -34,7 +39,7 @@ <has_line line="The bool is really true" /></assert_stderr></test> - <test> + <test expect_exit_code="2"><param name="thebool" value="false" /><output name="out_file1" file="simple_line_alternative.txt" /><assert_command> @@ -47,6 +52,10 @@ <has_line line="The bool is very not true" /></assert_stderr></test> + <test expect_exit_coded="127" expect_failure="true"> + <param name="thebool" value="true" /> + <param name="failbool" value="true" /> + </test></tests><help></help> https://bitbucket.org/galaxy/galaxy-central/commits/5e5c0cf930bb/ Changeset: 5e5c0cf930bb User: jmchilton Date: 2014-12-17 20:47:54+00:00 Summary: Merged in jmchilton/galaxy-central-fork-1 (pull request #617) Allow tool tests to assert properties about command, standard output, and standard error. Affected #: 6 files diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/parser/xml.py --- a/lib/galaxy/tools/parser/xml.py +++ b/lib/galaxy/tools/parser/xml.py @@ -186,6 +186,11 @@ rval = dict( outputs=__parse_output_elems(test_elem), inputs=__parse_input_elems(test_elem, i), + command=__parse_assert_list_from_elem( test_elem.find("assert_command") ), + stdout=__parse_assert_list_from_elem( test_elem.find("assert_stdout") ), + stderr=__parse_assert_list_from_elem( test_elem.find("assert_stderr") ), + expect_exit_code=test_elem.get("expect_exit_code"), + expect_failure=string_as_bool(test_elem.get("expect_failure", False)), ) _copy_to_dict_if_present(test_elem, rval, ["interactor", "num_outputs"]) return rval @@ -222,6 +227,11 @@ return name, file, attributes +def __parse_command_elem( test_elem ): + assert_elem = test_elem.find("command") + return __parse_assert_list_from_elem( assert_elem ) + + def __parse_test_attributes( output_elem, attrib ): assert_list = __parse_assert_list( output_elem ) file = attrib.pop( 'file', None ) @@ -252,6 +262,10 @@ def __parse_assert_list( output_elem ): assert_elem = output_elem.find("assert_contents") + return __parse_assert_list_from_elem( assert_elem ) + + +def __parse_assert_list_from_elem( assert_elem ): assert_list = None def convert_elem(elem): diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/parser/yaml.py --- a/lib/galaxy/tools/parser/yaml.py +++ b/lib/galaxy/tools/parser/yaml.py @@ -152,27 +152,47 @@ attributes["metadata"] = {} # TODO assert_list = [] - for key, assertion in attributes.get("asserts", {}).iteritems(): - # TODO: not handling nested assertions correctly, - # not sure these are used though. - children = [] - if "children" in assertion: - children = assertion["children"] - del assertion["children"] - assert_dict = dict( - tag=key, - attributes=assertion, - children=children, - ) - assert_list.append(assert_dict) + assert_list = __to_test_assert_list( attributes.get("asserts", [] ) ) attributes["assert_list"] = assert_list - _ensure_has(attributes, defaults) test_dict["outputs"] = new_outputs + test_dict["command"] = __to_test_assert_list( test_dict.get( "command", [] ) ) + test_dict["stdout"] = __to_test_assert_list( test_dict.get( "stdout", [] ) ) + test_dict["stderr"] = __to_test_assert_list( test_dict.get( "stderr", [] ) ) + test_dict["expect_exit_code"] = test_dict.get( "expect_exit_code", None ) + test_dict["expect_failure"] = test_dict.get( "expect_exit_code", False ) return test_dict +def __to_test_assert_list(assertions): + def expand_dict_form(item): + key, value = item + new_value = value.copy() + new_value["that"] = key + return new_value + + if isinstance( assertions, dict ): + assertions = map(expand_dict_form, assertions.items() ) + + assert_list = [] + for assertion in assertions: + # TODO: not handling nested assertions correctly, + # not sure these are used though. + children = [] + if "children" in assertion: + children = assertion["children"] + del assertion["children"] + assert_dict = dict( + tag=assertion["that"], + attributes=assertion, + children=children, + ) + assert_list.append(assert_dict) + + return assert_list or None # XML variant is None if no assertions made + + class YamlPageSource(PageSource): def __init__(self, inputs_list): diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -127,6 +127,11 @@ if num_outputs: num_outputs = int( num_outputs ) self.num_outputs = num_outputs + self.command_line = test_dict.get("command", None) + self.stdout = test_dict.get("stdout", None) + self.stderr = test_dict.get("stderr", None) + self.expect_exit_code = test_dict.get("expect_exit_code", None) + self.expect_failure = test_dict.get("expect_failure", False) except Exception, e: self.error = True self.exception = e diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/test_toolbox.py --- a/test/functional/test_toolbox.py +++ b/test/functional/test_toolbox.py @@ -1,6 +1,7 @@ import new import sys from base.twilltestcase import TwillTestCase +from base.asserts import verify_assertions from base.interactor import build_interactor, stage_data_in_history, RunToolException from base.instrument import register_job_data from galaxy.tools import DataManagerTool @@ -92,6 +93,9 @@ raise Exception( "Test parse failure" ) def _verify_outputs( self, testdef, history, jobs, shed_tool_id, data_list, galaxy_interactor ): + assert len(jobs) == 1, "Test framework logic error, somehow tool test resulted in more than one job." + job = jobs[ 0 ] + maxseconds = testdef.maxseconds if testdef.num_outputs is not None: expected = testdef.num_outputs @@ -102,7 +106,33 @@ raise Exception( message ) found_exceptions = [] - job_stdio = None + if testdef.expect_failure: + if testdef.outputs: + raise Exception("Cannot specify outputs in a test expecting failure.") + + # Wait for the job to complete and register expections if the final + # status was not what test was expecting. + job_failed = False + try: + galaxy_interactor.wait_for_job( job[ 'id' ], history, maxseconds ) + except Exception as e: + job_failed = True + if not testdef.expect_failure: + found_exceptions.append(e) + + if not job_failed and testdef.expect_failure: + error = AssertionError("Expected job to fail but Galaxy indicated the job successfully completed.") + found_exceptions.append(error) + + job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) + + expect_exit_code = testdef.expect_exit_code + if expect_exit_code is not None: + exit_code = job_stdio["exit_code"] + if str(expect_exit_code) != str(exit_code): + error = AssertionError("Expected job to complete with exit code %s, found %s" % (expect_exit_code, exit_code)) + found_exceptions.append(error) + for output_index, output_tuple in enumerate(testdef.outputs): # Get the correct hid name, outfile, attributes = output_tuple @@ -123,14 +153,25 @@ except Exception as e: if not found_exceptions: # Only print this stuff out once. - for job in jobs: - job_stdio = galaxy_interactor.get_job_stdio( job[ 'id' ] ) - for stream in ['stdout', 'stderr']: - if stream in job_stdio: - print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) + for stream in ['stdout', 'stderr']: + if stream in job_stdio: + print >>sys.stderr, self._format_stream( job_stdio[ stream ], stream=stream, format=True ) found_exceptions.append(e) - if job_stdio is None: - job_stdio = galaxy_interactor.get_job_stdio( jobs[0][ 'id' ] ) + + other_checks = { + "command_line": "Command produced by the job", + "stdout": "Standard output of the job", + "stderr": "Standard error of the job", + } + for what, description in other_checks.items(): + if getattr( testdef, what, None ) is not None: + try: + data = job_stdio[what] + verify_assertions( data, getattr( testdef, what ) ) + except AssertionError, err: + errmsg = '%s different than expected\n' % description + errmsg += str( err ) + found_exceptions.append( AssertionError( errmsg ) ) if found_exceptions: raise JobOutputsError(found_exceptions, job_stdio) diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/tools/job_properties.xml --- /dev/null +++ b/test/functional/tools/job_properties.xml @@ -0,0 +1,62 @@ +<tool id="job_properties" name="Test Job Properties"> + <command> + #if $thebool + echo "The bool is true"; + echo "The bool is really true" 1>&2; + echo "This is a line of text." > $out_file1 + #else + echo "The bool is not true"; + echo "The bool is very not true" 1>&2; + echo "This is a different line of text." > $out_file1; + sh -c "exit 2" + #end if + #if $failbool + ; sh -c "exit 127" + #end if + + </command> + <inputs> + <param name="thebool" type="boolean" label="The boolean property" /> + <param name="failbool" type="boolean" label="The failure property" checked="false" /> + </inputs> + <outputs> + <data name="out_file1" /> + </outputs> + <stdio> + <exit_code range="127" level="fatal" description="Failing exit code." /> + </stdio> + <tests> + <test expect_exit_code="0"> + <param name="thebool" value="true" /> + <output name="out_file1" file="simple_line.txt" /> + <assert_command> + <has_text text="really" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is really true" /> + </assert_stderr> + </test> + <test expect_exit_code="2"> + <param name="thebool" value="false" /> + <output name="out_file1" file="simple_line_alternative.txt" /> + <assert_command> + <has_text text="very not" /> + </assert_command> + <assert_stdout> + <has_line line="The bool is not true" /> + </assert_stdout> + <assert_stderr> + <has_line line="The bool is very not true" /> + </assert_stderr> + </test> + <test expect_exit_coded="127" expect_failure="true"> + <param name="thebool" value="true" /> + <param name="failbool" value="true" /> + </test> + </tests> + <help> + </help> +</tool> diff -r f24461d3e0b3f737f8e4c99b23475b571c32c61e -r 5e5c0cf930bb96bf8af0bd19daad1a999b40c374 test/functional/tools/samples_tool_conf.xml --- a/test/functional/tools/samples_tool_conf.xml +++ b/test/functional/tools/samples_tool_conf.xml @@ -13,6 +13,7 @@ <tool file="multi_output_assign_primary.xml" /><tool file="composite_output.xml" /><tool file="metadata.xml" /> + <tool file="job_properties.xml" /><tool file="gzipped_inputs.xml" /><tool file="output_order.xml" /><tool file="output_format.xml" /> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/9f8b9ad3a85e/ Changeset: 9f8b9ad3a85e User: dannon Date: 2014-12-17 17:06:23+00:00 Summary: Add thead element to tabular dataset view. Affected #: 3 files diff -r 867635153252d0830dac64ba72ede3cb637e2abf -r 9f8b9ad3a85e1707ba37ba763e50cad01a9f79bb client/galaxy/scripts/mvc/data.js --- a/client/galaxy/scripts/mvc/data.js +++ b/client/galaxy/scripts/mvc/data.js @@ -169,7 +169,8 @@ }); this.$el.append(data_table); var column_names = this.model.get_metadata('column_names'), - header_row = $('<tr/>').appendTo(data_table); + header_container = $('<thead/>').appendTo(data_table), + header_row = $('<tr/>').appendTo(header_container); if (column_names) { header_row.append('<th>' + column_names.join('</th><th>') + '</th>'); } else { diff -r 867635153252d0830dac64ba72ede3cb637e2abf -r 9f8b9ad3a85e1707ba37ba763e50cad01a9f79bb static/scripts/mvc/data.js --- a/static/scripts/mvc/data.js +++ b/static/scripts/mvc/data.js @@ -169,7 +169,8 @@ }); this.$el.append(data_table); var column_names = this.model.get_metadata('column_names'), - header_row = $('<tr/>').appendTo(data_table); + header_container = $('<thead/>').appendTo(data_table), + header_row = $('<tr/>').appendTo(header_container); if (column_names) { header_row.append('<th>' + column_names.join('</th><th>') + '</th>'); } else { diff -r 867635153252d0830dac64ba72ede3cb637e2abf -r 9f8b9ad3a85e1707ba37ba763e50cad01a9f79bb static/scripts/packed/mvc/data.js --- a/static/scripts/packed/mvc/data.js +++ b/static/scripts/packed/mvc/data.js @@ -1,1 +1,1 @@ -define(["mvc/ui/ui-modal","mvc/ui/ui-frames","mvc/ui/icon-button"],function(k,j,f){var h=Backbone.Model.extend({});var b=Backbone.Model.extend({defaults:{id:"",type:"",name:"",hda_ldda:"hda",metadata:null},initialize:function(){if(!this.get("metadata")){this._set_metadata()}this.on("change",this._set_metadata,this)},_set_metadata:function(){var n=new h();_.each(_.keys(this.attributes),function(o){if(o.indexOf("metadata_")===0){var p=o.split("metadata_")[1];n.set(p,this.attributes[o]);delete this.attributes[o]}},this);this.set("metadata",n,{silent:true})},get_metadata:function(n){return this.attributes.metadata.get(n)},urlRoot:galaxy_config.root+"api/datasets"});var i=b.extend({defaults:_.extend({},b.prototype.defaults,{chunk_url:null,first_data_chunk:null,chunk_index:-1,at_eof:false}),initialize:function(n){b.prototype.initialize.call(this);this.attributes.chunk_index=(this.attributes.first_data_chunk?1:0);this.attributes.chunk_url=galaxy_config.root+"dataset/display?dataset_id="+this.id;this.attributes.url_viz=galaxy_config.root+"visualization"},get_next_chunk:function(){if(this.attributes.at_eof){return null}var n=this,o=$.Deferred();$.getJSON(this.attributes.chunk_url,{chunk:n.attributes.chunk_index++}).success(function(p){var q;if(p.ck_data!==""){q=p}else{n.attributes.at_eof=true;q=null}o.resolve(q)});return o}});var e=Backbone.Collection.extend({model:b});var a=Backbone.View.extend({initialize:function(n){this.row_count=0;this.loading_chunk=false;new d({model:n.model,$el:this.$el})},expand_to_container:function(){if(this.$el.height()<this.scroll_elt.height()){this.attempt_to_fetch()}},attempt_to_fetch:function(o){var n=this;if(!this.loading_chunk&&this.scrolled_to_bottom()){this.loading_chunk=true;this.loading_indicator.show();$.when(n.model.get_next_chunk()).then(function(p){if(p){n._renderChunk(p);n.loading_chunk=false}n.loading_indicator.hide();n.expand_to_container()})}},render:function(){this.loading_indicator=$("<div/>").attr("id","loading_indicator");this.$el.append(this.loading_indicator);var r=$("<table/>").attr({id:"content_table",cellpadding:0});this.$el.append(r);var n=this.model.get_metadata("column_names"),s=$("<tr/>").appendTo(r);if(n){s.append("<th>"+n.join("</th><th>")+"</th>")}else{for(var q=1;q<=this.model.get_metadata("columns");q++){s.append("<th>"+q+"</th>")}}var p=this,o=this.model.get("first_data_chunk");if(o){this._renderChunk(o)}else{$.when(p.model.get_next_chunk()).then(function(t){p._renderChunk(t)})}this.scroll_elt.scroll(function(){p.attempt_to_fetch()})},scrolled_to_bottom:function(){return false},_renderCell:function(q,n,r){var o=$("<td>").text(q);var p=this.model.get_metadata("column_types");if(r!==undefined){o.attr("colspan",r).addClass("stringalign")}else{if(p){if(n<p.length){if(p[n]==="str"||p[n]==="list"){o.addClass("stringalign")}}}}return o},_renderRow:function(n){var o=n.split("\t"),q=$("<tr>"),p=this.model.get_metadata("columns");if(this.row_count%2!==0){q.addClass("dark_row")}if(o.length===p){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this)}else{if(o.length>p){_.each(o.slice(0,p-1),function(s,r){q.append(this._renderCell(s,r))},this);q.append(this._renderCell(o.slice(p-1).join("\t"),p-1))}else{if(p>5&&o.length===p-1){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this);q.append($("<td>"))}else{q.append(this._renderCell(n,0,p))}}}this.row_count++;return q},_renderChunk:function(n){var o=this.$el.find("table");_.each(n.ck_data.split("\n"),function(p,q){if(p!==""){o.append(this._renderRow(p))}},this)}});var g=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);scroll_elt=_.find(this.$el.parents(),function(o){return $(o).css("overflow")==="auto"});if(!scroll_elt){scroll_elt=window}this.scroll_elt=$(scroll_elt)},scrolled_to_bottom:function(){return(this.$el.height()-this.scroll_elt.scrollTop()-this.scroll_elt.height()<=0)}});var m=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);this.scroll_elt=this.$el.css({position:"relative",overflow:"scroll",height:this.options.height||"500px"})},scrolled_to_bottom:function(){return this.$el.scrollTop()+this.$el.innerHeight()>=this.el.scrollHeight}});var d=Backbone.View.extend({col:{chrom:null,start:null,end:null},url_viz:null,dataset_id:null,genome_build:null,file_ext:null,initialize:function(p){var s=parent.Galaxy;if(s&&s.modal){this.modal=s.modal}if(s&&s.frame){this.frame=s.frame}if(!this.modal||!this.frame){return}var o=p.model;var r=o.get("metadata");if(!o.get("file_ext")){return}this.file_ext=o.get("file_ext");if(this.file_ext=="bed"){if(r.get("chromCol")&&r.get("startCol")&&r.get("endCol")){this.col.chrom=r.get("chromCol")-1;this.col.start=r.get("startCol")-1;this.col.end=r.get("endCol")-1}else{console.log("TabularButtonTrackster : Bed-file metadata incomplete.");return}}if(this.file_ext=="vcf"){function q(u,v){for(var t=0;t<v.length;t++){if(v[t].match(u)){return t}}return -1}this.col.chrom=q("Chrom",r.get("column_names"));this.col.start=q("Pos",r.get("column_names"));this.col.end=null;if(this.col.chrom==-1||this.col.start==-1){console.log("TabularButtonTrackster : VCF-file metadata incomplete.");return}}if(this.col.chrom===undefined){return}if(o.id){this.dataset_id=o.id}else{console.log("TabularButtonTrackster : Dataset identification is missing.");return}if(o.get("url_viz")){this.url_viz=o.get("url_viz")}else{console.log("TabularButtonTrackster : Url for visualization controller is missing.");return}if(o.get("genome_build")){this.genome_build=o.get("genome_build")}var n=new f.IconButtonView({model:new f.IconButton({title:"Visualize",icon_class:"chart_curve",id:"btn_viz"})});this.setElement(p.$el);this.$el.append(n.render().$el);this.hide()},events:{"mouseover tr":"show",mouseleave:"hide"},show:function(s){function r(x){return !isNaN(parseFloat(x))&&isFinite(x)}if(this.col.chrom===null){return}var w=$(s.target).parent();var t=w.children().eq(this.col.chrom).html();var n=w.children().eq(this.col.start).html();var p=this.col.end?w.children().eq(this.col.end).html():n;if(!t.match("^#")&&t!==""&&r(n)){var v={dataset_id:this.dataset_id,gene_region:t+":"+n+"-"+p};var q=w.offset();var o=q.left-10;var u=q.top-$(window).scrollTop()+3;$("#btn_viz").css({position:"fixed",top:u+"px",left:o+"px"});$("#btn_viz").off("click");$("#btn_viz").click(this.create_trackster_action(this.url_viz,v,this.genome_build));$("#btn_viz").show()}else{$("#btn_viz").hide()}},hide:function(){this.$el.find("#btn_viz").hide()},create_trackster_action:function(n,q,p){var o=this;return function(){var r={};if(p){r["f-dbkey"]=p}$.ajax({url:n+"/list_tracks?"+$.param(r),dataType:"html",error:function(){o.modal.show({title:"Something went wrong!",body:"Unfortunately we could not add this dataset to the track browser. Please try again or contact us.",buttons:{Cancel:function(){o.modal.hide()}}})},success:function(s){o.modal.show({title:"View Data in a New or Saved Visualization",buttons:{Cancel:function(){o.modal.hide()},"View in saved visualization":function(){o.modal.show({title:"Add Data to Saved Visualization",body:s,buttons:{Cancel:function(){o.modal.hide()},"Add to visualization":function(){o.modal.hide();o.modal.$el.find("input[name=id]:checked").each(function(){var t=$(this).val();q.id=t;o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})})}}})},"View in new visualization":function(){o.modal.hide();o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})}}})}});return false}}});var l=function(q,o,r,n){var p=new o({model:new q(r)});p.render();if(n){n.append(p.$el)}return p};var c=function(p){if(!p.model){p.model=new i(p.dataset_config)}var o=p.parent_elt;var q=p.embedded;delete p.embedded;delete p.parent_elt;delete p.dataset_config;var n=(q?new m(p):new g(p));n.render();if(o){o.append(n.$el);n.expand_to_container()}return n};return{Dataset:b,TabularDataset:i,DatasetCollection:e,TabularDatasetChunkedView:a,createTabularDatasetChunkedView:c}}); \ No newline at end of file +define(["mvc/ui/ui-modal","mvc/ui/ui-frames","mvc/ui/icon-button"],function(k,j,f){var h=Backbone.Model.extend({});var b=Backbone.Model.extend({defaults:{id:"",type:"",name:"",hda_ldda:"hda",metadata:null},initialize:function(){if(!this.get("metadata")){this._set_metadata()}this.on("change",this._set_metadata,this)},_set_metadata:function(){var n=new h();_.each(_.keys(this.attributes),function(o){if(o.indexOf("metadata_")===0){var p=o.split("metadata_")[1];n.set(p,this.attributes[o]);delete this.attributes[o]}},this);this.set("metadata",n,{silent:true})},get_metadata:function(n){return this.attributes.metadata.get(n)},urlRoot:galaxy_config.root+"api/datasets"});var i=b.extend({defaults:_.extend({},b.prototype.defaults,{chunk_url:null,first_data_chunk:null,chunk_index:-1,at_eof:false}),initialize:function(n){b.prototype.initialize.call(this);this.attributes.chunk_index=(this.attributes.first_data_chunk?1:0);this.attributes.chunk_url=galaxy_config.root+"dataset/display?dataset_id="+this.id;this.attributes.url_viz=galaxy_config.root+"visualization"},get_next_chunk:function(){if(this.attributes.at_eof){return null}var n=this,o=$.Deferred();$.getJSON(this.attributes.chunk_url,{chunk:n.attributes.chunk_index++}).success(function(p){var q;if(p.ck_data!==""){q=p}else{n.attributes.at_eof=true;q=null}o.resolve(q)});return o}});var e=Backbone.Collection.extend({model:b});var a=Backbone.View.extend({initialize:function(n){this.row_count=0;this.loading_chunk=false;new d({model:n.model,$el:this.$el})},expand_to_container:function(){if(this.$el.height()<this.scroll_elt.height()){this.attempt_to_fetch()}},attempt_to_fetch:function(o){var n=this;if(!this.loading_chunk&&this.scrolled_to_bottom()){this.loading_chunk=true;this.loading_indicator.show();$.when(n.model.get_next_chunk()).then(function(p){if(p){n._renderChunk(p);n.loading_chunk=false}n.loading_indicator.hide();n.expand_to_container()})}},render:function(){this.loading_indicator=$("<div/>").attr("id","loading_indicator");this.$el.append(this.loading_indicator);var r=$("<table/>").attr({id:"content_table",cellpadding:0});this.$el.append(r);var n=this.model.get_metadata("column_names"),s=$("<thead/>").appendTo(r),t=$("<tr/>").appendTo(s);if(n){t.append("<th>"+n.join("</th><th>")+"</th>")}else{for(var q=1;q<=this.model.get_metadata("columns");q++){t.append("<th>"+q+"</th>")}}var p=this,o=this.model.get("first_data_chunk");if(o){this._renderChunk(o)}else{$.when(p.model.get_next_chunk()).then(function(u){p._renderChunk(u)})}this.scroll_elt.scroll(function(){p.attempt_to_fetch()})},scrolled_to_bottom:function(){return false},_renderCell:function(q,n,r){var o=$("<td>").text(q);var p=this.model.get_metadata("column_types");if(r!==undefined){o.attr("colspan",r).addClass("stringalign")}else{if(p){if(n<p.length){if(p[n]==="str"||p[n]==="list"){o.addClass("stringalign")}}}}return o},_renderRow:function(n){var o=n.split("\t"),q=$("<tr>"),p=this.model.get_metadata("columns");if(this.row_count%2!==0){q.addClass("dark_row")}if(o.length===p){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this)}else{if(o.length>p){_.each(o.slice(0,p-1),function(s,r){q.append(this._renderCell(s,r))},this);q.append(this._renderCell(o.slice(p-1).join("\t"),p-1))}else{if(p>5&&o.length===p-1){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this);q.append($("<td>"))}else{q.append(this._renderCell(n,0,p))}}}this.row_count++;return q},_renderChunk:function(n){var o=this.$el.find("table");_.each(n.ck_data.split("\n"),function(p,q){if(p!==""){o.append(this._renderRow(p))}},this)}});var g=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);scroll_elt=_.find(this.$el.parents(),function(o){return $(o).css("overflow")==="auto"});if(!scroll_elt){scroll_elt=window}this.scroll_elt=$(scroll_elt)},scrolled_to_bottom:function(){return(this.$el.height()-this.scroll_elt.scrollTop()-this.scroll_elt.height()<=0)}});var m=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);this.scroll_elt=this.$el.css({position:"relative",overflow:"scroll",height:this.options.height||"500px"})},scrolled_to_bottom:function(){return this.$el.scrollTop()+this.$el.innerHeight()>=this.el.scrollHeight}});var d=Backbone.View.extend({col:{chrom:null,start:null,end:null},url_viz:null,dataset_id:null,genome_build:null,file_ext:null,initialize:function(p){var s=parent.Galaxy;if(s&&s.modal){this.modal=s.modal}if(s&&s.frame){this.frame=s.frame}if(!this.modal||!this.frame){return}var o=p.model;var r=o.get("metadata");if(!o.get("file_ext")){return}this.file_ext=o.get("file_ext");if(this.file_ext=="bed"){if(r.get("chromCol")&&r.get("startCol")&&r.get("endCol")){this.col.chrom=r.get("chromCol")-1;this.col.start=r.get("startCol")-1;this.col.end=r.get("endCol")-1}else{console.log("TabularButtonTrackster : Bed-file metadata incomplete.");return}}if(this.file_ext=="vcf"){function q(u,v){for(var t=0;t<v.length;t++){if(v[t].match(u)){return t}}return -1}this.col.chrom=q("Chrom",r.get("column_names"));this.col.start=q("Pos",r.get("column_names"));this.col.end=null;if(this.col.chrom==-1||this.col.start==-1){console.log("TabularButtonTrackster : VCF-file metadata incomplete.");return}}if(this.col.chrom===undefined){return}if(o.id){this.dataset_id=o.id}else{console.log("TabularButtonTrackster : Dataset identification is missing.");return}if(o.get("url_viz")){this.url_viz=o.get("url_viz")}else{console.log("TabularButtonTrackster : Url for visualization controller is missing.");return}if(o.get("genome_build")){this.genome_build=o.get("genome_build")}var n=new f.IconButtonView({model:new f.IconButton({title:"Visualize",icon_class:"chart_curve",id:"btn_viz"})});this.setElement(p.$el);this.$el.append(n.render().$el);this.hide()},events:{"mouseover tr":"show",mouseleave:"hide"},show:function(s){function r(x){return !isNaN(parseFloat(x))&&isFinite(x)}if(this.col.chrom===null){return}var w=$(s.target).parent();var t=w.children().eq(this.col.chrom).html();var n=w.children().eq(this.col.start).html();var p=this.col.end?w.children().eq(this.col.end).html():n;if(!t.match("^#")&&t!==""&&r(n)){var v={dataset_id:this.dataset_id,gene_region:t+":"+n+"-"+p};var q=w.offset();var o=q.left-10;var u=q.top-$(window).scrollTop()+3;$("#btn_viz").css({position:"fixed",top:u+"px",left:o+"px"});$("#btn_viz").off("click");$("#btn_viz").click(this.create_trackster_action(this.url_viz,v,this.genome_build));$("#btn_viz").show()}else{$("#btn_viz").hide()}},hide:function(){this.$el.find("#btn_viz").hide()},create_trackster_action:function(n,q,p){var o=this;return function(){var r={};if(p){r["f-dbkey"]=p}$.ajax({url:n+"/list_tracks?"+$.param(r),dataType:"html",error:function(){o.modal.show({title:"Something went wrong!",body:"Unfortunately we could not add this dataset to the track browser. Please try again or contact us.",buttons:{Cancel:function(){o.modal.hide()}}})},success:function(s){o.modal.show({title:"View Data in a New or Saved Visualization",buttons:{Cancel:function(){o.modal.hide()},"View in saved visualization":function(){o.modal.show({title:"Add Data to Saved Visualization",body:s,buttons:{Cancel:function(){o.modal.hide()},"Add to visualization":function(){o.modal.hide();o.modal.$el.find("input[name=id]:checked").each(function(){var t=$(this).val();q.id=t;o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})})}}})},"View in new visualization":function(){o.modal.hide();o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})}}})}});return false}}});var l=function(q,o,r,n){var p=new o({model:new q(r)});p.render();if(n){n.append(p.$el)}return p};var c=function(p){if(!p.model){p.model=new i(p.dataset_config)}var o=p.parent_elt;var q=p.embedded;delete p.embedded;delete p.parent_elt;delete p.dataset_config;var n=(q?new m(p):new g(p));n.render();if(o){o.append(n.$el);n.expand_to_container()}return n};return{Dataset:b,TabularDataset:i,DatasetCollection:e,TabularDatasetChunkedView:a,createTabularDatasetChunkedView:c}}); \ No newline at end of file https://bitbucket.org/galaxy/galaxy-central/commits/5b418e54bcdd/ Changeset: 5b418e54bcdd User: dannon Date: 2014-12-17 20:40:30+00:00 Summary: Pointed out by Eric, Apache based remote_user_headers can possibly be (null)(a)domain.edu for some local configurations and this allows REMOTE_AUTH to work correctly in that situation as well as the usual (null) header. Affected #: 1 file diff -r cc7147514d77799989777fd21aa818e43d9f3d39 -r 5b418e54bcdddbe7f3b67bd1705933aa9aa37023 lib/galaxy/web/framework/middleware/remoteuser.py --- a/lib/galaxy/web/framework/middleware/remoteuser.py +++ b/lib/galaxy/web/framework/middleware/remoteuser.py @@ -59,7 +59,7 @@ # Rewrite* method for passing REMOTE_USER and a user is # un-authenticated. Any other possible values need to go here as well. path_info = environ.get('PATH_INFO', '') - if environ.get(self.remote_user_header, '(null)') != '(null)': + if not environ.get(self.remote_user_header, '(null)').startswith('(null)'): if not environ[ self.remote_user_header ].count( '@' ): if self.maildomain is not None: environ[ self.remote_user_header ] += '@' + self.maildomain https://bitbucket.org/galaxy/galaxy-central/commits/f24461d3e0b3/ Changeset: f24461d3e0b3 User: dannon Date: 2014-12-17 20:41:21+00:00 Summary: Merge. Affected #: 3 files diff -r 5b418e54bcdddbe7f3b67bd1705933aa9aa37023 -r f24461d3e0b3f737f8e4c99b23475b571c32c61e client/galaxy/scripts/mvc/data.js --- a/client/galaxy/scripts/mvc/data.js +++ b/client/galaxy/scripts/mvc/data.js @@ -169,7 +169,8 @@ }); this.$el.append(data_table); var column_names = this.model.get_metadata('column_names'), - header_row = $('<tr/>').appendTo(data_table); + header_container = $('<thead/>').appendTo(data_table), + header_row = $('<tr/>').appendTo(header_container); if (column_names) { header_row.append('<th>' + column_names.join('</th><th>') + '</th>'); } else { diff -r 5b418e54bcdddbe7f3b67bd1705933aa9aa37023 -r f24461d3e0b3f737f8e4c99b23475b571c32c61e static/scripts/mvc/data.js --- a/static/scripts/mvc/data.js +++ b/static/scripts/mvc/data.js @@ -169,7 +169,8 @@ }); this.$el.append(data_table); var column_names = this.model.get_metadata('column_names'), - header_row = $('<tr/>').appendTo(data_table); + header_container = $('<thead/>').appendTo(data_table), + header_row = $('<tr/>').appendTo(header_container); if (column_names) { header_row.append('<th>' + column_names.join('</th><th>') + '</th>'); } else { diff -r 5b418e54bcdddbe7f3b67bd1705933aa9aa37023 -r f24461d3e0b3f737f8e4c99b23475b571c32c61e static/scripts/packed/mvc/data.js --- a/static/scripts/packed/mvc/data.js +++ b/static/scripts/packed/mvc/data.js @@ -1,1 +1,1 @@ -define(["mvc/ui/ui-modal","mvc/ui/ui-frames","mvc/ui/icon-button"],function(k,j,f){var h=Backbone.Model.extend({});var b=Backbone.Model.extend({defaults:{id:"",type:"",name:"",hda_ldda:"hda",metadata:null},initialize:function(){if(!this.get("metadata")){this._set_metadata()}this.on("change",this._set_metadata,this)},_set_metadata:function(){var n=new h();_.each(_.keys(this.attributes),function(o){if(o.indexOf("metadata_")===0){var p=o.split("metadata_")[1];n.set(p,this.attributes[o]);delete this.attributes[o]}},this);this.set("metadata",n,{silent:true})},get_metadata:function(n){return this.attributes.metadata.get(n)},urlRoot:galaxy_config.root+"api/datasets"});var i=b.extend({defaults:_.extend({},b.prototype.defaults,{chunk_url:null,first_data_chunk:null,chunk_index:-1,at_eof:false}),initialize:function(n){b.prototype.initialize.call(this);this.attributes.chunk_index=(this.attributes.first_data_chunk?1:0);this.attributes.chunk_url=galaxy_config.root+"dataset/display?dataset_id="+this.id;this.attributes.url_viz=galaxy_config.root+"visualization"},get_next_chunk:function(){if(this.attributes.at_eof){return null}var n=this,o=$.Deferred();$.getJSON(this.attributes.chunk_url,{chunk:n.attributes.chunk_index++}).success(function(p){var q;if(p.ck_data!==""){q=p}else{n.attributes.at_eof=true;q=null}o.resolve(q)});return o}});var e=Backbone.Collection.extend({model:b});var a=Backbone.View.extend({initialize:function(n){this.row_count=0;this.loading_chunk=false;new d({model:n.model,$el:this.$el})},expand_to_container:function(){if(this.$el.height()<this.scroll_elt.height()){this.attempt_to_fetch()}},attempt_to_fetch:function(o){var n=this;if(!this.loading_chunk&&this.scrolled_to_bottom()){this.loading_chunk=true;this.loading_indicator.show();$.when(n.model.get_next_chunk()).then(function(p){if(p){n._renderChunk(p);n.loading_chunk=false}n.loading_indicator.hide();n.expand_to_container()})}},render:function(){this.loading_indicator=$("<div/>").attr("id","loading_indicator");this.$el.append(this.loading_indicator);var r=$("<table/>").attr({id:"content_table",cellpadding:0});this.$el.append(r);var n=this.model.get_metadata("column_names"),s=$("<tr/>").appendTo(r);if(n){s.append("<th>"+n.join("</th><th>")+"</th>")}else{for(var q=1;q<=this.model.get_metadata("columns");q++){s.append("<th>"+q+"</th>")}}var p=this,o=this.model.get("first_data_chunk");if(o){this._renderChunk(o)}else{$.when(p.model.get_next_chunk()).then(function(t){p._renderChunk(t)})}this.scroll_elt.scroll(function(){p.attempt_to_fetch()})},scrolled_to_bottom:function(){return false},_renderCell:function(q,n,r){var o=$("<td>").text(q);var p=this.model.get_metadata("column_types");if(r!==undefined){o.attr("colspan",r).addClass("stringalign")}else{if(p){if(n<p.length){if(p[n]==="str"||p[n]==="list"){o.addClass("stringalign")}}}}return o},_renderRow:function(n){var o=n.split("\t"),q=$("<tr>"),p=this.model.get_metadata("columns");if(this.row_count%2!==0){q.addClass("dark_row")}if(o.length===p){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this)}else{if(o.length>p){_.each(o.slice(0,p-1),function(s,r){q.append(this._renderCell(s,r))},this);q.append(this._renderCell(o.slice(p-1).join("\t"),p-1))}else{if(p>5&&o.length===p-1){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this);q.append($("<td>"))}else{q.append(this._renderCell(n,0,p))}}}this.row_count++;return q},_renderChunk:function(n){var o=this.$el.find("table");_.each(n.ck_data.split("\n"),function(p,q){if(p!==""){o.append(this._renderRow(p))}},this)}});var g=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);scroll_elt=_.find(this.$el.parents(),function(o){return $(o).css("overflow")==="auto"});if(!scroll_elt){scroll_elt=window}this.scroll_elt=$(scroll_elt)},scrolled_to_bottom:function(){return(this.$el.height()-this.scroll_elt.scrollTop()-this.scroll_elt.height()<=0)}});var m=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);this.scroll_elt=this.$el.css({position:"relative",overflow:"scroll",height:this.options.height||"500px"})},scrolled_to_bottom:function(){return this.$el.scrollTop()+this.$el.innerHeight()>=this.el.scrollHeight}});var d=Backbone.View.extend({col:{chrom:null,start:null,end:null},url_viz:null,dataset_id:null,genome_build:null,file_ext:null,initialize:function(p){var s=parent.Galaxy;if(s&&s.modal){this.modal=s.modal}if(s&&s.frame){this.frame=s.frame}if(!this.modal||!this.frame){return}var o=p.model;var r=o.get("metadata");if(!o.get("file_ext")){return}this.file_ext=o.get("file_ext");if(this.file_ext=="bed"){if(r.get("chromCol")&&r.get("startCol")&&r.get("endCol")){this.col.chrom=r.get("chromCol")-1;this.col.start=r.get("startCol")-1;this.col.end=r.get("endCol")-1}else{console.log("TabularButtonTrackster : Bed-file metadata incomplete.");return}}if(this.file_ext=="vcf"){function q(u,v){for(var t=0;t<v.length;t++){if(v[t].match(u)){return t}}return -1}this.col.chrom=q("Chrom",r.get("column_names"));this.col.start=q("Pos",r.get("column_names"));this.col.end=null;if(this.col.chrom==-1||this.col.start==-1){console.log("TabularButtonTrackster : VCF-file metadata incomplete.");return}}if(this.col.chrom===undefined){return}if(o.id){this.dataset_id=o.id}else{console.log("TabularButtonTrackster : Dataset identification is missing.");return}if(o.get("url_viz")){this.url_viz=o.get("url_viz")}else{console.log("TabularButtonTrackster : Url for visualization controller is missing.");return}if(o.get("genome_build")){this.genome_build=o.get("genome_build")}var n=new f.IconButtonView({model:new f.IconButton({title:"Visualize",icon_class:"chart_curve",id:"btn_viz"})});this.setElement(p.$el);this.$el.append(n.render().$el);this.hide()},events:{"mouseover tr":"show",mouseleave:"hide"},show:function(s){function r(x){return !isNaN(parseFloat(x))&&isFinite(x)}if(this.col.chrom===null){return}var w=$(s.target).parent();var t=w.children().eq(this.col.chrom).html();var n=w.children().eq(this.col.start).html();var p=this.col.end?w.children().eq(this.col.end).html():n;if(!t.match("^#")&&t!==""&&r(n)){var v={dataset_id:this.dataset_id,gene_region:t+":"+n+"-"+p};var q=w.offset();var o=q.left-10;var u=q.top-$(window).scrollTop()+3;$("#btn_viz").css({position:"fixed",top:u+"px",left:o+"px"});$("#btn_viz").off("click");$("#btn_viz").click(this.create_trackster_action(this.url_viz,v,this.genome_build));$("#btn_viz").show()}else{$("#btn_viz").hide()}},hide:function(){this.$el.find("#btn_viz").hide()},create_trackster_action:function(n,q,p){var o=this;return function(){var r={};if(p){r["f-dbkey"]=p}$.ajax({url:n+"/list_tracks?"+$.param(r),dataType:"html",error:function(){o.modal.show({title:"Something went wrong!",body:"Unfortunately we could not add this dataset to the track browser. Please try again or contact us.",buttons:{Cancel:function(){o.modal.hide()}}})},success:function(s){o.modal.show({title:"View Data in a New or Saved Visualization",buttons:{Cancel:function(){o.modal.hide()},"View in saved visualization":function(){o.modal.show({title:"Add Data to Saved Visualization",body:s,buttons:{Cancel:function(){o.modal.hide()},"Add to visualization":function(){o.modal.hide();o.modal.$el.find("input[name=id]:checked").each(function(){var t=$(this).val();q.id=t;o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})})}}})},"View in new visualization":function(){o.modal.hide();o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})}}})}});return false}}});var l=function(q,o,r,n){var p=new o({model:new q(r)});p.render();if(n){n.append(p.$el)}return p};var c=function(p){if(!p.model){p.model=new i(p.dataset_config)}var o=p.parent_elt;var q=p.embedded;delete p.embedded;delete p.parent_elt;delete p.dataset_config;var n=(q?new m(p):new g(p));n.render();if(o){o.append(n.$el);n.expand_to_container()}return n};return{Dataset:b,TabularDataset:i,DatasetCollection:e,TabularDatasetChunkedView:a,createTabularDatasetChunkedView:c}}); \ No newline at end of file +define(["mvc/ui/ui-modal","mvc/ui/ui-frames","mvc/ui/icon-button"],function(k,j,f){var h=Backbone.Model.extend({});var b=Backbone.Model.extend({defaults:{id:"",type:"",name:"",hda_ldda:"hda",metadata:null},initialize:function(){if(!this.get("metadata")){this._set_metadata()}this.on("change",this._set_metadata,this)},_set_metadata:function(){var n=new h();_.each(_.keys(this.attributes),function(o){if(o.indexOf("metadata_")===0){var p=o.split("metadata_")[1];n.set(p,this.attributes[o]);delete this.attributes[o]}},this);this.set("metadata",n,{silent:true})},get_metadata:function(n){return this.attributes.metadata.get(n)},urlRoot:galaxy_config.root+"api/datasets"});var i=b.extend({defaults:_.extend({},b.prototype.defaults,{chunk_url:null,first_data_chunk:null,chunk_index:-1,at_eof:false}),initialize:function(n){b.prototype.initialize.call(this);this.attributes.chunk_index=(this.attributes.first_data_chunk?1:0);this.attributes.chunk_url=galaxy_config.root+"dataset/display?dataset_id="+this.id;this.attributes.url_viz=galaxy_config.root+"visualization"},get_next_chunk:function(){if(this.attributes.at_eof){return null}var n=this,o=$.Deferred();$.getJSON(this.attributes.chunk_url,{chunk:n.attributes.chunk_index++}).success(function(p){var q;if(p.ck_data!==""){q=p}else{n.attributes.at_eof=true;q=null}o.resolve(q)});return o}});var e=Backbone.Collection.extend({model:b});var a=Backbone.View.extend({initialize:function(n){this.row_count=0;this.loading_chunk=false;new d({model:n.model,$el:this.$el})},expand_to_container:function(){if(this.$el.height()<this.scroll_elt.height()){this.attempt_to_fetch()}},attempt_to_fetch:function(o){var n=this;if(!this.loading_chunk&&this.scrolled_to_bottom()){this.loading_chunk=true;this.loading_indicator.show();$.when(n.model.get_next_chunk()).then(function(p){if(p){n._renderChunk(p);n.loading_chunk=false}n.loading_indicator.hide();n.expand_to_container()})}},render:function(){this.loading_indicator=$("<div/>").attr("id","loading_indicator");this.$el.append(this.loading_indicator);var r=$("<table/>").attr({id:"content_table",cellpadding:0});this.$el.append(r);var n=this.model.get_metadata("column_names"),s=$("<thead/>").appendTo(r),t=$("<tr/>").appendTo(s);if(n){t.append("<th>"+n.join("</th><th>")+"</th>")}else{for(var q=1;q<=this.model.get_metadata("columns");q++){t.append("<th>"+q+"</th>")}}var p=this,o=this.model.get("first_data_chunk");if(o){this._renderChunk(o)}else{$.when(p.model.get_next_chunk()).then(function(u){p._renderChunk(u)})}this.scroll_elt.scroll(function(){p.attempt_to_fetch()})},scrolled_to_bottom:function(){return false},_renderCell:function(q,n,r){var o=$("<td>").text(q);var p=this.model.get_metadata("column_types");if(r!==undefined){o.attr("colspan",r).addClass("stringalign")}else{if(p){if(n<p.length){if(p[n]==="str"||p[n]==="list"){o.addClass("stringalign")}}}}return o},_renderRow:function(n){var o=n.split("\t"),q=$("<tr>"),p=this.model.get_metadata("columns");if(this.row_count%2!==0){q.addClass("dark_row")}if(o.length===p){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this)}else{if(o.length>p){_.each(o.slice(0,p-1),function(s,r){q.append(this._renderCell(s,r))},this);q.append(this._renderCell(o.slice(p-1).join("\t"),p-1))}else{if(p>5&&o.length===p-1){_.each(o,function(s,r){q.append(this._renderCell(s,r))},this);q.append($("<td>"))}else{q.append(this._renderCell(n,0,p))}}}this.row_count++;return q},_renderChunk:function(n){var o=this.$el.find("table");_.each(n.ck_data.split("\n"),function(p,q){if(p!==""){o.append(this._renderRow(p))}},this)}});var g=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);scroll_elt=_.find(this.$el.parents(),function(o){return $(o).css("overflow")==="auto"});if(!scroll_elt){scroll_elt=window}this.scroll_elt=$(scroll_elt)},scrolled_to_bottom:function(){return(this.$el.height()-this.scroll_elt.scrollTop()-this.scroll_elt.height()<=0)}});var m=a.extend({initialize:function(n){a.prototype.initialize.call(this,n);this.scroll_elt=this.$el.css({position:"relative",overflow:"scroll",height:this.options.height||"500px"})},scrolled_to_bottom:function(){return this.$el.scrollTop()+this.$el.innerHeight()>=this.el.scrollHeight}});var d=Backbone.View.extend({col:{chrom:null,start:null,end:null},url_viz:null,dataset_id:null,genome_build:null,file_ext:null,initialize:function(p){var s=parent.Galaxy;if(s&&s.modal){this.modal=s.modal}if(s&&s.frame){this.frame=s.frame}if(!this.modal||!this.frame){return}var o=p.model;var r=o.get("metadata");if(!o.get("file_ext")){return}this.file_ext=o.get("file_ext");if(this.file_ext=="bed"){if(r.get("chromCol")&&r.get("startCol")&&r.get("endCol")){this.col.chrom=r.get("chromCol")-1;this.col.start=r.get("startCol")-1;this.col.end=r.get("endCol")-1}else{console.log("TabularButtonTrackster : Bed-file metadata incomplete.");return}}if(this.file_ext=="vcf"){function q(u,v){for(var t=0;t<v.length;t++){if(v[t].match(u)){return t}}return -1}this.col.chrom=q("Chrom",r.get("column_names"));this.col.start=q("Pos",r.get("column_names"));this.col.end=null;if(this.col.chrom==-1||this.col.start==-1){console.log("TabularButtonTrackster : VCF-file metadata incomplete.");return}}if(this.col.chrom===undefined){return}if(o.id){this.dataset_id=o.id}else{console.log("TabularButtonTrackster : Dataset identification is missing.");return}if(o.get("url_viz")){this.url_viz=o.get("url_viz")}else{console.log("TabularButtonTrackster : Url for visualization controller is missing.");return}if(o.get("genome_build")){this.genome_build=o.get("genome_build")}var n=new f.IconButtonView({model:new f.IconButton({title:"Visualize",icon_class:"chart_curve",id:"btn_viz"})});this.setElement(p.$el);this.$el.append(n.render().$el);this.hide()},events:{"mouseover tr":"show",mouseleave:"hide"},show:function(s){function r(x){return !isNaN(parseFloat(x))&&isFinite(x)}if(this.col.chrom===null){return}var w=$(s.target).parent();var t=w.children().eq(this.col.chrom).html();var n=w.children().eq(this.col.start).html();var p=this.col.end?w.children().eq(this.col.end).html():n;if(!t.match("^#")&&t!==""&&r(n)){var v={dataset_id:this.dataset_id,gene_region:t+":"+n+"-"+p};var q=w.offset();var o=q.left-10;var u=q.top-$(window).scrollTop()+3;$("#btn_viz").css({position:"fixed",top:u+"px",left:o+"px"});$("#btn_viz").off("click");$("#btn_viz").click(this.create_trackster_action(this.url_viz,v,this.genome_build));$("#btn_viz").show()}else{$("#btn_viz").hide()}},hide:function(){this.$el.find("#btn_viz").hide()},create_trackster_action:function(n,q,p){var o=this;return function(){var r={};if(p){r["f-dbkey"]=p}$.ajax({url:n+"/list_tracks?"+$.param(r),dataType:"html",error:function(){o.modal.show({title:"Something went wrong!",body:"Unfortunately we could not add this dataset to the track browser. Please try again or contact us.",buttons:{Cancel:function(){o.modal.hide()}}})},success:function(s){o.modal.show({title:"View Data in a New or Saved Visualization",buttons:{Cancel:function(){o.modal.hide()},"View in saved visualization":function(){o.modal.show({title:"Add Data to Saved Visualization",body:s,buttons:{Cancel:function(){o.modal.hide()},"Add to visualization":function(){o.modal.hide();o.modal.$el.find("input[name=id]:checked").each(function(){var t=$(this).val();q.id=t;o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})})}}})},"View in new visualization":function(){o.modal.hide();o.frame.add({title:"Trackster",type:"url",content:n+"/trackster?"+$.param(q)})}}})}});return false}}});var l=function(q,o,r,n){var p=new o({model:new q(r)});p.render();if(n){n.append(p.$el)}return p};var c=function(p){if(!p.model){p.model=new i(p.dataset_config)}var o=p.parent_elt;var q=p.embedded;delete p.embedded;delete p.parent_elt;delete p.dataset_config;var n=(q?new m(p):new g(p));n.render();if(o){o.append(n.$el);n.expand_to_container()}return n};return{Dataset:b,TabularDataset:i,DatasetCollection:e,TabularDatasetChunkedView:a,createTabularDatasetChunkedView:c}}); \ No newline at end of file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Remove old comment.
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/cc7147514d77/ Changeset: cc7147514d77 User: jmchilton Date: 2014-12-17 17:32:50+00:00 Summary: Remove old comment. Affected #: 1 file diff -r 867635153252d0830dac64ba72ede3cb637e2abf -r cc7147514d77799989777fd21aa818e43d9f3d39 lib/galaxy/tools/parser/xml.py --- a/lib/galaxy/tools/parser/xml.py +++ b/lib/galaxy/tools/parser/xml.py @@ -254,9 +254,6 @@ assert_elem = output_elem.find("assert_contents") assert_list = None - # Trying to keep testing patch as localized as - # possible, this function should be relocated - # somewhere more conventional. def convert_elem(elem): """ Converts and XML element to a dictionary format, used by assertion checking code. """ tag = elem.tag Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/16d9f6306be0/ Changeset: 16d9f6306be0 User: jmchilton Date: 2014-12-17 16:01:16+00:00 Summary: Fixed for botched automated merge 78597d2. Makes me worry about all the automated merges I have done lately. Also - I hate you mercurial. Affected #: 1 file diff -r 78597d2278e28ac962b80179d15eacd5849a4c79 -r 16d9f6306be079016cbba3ebe9eac9c5a3937282 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -183,7 +183,8 @@ if isinstance( value, galaxy.tools.parameters.basic.DataToolParameter ): if not isinstance(param_value, list): param_value = [ param_value ] - processed_value = [ self.__add_uploaded_dataset( context.for_state(), v, param_extra, value ) for v in param_value ] + map( lambda v: self.__add_uploaded_dataset( context.for_state(), v, param_extra, value ), param_value ) + processed_value = param_value elif isinstance( value, galaxy.tools.parameters.basic.DataCollectionToolParameter ): assert 'collection' in param_extra collection_def = param_extra[ 'collection' ] https://bitbucket.org/galaxy/galaxy-central/commits/867635153252/ Changeset: 867635153252 User: jmchilton Date: 2014-12-17 16:02:45+00:00 Summary: Merge. Affected #: 1 file diff -r 16d9f6306be079016cbba3ebe9eac9c5a3937282 -r 867635153252d0830dac64ba72ede3cb637e2abf cron/cleanup_datasets.py --- a/cron/cleanup_datasets.py +++ b/cron/cleanup_datasets.py @@ -1,63 +1,8 @@ #!/usr/bin/env python -#This script removes deleted dataset files. -#Takes 3 arguments: -# 1: database directory to clean -# 2: postgres database name -# 3 (optional): number of days to allow as a buffer, defaults to 2 -#python cleanup_datasets.py /home/universe/server-home/wsgi-postgres/database/files/ galaxy_test 2 +import sys -import sys, os, tempfile, time -try: - database_dir = sys.argv[1] - database_name = sys.argv[2] - num_days = 2 - try: - num_days = int(sys.argv[3]) - except: - print "Using Default of 2 days buffer on delete" -except: - print "Usage: python %s path_to_files:/home/universe/server-home/wsgi-postgres/database/files/ database_name:galaxy_test [num_days_buffer:2]" % sys.argv[0] - sys.exit(0) -id_file = tempfile.NamedTemporaryFile('w') -id_filename = id_file.name -id_file.close() -ids = [] +print ("This script has been deprecated, replaced by the set of scripts in <galaxy_distribution>/scripts/cleanup_datsets/." + "See https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%2… for more information.") -command = "psql -d %s -c \"select id from dataset;\" -o %s" % (database_name, id_filename) -print "Getting IDs:", command -id_file = os.popen(command) -id_file.close() -for line in open(id_filename,'r'): - try: - ids.append(int(line.strip())) - except: - print line.strip(),"is not a valid id, skipping." -os.unlink(id_filename) -if len(ids) < 1: - print "Less than 1 IDs have been found! Deleting proccess has been canceled." - sys.exit(0) -print "-----%i IDs Retrieved -----" % len(ids) -print "----- Checking database directory for deleted ids: %s -----" % database_dir -file_size = 0 -num_delete = 0 -for result in os.walk(database_dir): - this_base_dir,sub_dirs,files = result - for file in files: - if file.startswith("dataset_") and file.endswith(".dat"): - id = int(file.replace("dataset_","").replace(".dat","")) - file_name = os.path.join(this_base_dir,file) - if id not in ids: - file_time = os.path.getctime(file_name) - if time.time() > file_time + (num_days*60*60*24): #num_days (default=2) days buffer room - num_delete += 1 - size = os.path.getsize(file_name) - file_size += size - os.unlink(file_name) -print file_size, "bytes" -print float(file_size) / 1024, "kilobytes" -print float(file_size) / 1024 / 1024, "Megabytes" -print float(file_size) / 1024 / 1024 / 1024, "Gigabytes" -print "%i files deleted" % num_delete - -sys.exit(0) \ No newline at end of file +sys.exit(1) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: Delete *ancient* cron/cleanup_datsets.py content, but leave a nice deprecation warning and pointer for those who might actually be using it in cron jobs.
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/7e7a0c518d50/ Changeset: 7e7a0c518d50 User: dannon Date: 2014-12-17 16:00:29+00:00 Summary: Delete *ancient* cron/cleanup_datsets.py content, but leave a nice deprecation warning and pointer for those who might actually be using it in cron jobs. Affected #: 1 file diff -r 78597d2278e28ac962b80179d15eacd5849a4c79 -r 7e7a0c518d503cfc489dcb1fd8b1b9a113ed0e88 cron/cleanup_datasets.py --- a/cron/cleanup_datasets.py +++ b/cron/cleanup_datasets.py @@ -1,63 +1,8 @@ #!/usr/bin/env python -#This script removes deleted dataset files. -#Takes 3 arguments: -# 1: database directory to clean -# 2: postgres database name -# 3 (optional): number of days to allow as a buffer, defaults to 2 -#python cleanup_datasets.py /home/universe/server-home/wsgi-postgres/database/files/ galaxy_test 2 +import sys -import sys, os, tempfile, time -try: - database_dir = sys.argv[1] - database_name = sys.argv[2] - num_days = 2 - try: - num_days = int(sys.argv[3]) - except: - print "Using Default of 2 days buffer on delete" -except: - print "Usage: python %s path_to_files:/home/universe/server-home/wsgi-postgres/database/files/ database_name:galaxy_test [num_days_buffer:2]" % sys.argv[0] - sys.exit(0) -id_file = tempfile.NamedTemporaryFile('w') -id_filename = id_file.name -id_file.close() -ids = [] +print ("This script has been deprecated, replaced by the set of scripts in <galaxy_distribution>/scripts/cleanup_datsets/." + "See https://wiki.galaxyproject.org/Admin/Config/Performance/Purge%20Histories%2… for more information.") -command = "psql -d %s -c \"select id from dataset;\" -o %s" % (database_name, id_filename) -print "Getting IDs:", command -id_file = os.popen(command) -id_file.close() -for line in open(id_filename,'r'): - try: - ids.append(int(line.strip())) - except: - print line.strip(),"is not a valid id, skipping." -os.unlink(id_filename) -if len(ids) < 1: - print "Less than 1 IDs have been found! Deleting proccess has been canceled." - sys.exit(0) -print "-----%i IDs Retrieved -----" % len(ids) -print "----- Checking database directory for deleted ids: %s -----" % database_dir -file_size = 0 -num_delete = 0 -for result in os.walk(database_dir): - this_base_dir,sub_dirs,files = result - for file in files: - if file.startswith("dataset_") and file.endswith(".dat"): - id = int(file.replace("dataset_","").replace(".dat","")) - file_name = os.path.join(this_base_dir,file) - if id not in ids: - file_time = os.path.getctime(file_name) - if time.time() > file_time + (num_days*60*60*24): #num_days (default=2) days buffer room - num_delete += 1 - size = os.path.getsize(file_name) - file_size += size - os.unlink(file_name) -print file_size, "bytes" -print float(file_size) / 1024, "kilobytes" -print float(file_size) / 1024 / 1024, "Megabytes" -print float(file_size) / 1024 / 1024 / 1024, "Gigabytes" -print "%i files deleted" % num_delete - -sys.exit(0) \ No newline at end of file +sys.exit(1) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Fix a74d320.
by commits-noreply＠bitbucket.org 17 Dec '14

17 Dec '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/05604a5d31c1/ Changeset: 05604a5d31c1 Branch: next-stable User: jmchilton Date: 2014-12-17 15:57:44+00:00 Summary: Fix a74d320. Affected #: 1 file diff -r a74d32014b0512613042c2a44d0b946703a1fbcd -r 05604a5d31c1b150d4a04347e68c0f30f2d0a598 lib/galaxy/tools/test.py --- a/lib/galaxy/tools/test.py +++ b/lib/galaxy/tools/test.py @@ -195,8 +195,7 @@ if isinstance( value, basic.DataToolParameter ): if not isinstance(param_value, list): param_value = [ param_value ] - map( lambda v: self.__add_uploaded_dataset( context.for_state(), v, param_extra, value ), -param_value ) + map( lambda v: self.__add_uploaded_dataset( context.for_state(), v, param_extra, value ), param_value ) processed_value = param_value elif isinstance( value, basic.DataCollectionToolParameter ): assert 'collection' in param_extra Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0