COMMENTARY

Evaluation, Feedback And Impact Of Programmes And Activities In The Civil Service : The Case of Training and Quality Initiatives

Public managers, through their respective institutions, almost always are assigned or given tasks, programmes and activities with set objectives and targets to be met. The implementation of development programmes and projects such as those in RM7 and RM8 are often measured using performance indicators - be it physical or financial measurements. If you can recall, Part Three of this Commentary dwells on the topic entitled “ Project Implementation, Monitoring and Evaluation”. Corollary to that it is also relevant to focus on the sometimes forgotten aspect of evaluation, feedback and impact of the myriad programmes and activities, other than the development programmes and projects, undertaken by the various government agencies.

For example, how do we know and measure the impact of our development effort and activities apart from merely monitoring the financial and physical achievements? How do we fathom the impact of these programmes and projects on the clients and ultimate beneficiaries? In the implementation of development programmes and projects, measuring physical and financial results poses little difficulty. The crunch, however, comes when we begin to measure societal change and the non-physical elements. A good example is the impact of the programmes in the area of education and training for members of the public service, and the various Quality initiatives to improve service delivery to our clients.

For purpose of this commentary, we shall focus on two areas - training and quality initiatives - both aimed at impacting on the performance of the civil service. Much has been said about transforming the public service into an erudite and intellectual institution, the quality drivers and initiatives toward embedding the culture of excellence, professionalism, the learning organisation or on peak performance. Yet we know that it takes all these criteria and many more change efforts into account when we want to transform the civil service. Albeit the manifold programmes being launched, the cause for concern will continue to be one where there still exists gaps between what we want to achieve and what we actually achieved in real terms.

As far as training is concerned, we have a long list of programmes and courses designed to transform public servants : COSEC, Managerial, Supervisory and Employee Grid, AMP at various institutions of higher learning, and of late the SDP, MDP and the Balanced Scorecard. All these training courses are good courses and their contents are very up-to-date. Congratulations for the initiatives and foresight. While these trainings are ongoing, it is posited that our early evaluation is useful for at least four reasons. Firstly, to signal decisions about the future use of a training programme or technique. Secondly, to remind ourselves to continually ensure that the competency of trainees is addressed. Next, to contribute to a scientific understanding of the training process and lastly, for public relations purposes. The first two reasons seem to be areas that need continuous improvement and upgrading. This commentary is more concerned with the second reason i.e. the competency of the trainer in producing quality trainees.

Metaphorically speaking, trainees who are attending courses are akin to being placed on a “conveyor belt” or “assembly line”. One has also to ponder whether the emphasis of production is on quantity or quality. One also has to identify defective products before the end of the assembly line. This is where trainers cannot be of a mediocre standard. Top class trainers will almost invariably churn out outstanding trainees. In essence, the standard of the trainers must not be compromised. The issue here is to ensure the high standard of performance of trainers or resource persons tasked to deliver these core, support and special programmes.

Resource persons in training must be a “complete” trainer. He has to be articulate, experienced as well as a good communicator. By “articulate” we mean the trainer must possess both oratory and literate skills. By “experienced” we mean the trainer to possess maturity of thought laced with intellectual rigour, and an in-depth knowledge of the subject that he or she is going to deliver. By “good communicator” we mean the trainer can deliver his or her ideas in a convincing manner to the audience so that the desired impact of learning will ultimately benefit the trainees.

What does it take to be a competent resource person? Undoubtedly, one has to be analytical and possesses a probing mind. Trainers have to be tested and to reassess themselves, especially in the ratings given by course participants. A trainer must not only be equipped with training skills and techniques but can spontaneously unleash a repertoire of skills and explain the underpinnings of theories, perceptions and approaches of the topics that they are going to deliver. Another important asset for the trainer is being well-read. It goes without saying that only a well-researched trainer can rationalise and explain his arguments explicitly, explicatively and expeditiously to the participants. The trainer is not supposed to evangelise the course content only. The trainer’s oratory skills need to be critically assessed too. A trainer may be a champion toastmaster but it does not necessarily mean that he has an in-depth knowledge of the topic that he is about to deliver.

Let us now dwell on the recurring phrases of “myths and/or realities” in the contact of evaluation. A myth is a commonly-held belief that is not true or without foundation. Thus the assumption that if one has obtained a certificate, diploma, degree or postgraduate degree is adequate for the tasks to be implemented, is of course a myth. Using a jargon employed in development economics, obtaining a certificate in training is a “necessary but not sufficient condition” to become a quality trainer. Another common myth is participants’ opinion in evaluating a training programme. Since such evaluation is based on opinion rather than fact, in reality the trainee may not have learned much nor indepth, but perceives that a learning experience has occurred.

So much about the myths surrounding evaluation, let us now turn to the realities of training evaluations. As one writer defines it, “evaluation of a training programme implies a dichotomous outcome......to assess outcomes we need to document systematically how trainees actually behave back on their jobs, and the relevance of their behaviour to the objectives of the organization”. Therefore, a good practice in evaluation would be for Heads of Departments to insist on officers who have undergone training to brief, debrief and share their learning experiences with their fellow officers. He or she who has been trained accordingly should also become a “reference point” to their colleagues in his or her workplace. Each and everyone who has had been trained has a responsibility to put into practice the newly-acquired knowledge and skills to achieve peak performance for their own organization. This is in line with the State Secretary’s views on the sharing of knowledge as one of the sources of empowerment.

In essence, training is but a good example of where departmental and agency heads should initiate evaluation tools to gauge the performance of their staff who have undergone training. They can do this by getting the right feedbacks from them. If such data are compiled accordingly over the years, trends may emerge and impact of training on organisational culture, organisational learning and behavioural change can assist Human Resource Development managers to focus scientifically and decide on the right mix of courses and the production of competent trainers.

As far as the implementation of the quality programmes and initiatives are concerned, it is heartening to note that the quality bandwagon is steaming ahead. Close to 38 state agencies have been evaluated by assessors, chiefly MAMPU, as worthy recipients of world-class ISO 9000 quality system. Certification aside, it is incumbent for ISO 9000 certified organizations to continuously evaluate their quality systems through surveillance audits on a regular basis, including periodic internal auditing and management review meetings. Such evaluation and constant feedback from customers can only ensure one thing - excellence of service delivery.

To conclude, evaluation, feedback and impact of programmes and activities in the civil service must be accorded topmost attention. Otherwise, there is no way that stakeholders can ascertain the value-added benefits that the programmes have engendered. The individual agencies must be regularly vigilant and committed to internalize their concern for evaluation, feedback and impact of departmental programmes and activities, albeit the mysticism associated with such appraisal mechanisms. Thus we must draw lessons associated with evaluation in training and quality initiatives to other non-physical activities that we implement in the civil service like events management and what have you.