Significant Event Audit in Practice: A Preliminary Study Richard Westcott, Grace Sweeney and Jonathan Stead STRUCTURED ABSTRACT Background While well described and promoted as a useful activity, the process of Significant Event Audit (SEA) is not supported by research and the literature remains at a practice led, anecdotal and ‘quasi-theoretical’ level. Existing descriptions are grounded in neither the experiences of participants nor in any shared reality of the process. Objectives To identify participants’ perceptions of the benefits and problems associated with Significant Event Audit in the context of primary care, and to derive suggestions which might improve the process of SEA. Methods Semi-structured interviews of twelve participants from a variety of primary care disciplines, using grounded theory to analyse the results. Results A set of six perceptions and seven recommendations for the facilitation of SEA Conclusions SEA constitutes a powerful tool, which can contribute to team building, enhanced communication, improved patient care and represents a vital contributor to the development of clinical governance in primary care. However, its implementation and sustenance require sensitive handling for optimal benefit and to minimise difficulties. Our research has enabled us to propose suggestions to facilitate these processes. Keywords Significant Event Audit, Teambuilding, Quality improvement, Clinical Governance, Leadership. Introduction Significant Event Audit (SEA) has been defined as occurring when
The technique of SEA has been well described 1,2,3. But while a small number of research-based explorations of SEA in primary care have been undertaken 4,5, and there is evidence of the technique producing follow through into needs assessment and commissioning 6, the actual process and experience remain poorly described. There is little empirical evidence to support the particular model of SEA that has been advocated and widely adopted, and specifically, few efforts have been made to ascertain or evaluate:
In short, despite optimism about SEA, we have identified little evidence describing the benefits, problems and other experiences of this process in primary care. Aims In this preliminary study, we set about exploring the use of SEA within primary care teams. Specifically, we aimed to:
I Study Design & Methodological Approach The present study is placed firmly within the naturalistic paradigm, making use of the research interview as the strategy for collecting data, and of grounded theory as the method of data analysis. Grounded theory has two meanings7. First it involves the notion of grounding theory in experiences, accounts, and local contexts. The aim of the approach is seen as the generation of a ‘meaningful account’, which accurately represents the complex nature of the participants’ world. Second, the term ‘grounded theory’ is used to describe a particular method which involves specific analytical strategies formulated for handling and making sense of ill-structured qualitative data. Strategies within grounded theory are presented to the researcher as ‘aids to analysis’ rather than as methodological prescriptions. Glaser and Strauss8 describe the systematic development of an open-ended indexing system, in which the analyst works rigorously through the raw data, to generate codes that refer to both low-level concepts and more abstract categories. II Procedure (i) Data Collection Data was collected through a series of twelve one-to-one interviews, each of which lasted between 30 and 45 minutes. Absolute confidentiality was assured. Pools of potential participants were drawn from established SEA groups. Participants were recruited by approaching the Practice Manager in each of three geographically convenient primary care sites, who then provided a list of staff who were willing to participate. Subjects were selected to provide a representative from each of the occupational groups within the primary care team. With the participant’s consent each interview was tape-recorded. The interview schedule was designed to elicit descriptive and explanatory information representing the interviewee’s interpretation of the experience of SEA. Following an initial structured section (personal and situational variables and details about the practicalities of SEA), the interview was designed to be relatively unstructured. (iii) Participant Profile Study participants included two practice managers, two general practitioners, two practice nurses, two receptionists, one district nurse, one health visitor, one community psychiatric nurse, and one community physiotherapist. Ages ranged from 37 to 58 years (mean 46 years). Eight subjects were female and 4 were male. Only three participants working full time, the remainder working between 12 and 30 hours per week. Their length of time in post ranged from two to eighteen years (mean 8.6 years). (iii) Analysis of Data Initial indexing (‘coding’) proceeded by means of a tentative labelling of the phenomena which the analyst perceived in a specified piece of text and which was considered to be of potential relevance. The method employed to construct an indexing system was to examine each unit of text for analysis in turn, and ask ‘what categories, concepts, or labels do I need in order to account for the phenomena of importance in this paragraph?’ 9 It should be noted that within grounded theory, the aim is not to record all the reoccurrences of a phenomenon, but to collect a set of indicators describing the multiple facets of the concept. Initial coding resulted in the generation of 25 separate concept cards. At this stage in the analytical process it became apparent that many of the concepts were in fact describing different aspects of the same phenomena, and core analysis commenced. Separate concept cards were clustered under thirteen new headings, highlighting the most salient issues identified by the participants. These thirteen ‘higher order’ categories were then examined to further explore the underlying characteristics of each category. Saturation of categories occurred when any further analysis of this data failed to produce any additional examples or counter-examples of the interviewee’s experience of SEA. Results and Discussion Table 1 presents the distilled themes as described above. We now address each category in turn. Table 1: Results of grounded theory analysis grouped under two core headings
Perceptions of SEA by participants 1. Advantages Participants welcomed the opportunity, indeed permission, that SEA gave them to come forward with both problems and difficulties as well as suggestions and advice. They could relate such discussions to team building: the creation of trust, mutual understanding and appreciation of other members' contributions. As a result the work environment was enhanced and a better quality service could be offered, which itself further improved morale. They enjoyed the multidisciplinary format, offering opportunities to learn about others' experiences and opinions, to work together to problem solve and to resolve conflict, often without personalisation. Members could see how SEA encouraged individual development both within and alongside the team.
Some members - particularly those employed by the doctors and lower in the hierarchy - felt vulnerable in speaking out, especially if their contribution might be seen as critical of those perceived to be of higher status. SEA represented a new and often uncomfortable experience for most of the team (in general least so for the GPs), who found the critical process generally disconcerting and could be embarrassed by revelations of other members' shortcomings. There was a Pandora's Box fear: that lifting the lid might release uncontainable pressures with unexpected consequences. Finding the time for the meeting, especially when part time staff had to make special arrangements to attend, posed practical difficulties. And even when this could be resolved, these members often found they had to carry straight on at work, or to go home, without adequate opportunity to talk things over.
3. Concerns Members were uncertain about boundaries and anxious not to overstep margins. While the traditional demarcated roles within primary care might (and many could understand ought) to be temporarily abandoned during the SEA meeting, it might be difficult to return to the ancien regime immediately afterwards - until next time. Employed staff in particular worried about who was to lead SEA: without sufficiently strong and sympathetic chairing they felt vulnerable. All staff feared the Hornets' Nest: it may be better to let some things be. And even when some difficult issues can be tackled, they may not be adequately resolved: 'we still have to work together'. As a result, SEA sometimes confined itself to safe areas, which were trivial. Non doctors were concerned that GPs' topics might dominate the agenda.
4. Conflicts Employed staff felt a conflict in roles: they found it difficult to behave as equals during the meeting and then to return to 'employee status'. If the GP is your employer, it can be hard to speak out honestly. Other, externally employed, staff were differently stressed, with loyalties to the GPs and their own management structure sometimes in conflict. Some members (particularly nurses) felt their first loyalty was to their own staff group or discipline, which made could interfere with SEA. There could be a conflict between personal and professional matters for various participants.
5. Motivation The selection of topics affected motivation. Because the leaders were more inclined to choose events that involved them, clinical GP topics could dominate SEA, particularly at first. This could alienate non-clinical staff. Certainly some members felt more motivated to contribute than others. GPs' motivation was increased by PGEA approval for attending (with no similar reward for other groups) and a social benefit. Others with less need for the social contacts felt less motivated. These might be receptionists, who were sometimes able to talk about their work problems as they worked, and so had less need for SEA. Because the initiative for SEA often came from the GPs, their motivation to attend, contribute and make the meetings successful was greater than other members, who occasionally saw the exercise as imposed. There was a danger of a vicious circle developing, with those attending dominating the proceedings and thus making SEA even less attractive for the non-attenders. It was particularly important for 'external' attached workers (community psychiatric nurses for example) to see relevant topics to motivate them to attend. However, it was appreciated that the discussion of an event by those involved was a motivating experience and this encouraged people to attend the next time.
6. Solutions and Resolutions The ability of SEA to solve problems and resolve awkward issues was generally appreciated. Guidelines for managing a variety of different situations could be discussed and recommended. People could be reassured they were 'on track' and an agreed direction of travel established, with the opportunity for everyone present to contribute. But the short time available could mean that quick, easy or superficial solutions might be adopted. And members recognised that it was sometimes difficult to be sufficiently honest.
Facilitating Significant Event Audit 1. Rules and Guidelines Participants need a clear set of rules, particularly at the outset. Specifically, managing emotional issues and those affecting absent team members requires agreed ground rules. With an initiative as new and as powerful as SEA, workers feel the need for a defined structure which can provide limits for protection, and safety. As SEA evolves and participants become more comfortable, the structure can change appropriately with, if necessary, new rules. Our experience has shown that in any event rules need to be revisited and reiterated regularly. Some participants are prepared to take a more pragmatic approach and develop rules with the activity: these are a minority and must be sensitive to the needs of the (possibly less outspoken) majority.
2. Ownership and Commitment Commitment can only be assumed if the participants' need for safety (which the ground rules will have identified) has been addressed. The team has to take responsibility for making it practicable for its members to attend. Full time workers may find it easier to commit themselves than part time, job sharing and lower paid workers who can be disadvantaged in this respect. This uneven playing field has to be recognised. If the team can address the particular needs of various individuals, providing appropriate cover or in lieu off-duty arrangements, loyalty to SEA meetings can be won. The evolution of Practice Professional Development Plans (PPDPs) 10 can contribute to formal acknowledgement of SEA for all participants, which will help build corporate ownership. Commitment also requires judicious selection of topics to ensure continued interest, especially in the early stages (see below).
3. Selection of Topics The needs of all team members must be borne in mind when the SEA agenda is drawn up. If one or more individuals can see little relevance in the topics, it is to be expected that their support will wane. In addition, a sensitive choice of topics is crucial in order to ensure generalisability: the risk of issues becoming personalised rather than universalised has to be remembered. Topics which can stimulate the group but which have the potential for resolution which can be dealt with safely are essential. Other problems with topic selection include the need for awareness of a hot topic becoming cold, items suggested only because no one else has thought of any ('scraping the barrel') and hidden agendas by one or more individuals. The important issue of individuals' rights to remove proposed topics is addressed separately (see below). Successful agenda construction (and appropriate modification as the meeting proceeds) is a vital determinant of SEA success and dependent on quality leadership (see below).
4. Leadership Skilful leadership is essential to give confidence and facilitate all these items. Less assertive team members depend upon effective leadership to overcome their hesitancy. The qualities expected from the SEA leadership are extensive, ranging from support and encouragement (before, during and after the actual meetings) (see Management of the Process below) through chairing to challenging, summarising, planning and debriefing. This set of qualities are probably not available in any one individual in a practice team, and certainly could seem to rule out rotating the leadership between several people, desirable as this may be. However the requirements for leadership are clear and have to be recognised. Once identified, they may be met by the team exploiting its own skill mix, a team building activity which is both contributory to and helped by the process of SEA. Ensuring a corporate responsibility for leadership is also needed to counteract the perennial problem of inequality, both perceived and actual, in the primary care team. Some suggested that the best placed individual, regardless of personal attributes, to co-ordinate if not provide most of this input, is the practice manager. Certainly we were made aware of the difficulties flowing from the often automatic assumption of doctor leadership. We encountered gender issues here: participants suggested that female approaches to the challenge of leadership might in general represent a more appropriate model, particularly with regard to ongoing care (see Debriefing below).
5. Debriefing There is an under recognised need to offer time and support to individuals or groups after SEA meetings. While the ideal is to address and complete all outstanding issues (particularly emotional) within the meeting, it has to be acknowledged that participants are left with unresolved matters which may be carried on into their next clinical session which might follow immediately. Just as individual learning requires a period of reflection, so too do groups: SEA meetings need to devote attention to this need. Whether individual debriefing should be the responsibility of the leader, the practice manager, a defined group or the whole team, and when and how it is addressed, is less important than the recognition of its need.
6. Censoring and Vetting Some topics are inappropriate for SEA. We have learnt that individual poor performance needs very sensitive and intimate handling which should not be addressed in a SEA forum. Less professional but no less personal matters (for example personal hygiene) are also obviously better dealt with elsewhere. Important as they may be, and indeed needful of attention, there will be confidential (staff health for example), contractual, sex discrimination and other issues which, if the involved individuals do not have the right to strike them from the agenda, can cause severe personal stress and disruption to SEA, if not the practice itself.
7. Management of the Process The whole process of SEA then requires ongoing supervision. The meetings themselves are but part of a continuing developmental spiral, characterised by discussions and preparation leading to SEA which produces actions, review, reflection and reinforcement with perhaps new approaches. All these require active management - management of the practice itself, management of people and management of change. A delicate balance has to be struck between the provision of safety and support, and the stimulation of challenge to improve and ensure quality in all its aspects - a dynamic task, which itself will evolve. So there has to be adequate opportunity for the team to reflect upon the pace of change, the methods chosen and the progress achieved, and especially the failures - where SEA may be used to audit the process of SEA. Those implementing SEA must present practices with the chance to review, as part of the process. This could be addressed through dedicating a meeting at regular intervals to stocktaking, or to join with one or more other practices to share experiences, or to contribute to a workshop at Authority level. Teams may or may not want to use their own resources with or without imported experts. Possibly the evolving role of clinical governance might facilitate such developments, with its emphasis on joint working at inter- and supra- practice level 11.
Conclusions Many of the benefits of SEA, such as its ability to stimulate clinical audit and needs assessment, to inform commissioning and improve quality, have been well documented1,2,6. In terms of the process, it can be seen that SEA represents a powerful team building experience for primary care team members, capable of involving all members in a multidisciplinary approach and creating better morale. All members can appreciate the individual benefits along with the better communication, improved mutual understanding, happier work environment and the resulting enhanced patient care. These are important contributions to the well functioning primary care team 10, 11 demanded by today’s National Health Service 11. But there are substantial difficulties, not previously documented, which can not only prevent the successful implementation of SEA but which can alienate individuals and cause damage to teams. Members fear exposure, find it difficult to step out of role, worry about causing offence (especially to GPs who may be their employers) and need sensitive encouragement based upon an awareness of these various anxieties. However, by establishing clear rules, ensuring general ownership, carefully selecting the right topics and using good leadership skills, allowing for proper support and protecting individuals, SEA may be successfully implemented. This qualitative study has shown that teams and their members have derived many personal, professional and corporate benefits from SEA. The new emphasis upon Clinical Governance 11 requires structures and processes which can create and sustain a framework for individual professionals and teams to respond to the new agenda for quality improvement. Our research leaves us in no doubt that SEA represents a crucially important tool for primary care teams. As with all new tools, both training and support are needed to ensure optimal use and indeed prevent damage. But by following our suggestions for facilitating the process, we believe teams will be able to introduce and maintain SEA as a powerful opportunity to strengthen teamwork, enhance quality and improve patient care, its benefits far outweighing its disadvantages. Acknowledgements We are grateful to our colleague Liz Cosford, and to Professor Mike Pringle for their helpful comments on earlier drafts of this manuscript. We record with thanks funding from the Somerset and North Devon Education Purchasing Consortium, and acknowledge support of Research and Development funding of Wyndham House Surgery, Silverton as a Research Practice. Conflict of Interests None. References 1 Pringle M., Bradley CP, Carmichael CM., Wallis H, Moore A. Significant Event Auditing. Occasional Paper 70. Exeter: Royal College of General Practitioners,1995.2 Pringle M, Bradley A. Significant Event Auditing: A user’s guide. Audit Trends 1994;2:70-73.3 Robinson LA, Stacy R., Spencer JA, Bhopal RS. Use facilitated case discussions for significant event auditing. British Medical Journal 1995;311:315-318.4 Benett IJ, Danczak AF. Terminal care: improving teamwork in primary care using Significant Event Analysis. European Journal of Cancer Care 1994;3:54-57.5 Berlin A, Spencer JA, Bhopal RS, Zwanenberg van TD. Audits of deaths in general practice: pilot study of the critical incident technique. Quality in Health Care 1992;1:231-235.6 Pringle M. Preventing ischaemic heart disease in one general practice: from one patient, through clinical audit, needs assessment, and commissioning into quality improvement. British Medical Journal 1998,317,1120-11247 Strauss AL, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, C.A.: Sage Publications Ltd, 1990.8 Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine, 1967.9 Turner BA, Some practical aspects of qualitative data analysis: one way of organising the cognitive processes associated with the generation of grounded theory. Quality and Quantity, 1981, 15, 225-247.10 Chief Medical Officer. A Review of Continuing Professional Development in General Practice. London: Department of Health, 1998.11 Department of Health. A First Class Service. London: Department of Health, 1998. |