Australian Palliative Care Outcome Collaboration (PCOC) phases: cross cultural adaptation and psychometric validation for Polish palliative settings

Abstract

Background

Measuring palliative care quality requires the application of evaluation methods to compare clinically meaningful groups of patients across different settings. Such protocols are currently lacking in Poland. The Australian Palliative Care Outcome Collaboration (PCOC) concept of Palliative phases precisely defines patients, enables episodes of care extraction for benchmarking and further assessment of service delivery. The present study is aimed at developing cross-cultural adaptation and psychometric validation for a Polish translation of Palliative phases.

Methods

Forward and backward translation was performed to obtain a Polish draft version regarding definitions of the PCOC phases. The draft was then subjected to linguistic and graphical transformations in the process of cognitive interviewing. The acceptability of the Polish version was assessed based on staff perceptions of fit, ease of assignment and familiarity with the patient’s and family’s situation. Finally, cross-sectional analysis was conducted among 313 hospice and home-care palliative patients. The attending doctor and nurse independently evaluated the same patients using the Polish version of the PCOC phases to establish inter-rater reliability values. Then, to determine its construct validity, the PCOC indicators were referred to patients’ prognosis, functioning level and PALCOM scale scores.

Results

A Polish draft version of the PCOC phases was prepared. Seven of the 13 interviewees reported problems with comprehending this proposal. This prompted changes being made to linguistic and graphical aspects of the tool. The majority of respondents preferred the final graphical scheme of phases, prepared after round-two of interviews. Scheme application acceptability was confirmed in a practical trial. The respondents’ overall conviction regarding degree of fit and assignment ease was high. Seventy percent of PCOC phase assignments was in agreement, and a moderate level or inter-rater reliability was obtained (kappa 0.573). The highest proportion of totally bed-bound patients with the shortest survival prognosis was observed for the terminal phase, while the highest complexity of palliative care needs was noted for the unstable one.

Conclusions

The PCOC phases tool appears to be a valuable resource for specialists in palliative care settings to support audit measures. Practical training is recommended prior to its implementation in routine practice.

Background

Health care aims to ensure universal access to safe, secure, and dignified care, with focus on symptom palliation, especially at the end-of-life phase. Serious health-related suffering that needs to be alleviated with medical intervention appears to be a universal problem [1]. Due to the enormous and escalating global burden of severe suffering [2], it is well recognized that at least two palliative care levels should be provided: an approach to non-specialized settings and for specialist care [3]. The level, form and intensity of palliative care should be objectively determined to achieve accessibility of the most adequate support, that preferred by patients and their family, delivered at an optimal time and in the best setting [4]. This process requires strict definition of palliative care patients, particularly in elders strongly affected by non-cancer conditions [5].

Defining a ‘palliative care patient’ is an ongoing process requiring clarity and practicality [6]. Basically, admission criteria to specialist palliative care are based on patients’ limited prognosis and their complex needs [7]. These discriminants should preferably be used in clusters, through various specific tools. One of the recently developed tools is the Complexity Scale of Palliative Care Needs (PALCOM) [8], which allows to emphasize the value of problem complexness. Another, complementary description perspective on palliative patients through defined periods of care has been found in the Australian system. A simple clinical assessment tool, based on the situation of the patient & family’s (recognized as a unity) care needs and the suitability of the current care plan, was proposed by the Australian Association for Hospice and Palliative Care through the concept of “Palliative Care Phase” in 1993. Since then, it has been revisited twice and also validated [9].

Five phases were distinguished in this concept: ‘stable’, ‘unstable’, ‘deteriorating’, ‘terminal’ and ‘bereavement’. Each phase describes both the patient’s and his or her caregiver’s situation, focusing on personal goals, needs and priorities. The phases are not linear and can alternate multiple times, focusing on the patient’s and family’s evolving needs rather than solely on disease progression. These phases are significantly associated with the patient’s functional status, personal needs, and also with the resource used [10]. This approach can be applied to address clinical interventions based on quality measures and to potentially underpin value-based reimbursement strategies in palliative care. Characteristics of phase changes (e.g. duration of ‘unstable’ one) can be considered as a quality care indicator for palliative services or even as one of the national benchmarks [11]. These phases can be also regarded as one of the elements of the complex adaptive system which help differentiate patients for whom a more elementary approach is sufficient from those in need of more intensive specialist palliative care [12]. In Australia, the Palliative Care Phase concept is universally recognized as a common unit of counting, enabling the national benchmarking assessment of care quality through symptom burden, problem severity, performance activity impairment and also resource utilization dependency.

Building on these phases, a detailed and comprehensive assessment framework was proposed to define separate episodes of care. This outline includes problem assessment (using the Symptom Assessment Scale and Problem Severity Score), evaluation of physical activity (through the Australia-modified Karnofsky Performance Status) and assessment of staff workload (via the Resource Utilisation Group—Activities of Daily Living). The implementation of routine assessments and universal outcome measurements has been shown to enhance patient experiences and improve care outcomes [13]. In Taiwan, such a complex assessment was implemented in 2020, enabling categorization of hospice facilities depending on the patient’s condition and the resources of the healthcare facility [14]. As a consequence of this intervention, a four-fold decrease in the risk of hospital deaths was noted (OR = 0.26, 95% CI, 0.16–0.41, P < 0.001).

The Polish reimbursement system supports palliative and hospice care for patients with a certain few, strictly defined diseases, which are incurable, progressive, life-limiting or unresponsive to disease-modifying therapy (i.e. mainly cancers) [15]. However, neither strict medical criteria for specialist care nor do evidence-based systems used to differentiate admission demands for generalist and specialist care exist [7]. Moreover, there is no national consensus or guidelines on routine outcome measurements common to different palliative care settings. Quality indicators of care are rarely used and if so, only in some institutions.

In this study, we took an initial step towards establishing quality and outcome measurements in palliative care. By adapting the Australian Palliative Care Outcome Collaboration (PCOC) phases to Polish conditions, we proposed a national palliative case-mix system for use in comparative analyses and reimbursement frameworks. Additionally, this study aimed to confirm the relationships between specific PCOC phases and patient performance levels or distinct needs.

Methods

The mixed method approach encompassed two stages: cross-cultural adaptation and psychometric validation of the PCOC phases.

Cross-cultural adaptation

Procedure and participants

After obtaining consent from the PCOC, forward and backward blind translation of the Australian phase definitions [16] was performed by a native speaker and medical staff experienced in palliative care, with two expert reviews (ABJ and TG) after each translation. This translation followed the guidelines outlined by Koller et al. [17].

As the next step, interviews were conducted with all key medical staff from the 45-bed, in-patient, free-standing palliative care unit. These staff members were experienced in palliative care, and the interviews were performed by ABJ, a registered nurse with 30 years of specialized experience in the field. The interviews were based on cognitive interviewing techniques and conducted iteratively in two rounds, as outlined by Lehmann et al. [18]. This process adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ – see ‘Additional File 1’). [19]. For this part of the study, the participants were chosen to cover various professional views on palliative care (physicians, nurses, psychologists and social workers) and were recruited personally within the institution. Each participant who agreed to participate in the study was interviewed twice, for approximately 30 min. During both interviews they were initially asked to ‘think aloud’ while reading each phase definition, then received specific questions about the general understanding and precision in defining the particular phase, and suggestions for any improvement or modifications (language or graphical). An open question about the whole phase description ended each part of all the interviews. None of the interviews need to be repeated. Neither audio nor video recordings required any particular coding software; instead, the interviewer took notes during each interview to derivate the emerged themes, which were summed up at the end of the meeting. Participants were also asked to provide feedback on the findings. An item-by-item phrases analysis was performed, leaving those which were understandable and universally accepted until conviction was obtained on sufficient saturation.

The original phase model was also the subject of graphical transformations (ABJ, EW, and TG) based on coded findings within a confirmatory analysis. The table and graphical synthesis documents obtained after the first round of the interviews was checked on the same model during the second round.

Acceptability

The acceptability of the final version of the Australian phase definitions obtained after the second round of interviewing was based on the staff’s perception. For this purpose, each participant was asked to assess 25 patients who were consecutively admitted. Three questions were asked, which were to be answered on a five-point scale regarding:

• 1) the accuracy of the current patient & family’s situation described by the particular phase (How appropriately does the assigned phase definition describes the current situation?; with answers from 1—‘not appropriate’ to 5—‘very appropriately’),

• 2) difficulty in assigning the situation to the Palliative phase (How difficult it was to assign the current phase?; with answers from 1—‘very difficult’ to 5—‘ very easy’), and also

• 3) the knowledge of the patient & family’s situation (How familiar are you with this situation?; with answers from 1—‘unfamiliar’ to—‘ very familiar’).

Psychometric validation – cross-sectional study

Procedure, setting and participants

A study of cross-sectional psychometric validation was performed in compliance with Strengthening the Reporting of Observational Studies in Epidemiology (STROBE – see Additional File 1) [20]. Patients with life-limiting diseases, predominantly cancer, consecutively admitted to the palliative care unit within a free-standing hospice and hospice home-care (within the same institution), were included in this part of the study. Patient characteristics, including gender, age, primary diagnosis and disease stages, identified by the attending physician using Gold Standard Framework needs-based coding (GSF) [21], were obtained from medical records.

The attending doctor and nurse were taught; within one 30-min session on the practical usage of PCOC phases concept and were equipped with both tabular and graphical synthesis documents. They were asked to independently (without assessment comparisons among themselves) assign the PCOC phase to the same patient & family, on the same day indicated by the researcher (MK), within the first two weeks following the patient’s admission. In parallel with assessment of the PCOC phases, = advancement of other illnesses and prognosis indicators were obtained from the medical records of all the chosen patients.

Concurrent validity measures

The palliative care phases defined as from ‘stable’ to ‘terminal’ according to PCOC [16] (the researchers did not take the ‘bereavement phase’ into account in this study) were compared with patients’ functional status, stage of the disease and their complex needs.

The patient’s current functional level and prognosis were estimated according to the Palliative Performance Scale Version 2 (PPS)—in the Polish adaptation [22]. The PPS was a modified Karnofsky Performance Scale which was developed by the Victoria Hospice Society [23] and later validated [24]. In our study, we used the Polish adaptation of the original second version of the PPS [22]. It consists of 11 levels expressed in percentages from 0 (deceased) to 100% (fully ambulatory and healthy with maximum performance state), taking five functional dimensions into account: ambulation, activity level, evidence of disease, self-care, oral intake and level of consciousness. This tool can be used to demonstrate the track of physical decline which, in consequence, may facilitate communication between staff members, timely palliative care admissions, and also end-of-life discussions in advance.

Illness advancement and spectrum of needs were assessed by using the PALCOM scale [8]. It is a five-domain multidimensional tool. It was developed to identify the specific complexness and needs of the patient & family. It has recently been validated for advanced cancer patients [25, 26]. It consists of introductory surprise questions and five domains of multidimensional assessment: symptom burden, refractory pain, performance status, socio-familial risk and existential/ethical issues. Each domain is scored dichotomously: 0 – ‘absence’ or 1 – ‘presence’, and the sum (from 0 to 5) is the total score of this scale. The final score of 0–1 point expresses ‘low complexity of palliative care needs’, 3–4 points – ‘medium level’, and 5–6 points—‘high complexity level’. In the present study, the Polish adaptation of the PALCOM scale was used [27].

Data collection took place from November 2023 to June 2024.

Statistical analysis

The minimal sample size of 300 patients was determined according to Lehman et al. [18]. The categorical variable data were presented as proportions, and the continuous variables as means (with standard deviations, SD) in normally distributed data (according to the Shapiro–Wilk test), or medians (with interquartile 25%-75% ranges). The inter-rater reliability was estimated using Cohen’s kappa statistics, with the strength of the agreement assumed according to Masso et al. [28]. The Chi square test was used to check the relationships between patients’ performance status and estimated prognosis with PCOC phases. The Dunn’s test (post-hoc for Kruskal–Wallis test) was applied to compare complex palliative care needs with PCOC phases. The P-value of < 0.05 was considered the level of statistical significance. Data were analyzed using the R program (version 4.2.2), a language and environment for statistical computing (Vienna, Austria).

All methods were performed in accordance with the 1964 Declaration of Helsinki, and participants provided informed consent for obtaining data and anonymous publication. The ethical approval was obtained by the research ethic committee of Andrzej Frycz Modrzewski Krakow University (reference number: KBKA/45/O/2023).

Results

Forward–backward translation

As a result of translation, a draft Polish version of the PCOC phases was prepared (see ‘Additional File 2’), which was used for the further interviewing process.

Cognitive interviews

Thirteen of the 14 invited professionals agreed to participate in the interviews. The demographic details are shown in Table 1.

Table 1 Demographics of 13 medical staff participants

In the first round of interviews, the general sense of the Polish PCOC phases translation was assessed. During this process, seven of the 13 respondents reported experiencing comprehension problems. All participants initially tended to concentrate solely on the patient’s situation (ignoring the problems of their carers). The definition of at least one phase seemed to be not fully clear and predominantly applied to the Polish translation of the ‘terminal’ phase description. Specification given to the Polish equivalent of the term ‘terminal’ did not precisely explain prognosis (meaning both a few or dozen days), and suggested that returning to other phases seemed to be improbable. The sense of several terms was recognized as not precise enough (e.g. Polish translations of terms: ‘revision’, ‘distress’, ‘intensity’, as well as the verb ‘dies’ expressed in present time). A particular couple of terms related to different phases seemed to overlap with one another, and could be interpreted differently at different times and by different persons (e.g. Polish proposals of the terms ‘gradual’ and ‘rapid’). Moreover, the frequent usage of conjunctions such as ‘or’/ ‘and’ made the comprehensive process complicated. It also has been noticed that the specifications presented in the ‘START’ column are a repetition of the ‘END’ column content regarding the previously described phase. Four participants reported this fact as confusing. The majority of interviewers recommended changes in the column titles to make them more understandable. Other significant observations regarded the linear form of phases presented in tabular form. This suggested the fixed order of phases in the disease trajectory. It was not clear if the short prognosis of the ‘terminal’ phase includes the possibility of moving back to one of the other phases.

As a result of the feedback obtained during the first round of the interviews: 1) in the process of linguistic treatment, the final tabular form of the PCOC phases Polish translation was achieved (see ‘Additional File 3’), and 2) in the process of graphical transformation—to substitute of the table originally used by the Australian Association for Hospice and Palliative Care [16], the easy-to-use scheme showing all the PCOC phases was developed (see—‘Additional File 4’). A version with graphical tips, additionally facilitating the use of this scheme, was also prepared, which can be requested from the corresponding author.

In the second round of the interviews, in which both table and graphical documents were presented, eight of the 13 respondents admitted that the graphical synthesis was more readable and comprehensible, while five preferred the tabular form. Only a few propositions of graphical changes in the easy-to-use scheme were signaled. The examples of linguistic changes made during cognitive interviewing are given in Table 2.

Table 2 Examples of linguistic changes in the results of cognitive interviews

Acceptability

The staff beliefs in the degree of fit, ease of assignments and familiarity of the 313 obtained patient situations were grouped according to the different PCOC phases (Table 3). We observed higher scores in the accuracy perception and the difficulty in assigning the clinical situation for the stable and terminal phases. More difficulties were observed in the unstable or deteriorating ones.

Table 3 Staff belief in the degree of fit, ease of assignments and familiarity of patient’s situation

Inter-rater reliability

Ten pairs of attending doctors and nurses (three from the palliative care unit and seven from home care) within one institution took part in this study phase. Three hundred and thirteen patients (164 within hospice and 149 within home care) participated in the cross-sectional psychometric validation study (Table 4). More than 97% had cancer with the prevalence corresponding to the cancer incidence levels in Poland [29]. The majority were of poor performance and prognosis.

Table 4 Patient characteristics

The palliative care phases were indicated by using the adapted scheme (see ‘Additional File 4’). Seventy percent of 313 PCOC phase assignment pairs were in agreement, while 30% did not match (Table 5). In total, Cohen’s kappa value was 0.573 (95% CI: 0.522–0.624), representing a moderate level of agreement between the two raters. Similar moderate agreement was also obtained in the subgroups of in-patients (0.543; 95% CI: 0.469–0.618) and home-care subjects (0.594; 95% CI: 0.523–0.664). The lower level of agreement between two raters within the deteriorating phase corresponded with the lower degree of fit in this phase (Table 3).

Table 5 Characteristics of ratings by two clinicians (n = 626)

Concurrent validity

Relationships were observed between patient performance, prognosis and their complex needs. The highest proportion of totally dependent, bed-bound patients was observed for the terminal PCOC phase. Patients of better performance status were seen in the stable, unstable and deteriorating phases (P < 0.001; Fig. 1).

Fig. 1

Patient performance status in different PCOC phases. PPS – Palliative Performance Scale

The unstable, progressing primary diagnosis, with the expected prognosis in months (GSF-B), was the most prevalent within the stable PCOC phase, the deteriorating stage (GSF-C) in the deteriorating PCOC phase and the final stage (GSF-D) in the terminal phase (P < 0.001; Fig. 2).

Fig. 2

Patient stage of disease and prognosis assessment in different PCOC phases. GSF – Gold Standard Framework needs-based coding (GSF-B: Unstable, advanced disease with months expected prognosis; GSF-C: Deteriorating with weeks prognosis; GSF-D: Final days)

One hundred and forty-seven patients (47%) were in the subgroup of persons with high complexity palliative care needs for whom intensive specialized palliative care is systematically recommended. The PALCOM complexity needs in the unstable PCOC phase was significantly higher than in the stable or deteriorating phases. Moreover, in the terminal phase, the complexity was higher than in stable one (p = 0.005; Table 6).

Table 6 Different complex palliative care needs within various care phases

Discussion

We developed an adapted version of the Australian PCOC phases, and performed its validation. To our knowledge, this is a completely new concept in the assessment of patients’ and their carers’ situation within the Polish health care system, which enables continuous evaluation and improvement of palliative care via the benchmarking method. Such an audit, besides the usage of various accessible quality of care indicators, requires a case-mix system of grouping patients into clinically meaningful and resource homogenous clusters to be accurately compared in terms of structure, process and outcomes. The cognitive interviews conducted in our study showed that the idea of phases resembled a GSF method of patient prognosis presentation as a stage, which was previously practiced within our hospice team. In consequence, the professionals initially tended to concentrate solely on the patient, without family appraisal. Additionally, the tabular, quite complicated form of the tool, with repetitions in the columns, could not be easily comprehended by a few of interviewers at first. Some terms were imprecise and subject to varying interpretations by team members, highlighting the need for training before implementation. For this reason, the graphical scheme was proposed and positively accepted by the majority. The professionals recommended adding some hints to distinguish the deteriorating phase from the unstable one. Clearer indication that the terminal phase overrules other phases was also suggested. Our observations were consistent with the German adaptation study in which respondents noticed some problems with formal translation, terminology ambiguity and fair fitting to wording or local procedures, indicating the need for additional training before implementing this new concept [18].

Half of the patients in our cohort were in deteriorating or terminal phases (49.7% of agreed assignments), which was similar to the recent Australian observational study with pancreatic cancer patients admitted to a specialist palliative care facility [30]. High PALCOM complexity needs in nearly half of the assessed patients in our study were higher than in the recent Spanish project (30%) [26]. This allows to suggest adequate referral triage, which was performed remotely, based on an innovative referral form, supplied documentation, and—when necessary, to make a justified decision – complementary information acquired via telephone conversation [31]. In these patients representing the “unpredictable” group of high complexness, the higher risk of prolonged hospitalization and the greater likelihood of in-hospital death occurs [26]—if not provided assistance in a timely manner. They may require urgent admission and sudden changes in the car plan, and this subgroup was greatly represented within the unstable PCOC phase.

The inter-rater reliability in our study showed only a moderate degree of agreement (Cohen’s Kappa = 0.57). However, it was comparable to Australian research [9] and slightly higher than the results of German [18] and recent Chinese [32] studies. Seventy percent of ratings were in agreement, particularly for the deteriorating phase. This data was lower than in the Chinese (82.3%) [32], but higher than in the German (63.3%) trials [18]. The reasons for such an observation could be originated in insufficient initial staff training, specific pair constitution (physician and nurse) or poor recognition of the patient & family situation. It is probable that a regular, interprofessional team assessments would have an effect on obtaining more reliable and valuable results.

The overall ease of assignment together with the degree of fit convictions, which was better than observed in both the Australian and German studies, pointed to the fair acceptability of this new instrument for assessment in Polish settings. High grades of fit and ease of assignments could be seen as a result of implementing the graphical synthesis of the PCOC concept, and also the ongoing practical training during the course of the study.

We observed reasonable relationships between performance, prognosis, complex palliative needs and certain PCOC phases. The terminal phase closely corresponded with being totally dependent (PPS10-30%) and having the shortest prognosis (GSF D), and to the contrary, patients within the stable phase were more independent and had the best prognosis. What is more, the highest PALCOM complexity needs were observed within the unstable PCOC phase. These observations indicate that the use of PCOC phases could sensitively discriminate between persons’ different clinical conditions, also facilitating triage for specialist palliative care.

Some limitations arose as a consequence of the national reimbursement system, which mainly promotes cancer patients for admission to specialized palliative care. Thus, our study was primarily focused on this group of individuals. Due to frequent changes in the clinical situation of the assessed patients, there was no possibility of checking the test–retest reliability of the adapted PCOC phases. The reasons for achieving only moderate agreement measures between the raters were not analyzed in this study. Besides, only the PCOC phases concept was of main interest, not taking detailed, complex patient assessment into account. For this purpose, future studies should be planned within larger, multicenter samples. The essence of innovation in this study was the patient stratification within the Polish palliative setting, based on a large number of ratings retrieved from routine clinical care, involving in-patient and home-care facilities, which is a significant step to better managing and delivering palliative care in Polish conditions.

Conclusions

The Polish adaptation of the Australian PCOC phases concept is valid, moderately reliable and acceptable. It enables stratification of palliative care patients into clinically meaningful groups for auditing, benchmarking and comparison across micro- (institutional) and macro-settings (national). However, this concept should be based on the knowledge of the patient & family situation, preferably evaluated through multiprofessional team work. The PCOC phases definition can be useful for routine palliative care triage and monitoring in Poland, being the basis for point-of-care outcome assessment in clinical practice as well.