Observer reliability of CT angiography in the assessment of acute ischaemic stroke: data from the Third International Stroke Trial.

Mair, Grant; von Kummer, Rüdiger; Adami, Alessandro; White, Philip M; Adams, Matthew E; Yan, Bernard; Demchuk, Andrew M; Farrall, Andrew J; Sellar, Robin J; Ramaswamy, Rajesh; Mollison, Daisy; Boyd, Elena V; Rodrigues, Mark A; Samji, Karim; Baird, Andrew J; Cohen, Geoff; Sakka, Eleni; Palmer, Jeb; Perry, David; Lindley, Richard; Sandercock, Peter A G; Wardlaw, Joanna M

INTRODUCTION CT angiography (CTA) is often used for assessing patients with acute ischaemic stroke. Only limited observer reliability data exist. We tested inter- and intra-observer reliability for the assessment of CTA in acute ischaemic stroke. METHODS We selected 15 cases from the Third International Stroke Trial (IST-3, ISRCTN25765518) with various degrees of arterial obstruction in different intracranial locations on CTA. To assess inter-observer reliability, seven members of the IST-3 expert image reading panel (>5 years experience reading CTA) and seven radiology trainees (<2 years experience) rated all 15 scans independently and blind to clinical data for: presence (versus absence) of any intracranial arterial abnormality (stenosis or occlusion), severity of arterial abnormality using relevant scales (IST-3 angiography score, Thrombolysis in Cerebral Infarction (TICI) score, Clot Burden Score), collateral supply and visibility of a perfusion defect on CTA source images (CTA-SI). Intra-observer reliability was assessed using independently repeated expert panel scan ratings. We assessed observer agreement with Krippendorff's-alpha (K-alpha). RESULTS Among experienced observers, inter-observer agreement was substantial for the identification of any angiographic abnormality (K-alpha = 0.70) and with an angiography assessment scale (K-alpha = 0.60-0.66). There was less agreement for grades of collateral supply (K-alpha = 0.56) or for identification of a perfusion defect on CTA-SI (K-alpha = 0.32). Radiology trainees performed as well as expert readers when additional training was undertaken (neuroradiology specialist trainees). Intra-observer agreement among experts provided similar results (K-alpha = 0.33-0.72). CONCLUSION For most imaging characteristics assessed, CTA has moderate to substantial observer agreement in acute ischaemic stroke. Experienced readers and those with specialist training perform best.