Why I still like the verbal reasoning subtest of theCognitive Competency Test (and am even liking thepicture interpretation subtest, too)(Paper-based versus functional assessments)or(Can the verbal reasoning subtest tell yousomeone is safe to go home?)Lilli Ju Lin, OTReg(NB)Health & Aging Program, Horizon Health NetworkSt. Joseph’s Hospital, Saint John, New Brunswick

Outline Introduction (and why this interests me) Objectives Super duper fast background About the test What others have said about the test Test administration/scoring – a few tips and common errorsQuantitative vs. qualitative interpretationPicture interpretation really?A couple of casesQuestions to answer

Introduction (and why this interests me) Much discussion and debate about the merits of the Cognitive Competency Test and its utility as anassessment of cognition and its ability (or inability) toinform a client’s level of functional independence.Lots of CAOT webinars on cognition over the past fewyears; 2-day course in HalifaxRecent research (masters and doctoral level) by OTsMy background – psychology, psychology lab, workedfor a neuropsychologist as psychometrist in a healthyaging programReally want to talk about this with other OTs!!!

Disclaimer Even though I’m doing a presentation about a paper-based test I am still all about FUNCTION. Nothing is as informative as seeing someone dosomething!

Why? Verbal reasoning subtest often used alone Easy-to-administer with cutoff score But functional? But how do you rationalize when there are differencesin “score” and “function”

Objectives To provide a very brief background on the CCT. To discuss the value in analyzing the quality of the client’sperformance with the actual verbal reasoning score, andhow this can provide meaningful information. To provide case examples of how the picture interpretationsubtest can relate to the verbal reasoning subtest, andwhat this could mean functionally. To pose other questions around the verbal reasoningsubtest (e.g., how it compares to the Independent LivingScales subtest on Managing Health and Safety).

Ultimately To spark interesting discussion andcontribute to our confidence inhow we select and use screeningand assessment tools, in concertwith actual functional assessments.

“Occupational therapists commonly assess cognitivecapacities such as memory, attention, and problemsolving when working with older adults. Results ofcognitive assessments provide critical informationused to determine safety for independent living,driving ability and eligibility for a variety of services.OTs commonly use standardized screening measuressuch as the Mini Mental Status Exam or the MontrealCognitive Assessment. Interpretation of scoresbeyond the limits of the data to support them canresult in unethical health care, including incorrectidentification of dementias and incorrect decisionsabout driver’s licenses or placement.”Alison Douglas, PhD, OT Reg(Ont)Summit on Aging, OSOT, 31 October 2009

Super duper fast background Cognitive Competency Test Familiar Used across Canada (Alison Douglas et al., 2007) “Standardized” for older adults, with “norms” Has some construct validity, correlations with otherclinical measures (e.g., MMSE) (Briana Zur, 2011)

About the CCT Described by its developers as an assessment tool thatattempts to measure cognition in relation to everydayliving, and was designed to close a gap betweenpsychological assessment and everyday functioning (Wang,1990; Wang & Ennis, 1986). “an ability to know and to make use of knowledge” (Wang,Ennis, & Copland, 1987. p. 1) “incorporates the concept of multidimensionality ofcognitive skill and adopts a practical approach bysimulating daily living skills” (Wang & Ennis, 1986, p. 120) Face validity

What others have said about the test Douglas et al., 2007 – Critical review of cognitiveassessments for older adults Zur, 2011 – Assessment of occupational competence indementia: Identifying key components of cognitivecompetence and examining the validity of the CognitiveCompetency Test Debbie McQuillen 19 June 2012 - Assessment of cognition in seniors: Are youassessing what you think you are? Choosing assessment tools Functional assessments of cognition (CCT, ILS, AMPS) “Bad” vs. “good” score Can they do what they say they’d do?Ask the question, follow it with functional activity Medication, money, safety (calling superintendent, etc.)

What others have said about the test Debbie McQuillen 15 Jan 2013 - Assessing cognition in seniors: Makingsense of the numbers (beyond the cut off scores) April 2013 – 2 day CAOT workshop on cognition inHalifax Functional Subsections stand alone Cultural bias Outdated

What others have said about the test Sylvia Davidson 26 June 2012 – Occupational therapists and their rolewith older adults: The generalist as specialist? Occupational therapists have the knowledge and the skillsto be experts in the care of older adults We can build the relationships we need to be effectiveleaders in the care of older adults We can inspire others with our passion and our vision! “Function trumps numbers”

Test administration/scoring(a few tips and common errors) Client: Hearing, vision, language, fatigue, medication,position, comfort, cultural background, mental status,time of day Administrator: Tone of voice, speed of speech,position, rapport Generally encouraging, but not correcting Avoid: “That’s right!” Ok: “You’re doing just fine” Always introduce Sometimes you just have to cut them off

Test administration/scoring(a few tips and common errors) LOOK IT UP Not good enough to go by “feel” Know how many times you can cue! Appropriate cues: “Tell me more”; “Anything else” Repeating the question (as often as necessary) Inappropriate cues: “Who would you call?” Try to maintain standardized approach. But can ask supplemental questions (but note this)

Test administration/scoring(a few tips and common errors) Scoring: Specific examples are given Not cover everything Refer to general guidelines on scoring Grasps idea Safety for self or others Appropriate situational priorities Social convention/considerations

Test administration/scoring(a few tips and common errors) Some common issues/questions #2 – During a very bad winter storm, your electricpower goes out and you know it will be out for a verylong time, what will you do? Light, lamp, lantern, flashlight 1 point? #6 – What would you do if you saw thick smoke comingfrom under your neighbour’s door? Break down the door and save them 1 point? #10 – prioritizing urgent household tasks I’d throw out the cup and call the electrician.

Quantitative vs. qualitative interpretation Content What are they saying Are they even talking about the situation? Do they demonstrate understanding of thesituation? Context Where do they live? What supports do they have? What is their life experience?

Quantitative vs. qualitative interpretation CCT IS JUST A SCREEN!! Discussed with neuropsychologists Modifications to questions Note all cues

Qualitative interpretation Helpful observations to make while administering Participation – cooperative, etc. Tangentiality – “One time, the road was really slippery but I had just put on winter tires. Man, they really need to plow the streetsbetter ”Concrete – “I don’t smoke.” “We don’t use gas.” “No one wouldbreak into our home.”Perseverative – If found door open: “I’d check for smoke and fire.”Risk-taking – If there was smoke coming from my neighbour’s door:“I’d kick down the door and drag my neighbour out.”Vague – “You just always have to do the right thing and make sureyou contact someone!”Knowledge – “I don’t know. My husband takes care of all that.”Word-finding – If power out during winter storm: “Make sure I haveenough round things.”Need for cues – indicates need for prompting/supportNeed for question to be repeated – attention? memory?

Interpretation tips – reporting What I used to write: Where they lost their points Some qualitative observations Client’s score 11/20 is in the impaired range of the verbalreasoning subtest of the Cognitive Competency Test,which suggests client does not have adequate verbalreasoning/problem solving skills in safety situations.

Interpretation tips – reporting What I write now: Indicated knowledge in 8/10 safety situations presented Able to identify at least 1 appropriate action in 9/10situations presented Qualitative observations with examples Suggestive of being able to manage common/familiarsituations Impaired score suggests client may have difficulty withunfamiliar/more complex situations

Interpretation tips – discrepancies Explaining to team/families ****HELPFUL! High score, but poor performance functionally (e.g.,issues with meal prep, personal care) Verbal reasoning is strongly weighted on VERBAL skills. High score demonstrates client’s knowledge/intelligence andverbal skills, which is why they may present as seeming alright Functional assessments more telling (can’t “walk the walk” ) Impaired score, but good performance functionally Look at quality, context of the responses One lady scored 9/20 on verbal reasoning Intellectual disability; staff in apartment building and verystrong social/formal supports Safe behaviours, awareness/insight Responses were appropriate to her context/environment!

Picture interpretation really? Neuropsychologist who used it with difficult case Someone who scored “high” on everything Completed functional assessments well (ADLs and IADLs) But team and family still had questions based on hispersonal history Examples of individuals who’ve scored in normalrange of VR, but then bomb the PI what?

Picture interpretation really? Present pictures and ask them “Tell me what ishappening in this picture” Concrete/perceptual elements vs. inferential statements BE CAREFUL OF SCORING! Interpretation “Such cognitive skills are necessary prerequisites forappropriate social interaction.” (CCT) Client had difficulty interpreting cues in the social andphysical environment. This raises concerns about client’sability to appropriate interpret a social or safetysituation.

What is happening in this picture?

Case #1 Mr. JC VR – 16/20 If bad winter storm, power out – go to a friend’s or down to the store, see why Ihave no power, put on all my clothes to be warm Risk-taking actions – break down the door and save them PI – 4/10 Man fixing the wheel, car is for sale Guy swimming, dog, barn, trees, road, bridge, guy diving, guy holding ontosomething Personal care – “inappropriate sequencing,” not wash everywhere Great phone skills (phoned nursing unit when roommate neededhelp) Money management – unable to name income, describe how he paysbills, name his bank Meal prep – AMPS process was -0.43 (well below cutoff) Forgot to make toast Kettle whistling and steaming for 4 minutes History – living on own, unpaid bills, decreased hygiene, disorganizedhome

Case #2 Mr. DJ (diplomat) VR – 14/20 – even though impaired, score was “inflated” tangential perseveration (on fire) Disoriented (place and time) PI – 3/10 Details, details, but never the big picture Misinterpreted concrete elements, too Personal care – needed supervision/cuing Money Good math, reading price tag Unable to name coins “This is a kind of goose or a duck” Absolutely not able to give any current details Lots of irrelevant information (number of branches across Canada) History – Worked in embassies mainly in Asia; poor hygiene; livingalone; sister in BC; reclusive and eccentric; home incredibly dirty(sister found dead cat)

Future clinical questions to answer How does VR score compare with Diagnoses? ILS – managing health and safety subtest? Data collection initiated AMPS process score? Picture interpretation?

Take home Be confident Be client-centred Be contextual (always “depends” on something ) Don’t throw it out – it’s still useful It’s JUST A SCREEN!!! Never let any single test be thesole basis for diagnosis and discharge. Interdisciplinary Functional abilities, social network/environment aremore important for discharge Combine standardized assessments with directobservation and clinical reasoning

Take home “A test isn’t smarter thanthe person using it.”- Dr. Yves Turgeon, neuropsychologist

References Davidson, Sylvia. (2012). Occupational therapists and their role with older adults: The generalist as specialist? CAOT Lunch & Learn. 26 June2012.Douglas, Alison. (2009). Ontario Society of Occupational TherapistsSummit on Aging. 31 October 2009.Douglas, A., Liu, L., Warren, S., & Hopper, T. (2007). Cognitiveassessments for older adults: Which ones are used by Canadianoccupational therapists and why. Canadian Journal of OccupationalTherapy, 74(5), 370-371.McQuillen, Debbie. (2012). Assessment of cognition in seniors: Are youassessing what you think you are? Choosing assessment tools. CAOTLunch & Learn. 19 June 2012.McQuillen, Debbie. (2013). Assessing cognition in seniors: Makingsense of the numbers (beyond the cut off scores). CAOT Lunch &Learn. 15 January 2013.McQuillen, Debbie. (2013). Enabling Occupation through theAssessment and Treatment of Cognitive Impairments in Adults. CAOTWorkshop. April 2013.

References Wang, P. (1990). Assessment of cognitive competence. In D.Tupper & K. Cicerone (Eds.), The neuropsychology of everydaylife: Assessment and basic competencies. Norwell, MA: KluwerAcademic Publishers. Wang, P., & Ennis, K. (1986). Competency assessment in clinicalpopulations: An introduction to the Cognitive Competency Test.In B. Uzzell & Y. Gross (Eds.), Clinical neuropsychology ofintervention. Boston: Martinus Nijhoff. Wang, P., Ennis, K., & Copland, S. (1987). Cognitive CompetencyTest Manual. Mount Sinai Hospital, Toronto, Ontario. Zur, Briana M. (2011). Assessment of occupational competencein dementia: Identifying key components of cognitivecompetence and examining validity of the CognitiveCompetency Test. University of Western Ontario - ElectronicThesis and Dissertation Repository. Paper 114. Retrieved from

Special thanks to consultingneuropsychologists! Dr. Yves Turgeon Dr. Elizabeth Minerva Moore

Discuss!!Thank [email protected]

Objectives To provide a very brief background on the CCT. To discuss the value in analyzing the quality of the client’s performance with the actual verbal reasoning score, and