Article Discussion: Bigler's Review of Validity Testing
Let’s agree…
Dr. Erin Bigler (2014) covers a lot of ground with “Effort, symptom validity testing, performance validity testing and traumatic brain injury”, his overview of performance validity testing (PVT) that places special emphasis on how TBI-related neurological impairment/damage could influence interpretation of PVTs. Specifically, he attempts to highlight the interpretive challenges of those individuals falling in the “border-zone” on PVTs. The term “border-zone” is used to describe scores that are below empirically-based cut-offs, but well above chance levels of performance.
I think most would agree with Dr. Bigler that interpretation of border-zone PVT failures may not always be straightforward when compared to below chance performance (a clearer indicator of invalid performance) or comfortably passing validity measures (likely indicating valid test performance). He also makes valid points about the inevitable lack of precision with cut scores for any one test and their rigid interpretation in isolation. Most clinicians would also agree that failed PVTs do not negate the fact that: an injury occurred, there is imaging evidence of brain injury, or there may be residual cognitive deficits.
A recurrent theme throughout the review is that most clinicians determine the validity of an individual’s performance based on rigid cut-offs only. In fact, most clinicians utilize some variation of the Slick et al. criteria when interpreting cognitive test data (e.g., integrating multiple pieces of information from a variety of sources before one makes a conclusion about performance validity). The Slick criteria include the assessment of inconsistencies in presentation as well as interpretation of multiple PVTs, which then guide clinical judgment.
Again, most would agree with Dr. Bigler that PVTs can be failed by those with neurological conditions directly attributed to the extent/severity of the neurological disorder (e.g., TBI, dementia, mental retardation, psychotic disorders, Huntington’s disease). He mentioned that if those with even a severe TBI are able to repeat simple digits forward/backward, it stands to reason that those reporting a remote history of mTBI who cannot complete this task are displaying invalid performance. He appears to ignore this general dose-response principle, however, when providing several individual examples and research findings throughout the review, wherein patients with minimal to no cognitive deficits fail PVTs.
…to disagree
In contrast, in samples with clear neurological diagnoses and suspected impairments (refractory epilepsy, multiple sclerosis), he decides to essentially ignore any possible relevance of PVT failures. When memory performance falls within the expected “direction of impairment” (below the average range), PVT results are viewed as “irrelevant.” I believe most clinicians would disagree with this assumption and find that the severity of cognitive compromise is highly relevant when making determinations about a patient’s status (e.g., presence/severity of dementia, need for interventions, extent of disability, etc.) or increasing the knowledge base about a clinical condition (e.g., tracking the severity of cognitive deficits after sport concussion over time).
Dr. Bigler provides a series of theoretical explanations for why border-zone PVT failure could indicate one’s true level of neurological compromise (due to deficits in memory/attentional systems and motivational behaviors). Examples include: illness beliefs/diagnosis threat, Somatic Symptom Disorder, frontal lobe syndrome, depression, mental fatigue, PTSD, ADHD, white matter dysfunction, hippocampal pathology, basal ganglia degeneration, default mode network disruptions, PVTs requiring effortful processing, and any combination of the above.
In his review, Dr. Bigler appears to be overly-focused on the many possible combinations of explanations, rather than relying on the dose-response approach he noted earlier[1]. Individuals with conditions that result in minimal to no cognitive compromise (on very challenging cognitive measures) should not perform poorly on easy PVTs when compared to those with more severe neurological impairments (who frequently pass PVTs). For example, when he presents several examples of PVT failure in those with documented neurological injury (e.g., using neuroimaging, biochemical, or functional imaging techniques), these same neurological underpinnings should also lead to even worse performance on more challenging cognitive measures that require greater neural recruitment. This pattern would be neurologically consistent with a dose-response impairment profile; however, several of Dr. Bigler’s examples of PVT failure either (a) do not mention performance on more complex cognitive tasks or (b) ignore the fact that performance was within the average range or higher. One may also begin to question Dr. Bigler’s often implied 1:1 relationship between abnormalities noted on neuroimaging, fMRI, and/or MRS and performance on cognitive testing as several cases (and research samples, see Hetherington, 2014) often performed well within the average range.
In a section entitled, “A mTBI case study…The prototype problem for the SVT researcher and clinician,” he presents a case that is neither mild (GCS = 5, with neuroimaging abnormalities) nor prototypical (essentially above average IQ and memory performance) who performed at cut-offs and below on PVTs. If the individual in question was significantly impaired with daily functioning,[2] the reader may entertain the idea that low PVTs were possibly due (in part) to organic pathology; however, failed PVTs due to residual brain dysfunction would not be consistent with his above average cognitive/memory abilities on more complex cognitive testing. The individual’s statement that he “doesn’t give a shit about things!” might also be more suggestive of depression and a lack of persistence. Most would also not conclude that test scores were completely invalid but performance may slightly underestimate his true level of ability.
At one point Dr. Bigler expresses great concern about the field of neuropsychology in general when discussing the high rates of PVT failure (up to 60%) in a very specific population (Military and Veteran samples). Specifically, he appears worried that failed test results in these populations may render assessments “non-interpretable” and that “these high levels of [PVT] failure only mean invalid test performance.” It is unclear why he is in such distress over the relatively high rates of invalid testing in a group of individuals often claiming remote mTBI-related cognitive impairments in settings with clear external incentives for under-performance. If anything, being able to utilize empirically-based PVT cut-offs to help determine valid cognitive test results in Military and Veteran populations may have in fact elevated neuropsychology to the forefront of research, assessment, and clinical interventions for combat Veterans over the past decade.
Unfortunately/Fortunately
Throughout the review, Dr. Bigler poses 61 questions to the reader[3] in an apparent attempt to show that we know very little about the impact of a variety of biological, psychiatric, and/or environmental factors on PVT performance. Granted, several questions are relevant to the field, including what types of PVTs and cut-offs should be used with specific patient populations, possible order effects of PVTs within a test battery, and how many PVTs should be administered/interpreted. Nevertheless, after considering most of his questions, readers may be able to reasonably interpret “border-zone” PVT failures by referring to the edited works of Kyle Boone, Glenn Larrabee, Dominic Carone/Shane Bush, and Joel Morgan/Jerry Sweet, which nicely summarize the field of modern PVT assessment.
Similar Pages
Learn More. Know More.
