The Importance of Narrative Assessments in Speech Language Pathology!

As some of you may know, I am expecting my first baby (It's a BOY!) and will be taking some time off from work and blogging before and after his arrival. During that time, there will be several guest bloggers/SLPs that will be featured on my blog. I am so excited to share all of the amazing, informative posts they have come up with. I can not thank them all enough for taking the time to write these posts! Just another one of the many reasons I love being part of the SLP world. I hope you all enjoy reading everything over the next several weeks. Please feel free to leave comments and post questions.

The next guest post is written by Tatyana at Smart Speech Therapy! Enjoy!

 

The importance of narrative assessments in speech language pathology

As speech language pathologists we routinely administer a variety of testing batteries in order to assess our students’ speech-language abilities. Grammar, syntax, vocabulary, and sentence formulation get frequent and thorough attention. But how about narrative production? Does it get its fair share of attention when the clinicians are looking to determine the extent of the child’s language deficits? I was so curious about what the clinicians across the country were doing that I created a survey and posted a link to it on various social media sites (e.g., Facebook). I wanted to find out how many SLPs were performing narrative assessments, in which settings, and with which populations. From those who were performing these assessments I wanted to know what type of assessments were they using and how they were recording and documenting their findings. Since the purpose of this survey was non-research based (I wasn’t planning on submitting a research manuscript with my findings), I only analyzed the first 100 responses (the rest were very similar in nature) which came my way, in order to get the general flavor of current trends among clinicians, when it came to narrative assessments. Here’s a brief overview of my [limited] findings. Of the first 100 clinician’s whose responses were analyzed, the two largest groups were composed of SLPs who had been practicing for over 15 years (39%) as well as those who had been practicing from 5-10 years (18%). Of course it was not a surprise that 76% of these practitioners were based in school setting followed by 23% of practitioners in private practice. Out of them only about 74% of clinicians had ever assessed narratives. These numbers went further on decrease when asked how often these narrative assessments took place: 40% of clinicians stated that they’ve done it rarely with <25% of students, followed by 28% who stated that they performed narrative assessments only some of the time with <50% of students. When asked if the clinicians favored standardized instruments, non-standardized instruments, or both, the responses were split. Out of those SLPs who performed narrative assessments (see above), 52% favored standardized instruments while a whopping 68% favored non-standardized instruments, indicating an overlap in what instruments were used by clinicians.

However, it was when I asked the question: ‘What type of narrative assessments have you administered?’ that I received the most interesting responses. More than 50% of clinicians preferred the Test of Narrative Language. Another 27% preferred non-specific dynamic assessments and yet another 27% performed a wide variety of other assessments, which included but not limited to: informal language samples analyses (e.g., One Frog Too Many), Peter and the Cat Narrative Assessment, MindWing concepts’ Story Grammar Marker (SGM), Developmental Sentence Score (DSS), Michelle Garcia Winner's Dynamic Assessment Protocol, informal story retelling analysis via SALT software, Assessment of Preschool Narrative Skills (McCabe and Rollins, 1994), The Edmonton Narrative Norms Instrument (ENNI), as well as a numerous other instruments.

One thing which became immediately apparent by reviewing the survey results is that SLPs who used non-standardized instruments clearly used an impressively wide variety of assessments. Another was that some of the SLP-listed instruments did not actually assess narrative skills. For example some survey respondents stated that they used the Test of Word Finding in Discourse (TWFD). However, this test actually assesses conversational skills and not narrative skills. A few responders stated that they used “responses on the definitions subtest of the Word Test – 2”, as a narrative measure, however, those responses can at best count as informal assessment of spontaneous sentence formulation vs. a true narrative assessment. So what actually constitutes a true narrative assessment and why are narrative assessments so important? Well, in order for an assessment to count as a narrative assessment, an SLP must listen to some kind of prompted/unprompted story recounting some form of an event (instead of engaging in a back-and-forth conversational exchange with the child or listening to him/her formulate a brief response based on presented questions). There are several different types of narratives: Scripts -express knowledge of a familiar, recurring events Recounts - retell a personal experience when prompted and are shared by listener Event casts - explain an ongoing activity, report on a factual scene, or tell about a future plan Accounts - explain personal experience without the prompts and are not shared by the listener Fictional story retelling – involves the child retelling a story that he/she read or which was read to him/her Out of these narrative types, accounts of personal experience as well as retelling of fictional stories are used most frequently by researchers to assess children’s narrative competence. Here are just some of the reasons why these are preferred. Personal narratives constitute a large section of functional discourse commonly found in the spontaneous conversation of typically developing preschool and school-aged children. They are sensitive to narrative competence (Bliss & McCabe 2012). They require the speaker to organize an account or a recount of past experiences. They provide effective ways to tax the system due to unplanned discourse (Hadley, 1998; Hughes, 2001). Although they are natural to tell, they also require that the storyteller makes the story interesting to listeners so not to lose their attention (Hughes, 2001). Finally, they are important for connecting with family and friends as well as for literacy acquisition (Bliss & McCabe 2012).

Similarly, fictional narratives tap into the child’s knowledge of organizational framework and are sensitive to language impairment (McCabe & Rosenthal-Rollins, 1994). They have been routinely been identified as one of the single best predictors of future academic success (Bishop & Edmundson, 1987; Feegans & Appelbaum 1986). Poor discourse and narrative abilities place children at risk for learning and literacy-related difficulties including reading problems (McCabe & Rosenthal-Rollins, 1994). Narrative analyses help to distinguish children with language impairment from their typically developing peers (Allen et al 2012). Furthermore, language produced during story retelling is positively related to bilingual reading achievement (Miller et al, 2006). Narrative retelling bridges the gap between oral and written language and is important for appropriate reading and writing development (Snow et al, 1998). It provides insights into child’s verbal expression since it taps into multiple language features and organizational abilities simultaneously (Hoffman, 2009; McCabe & Bliss, 2003; Ukrainetz, 2006). This is not the case with standardized testing. Many children may evidence appropriate performance on standardized testing but still display significant difficulties in the area of narrative production. However, unless SLPs assess narratives in children specifically, they run the risk of overlooking weaknesses in this area of functioning. Finally, assessment of narratives is very important because poor storytelling skills are indicative of social pragmatic language impairment. In fact, part of the diagnostic criteria of social pragmatic communication disorder in the DSM-V™ is “difficulties following rules for conversation and storytelling”.

So what are some ways in which SLPs can assess narratives? When it comes to narrative assessment SLPs have many options available to them (as illustrated above). Depending on the age and type of the population they are working with (e.g., school-age vs. preschool) they can chose from a variety of standardized, non-standardized, or dynamic assessments. However, when choosing the assessment it is very important to ensure that it examines both microstructure and macrostructure domains of storytelling. When analyzing microstructure we are looking at the child’s grammar, syntax and vocabulary competence (content and form). In contrast, when we examine macrostructure we are looking at the child’s use of language. Is the story organized and cohesive? (Justice et al, 2006) Does it contain the story grammar elements commensurate with their age such as setting, initiating event, internal response, attempt, consequences, and reaction (Stein & Glenn, 1979)? For school children 5;00-11;11 years of age, the standardized assessment, Test of Narrative Language- TNL (2004) examines the microstructure and macrostructure elements, as well as measures the child’s ability to answer literal and inferential comprehension questions. It costs about $192, however, prior to purchasing it, SLPs should be aware that it does have some limitations. For starters, personally, I found it to be much more effective at identifying narrative deficits in severely language impaired children vs. children with more milder/subtler impairments. Additionally, on the TNL, test-retest reliabilities are not separately calculated for different ages. Thus, according to Baxter & Van Lingen, 2005, the test-retest data are based on a small, non-representative group and are not strong enough for clinical decision-making (p. 1041). TNL also does not equally represent all tested age-groups and SES levels. For example, the 5-year old group (n=83) was small and almost ½ of the number of the other groups and “upper-income” children were overrepresented in the sample (Baxter & Van Lingen, 2005, p. 1041). As many other standardized tests, TNL favors children with “mainstream” experience and is not recommended for use with multicultural/low SES children from “low print”/”low talk” environments. (Hayward et al 2008). To continue, if you are working with preschool/kindergarten aged children, then you might want to consider either the Narrative Assessment Protocol (NAP) by Pence, Justice, & Gosse (2007) or the Narrative Language Measures (NLM) by Petersen & Spencer (2012) both of which are available online for free.
 Let’s begin with NAP. Developed for children 3-6 years of age, it uses Mercer Mayer’s (1969) ‘Frog, Where are You?’ book to analyze the child’s story microstructure (e.g., sentence and phrase structures, modifiers, nouns and verbs). According to the manual, the macrostructure domain of the NAP is currently under development. However, since the most recent manual was last updated in August 2008, I am not quite certain how up-to-date this information really is. To continue NAP is fairly easy to use. The child is first shown a book then asked to tell a story using pictures as prompts. One of the advantages of NAP is that it does not require narrative transcription and manual utterance tallying, since the examiner scores the narrative as the story is being told. However, the lack of macrostructure analysis is a big disadvantage in my eyes, which is why I personally do not use this assessment when I analyze the narratives of preschool children. Next up is the NLM. Referred to by its authors as the free RTI focused assessment tool, the NLM has two developed versions: NLM:P and NLM:K for preschool and kindergarten children respectively. For the purposes of this post I’ll review the preschool version only, which according to the authors is the most researched one. The administration time per each subtest takes approximately 2–5 minutes. Subtests are scored in real time while the child is narrating. The scoring rubrics contain 0–2 or 0–3 point ratings and cover 2 critical subscales: story grammar and language complexity. Of course since this is a standardized test it does contain some limitations. Children who lack previous exposure to testing may do poorly, so cultural considerations are always an important factor. Authors also strongly urge that single subtest administration be avoided and if the child’s abilities are probed infrequently (e.g., seasonally), the administration of all 3 subtests is strongly recommended for validity purposes. However, to me the biggest issue is that the subtests lack norms or cut scores to help identify at risk children. Depending on the setting you are in, the administration of this test may make it difficult for you to qualify the child for services, which is why you might be better off with an informal assessment.

Personally, I prefer informal narrative assessments to the standardized ones. There are many advantages to informal assessments. They provide more detailed information regarding microstructural and macrostructural elements as well as child’s thought processes and socio-emotional functioning. They are based on least biased assessment principle (Goldstein, 2006), which means that you can make adaptations/modifications if you are using a standardized protocol, you can take into account cultural considerations, and of course spontaneous language sample measures that can supplement and clarify diagnostic information from standardized assessments (Rojas & Iglesias, 2009). Here’s how I perform mine. I use the books recommended by the SALT Software website because it offers convenient suggestions by age, ranging from preschool through 6^th grade. I typically read the story to the child first then cover up the words and ask the child to retell the story based on pictures alone. I read the story first because: “the model narrative presents the events, plot structure, and words that the narrator is to retell, which allows more reliable scoring than a generated story that can go in many directions” (Allen et al, 2012, p. 207). As the child is retelling the story I digitally record his/her narrative for transcription and thorough analysis. During story telling I use the prompts such as: ‘What else can you tell me?’ and ‘Can you tell me more?’ to elicit additional information, however, I try not to prompt the child excessively; otherwise the child is producing prompted responses vs. telling me a spontaneous story. So what am I looking at when I am performing my narrative analysis?
 Well, in the words of Pookins from Helen Lester’s ‘Pookins Gets Her Way’: “Lots!”

• Sequencing Ability
  • Is the story order appropriate or jumbled up?
• Working Memory
  • Is the child using enough details?
  • Are these details relevant to the story?
• Grammar
  • Does the sentence structure contains errors such as run-on sentences or incorrectly produced words
  • Is the child using temporal markers and cohesive ties to connect the story together
  • Is the child’s sentence complexity adequate for their age?
• Vocabulary
  • Is it immature or age-level?
  • Does the child have word retrieval issues or is s/he maintaining lexical fluency?
• Pragmatics
  • Is there story cohesion and coherence?
  • Is there adequate use of anaphoric references (pronouns identifying characters)?
• Perspective Taking
  • Does the child have appropriate insight into character’s feelings, beliefs, thoughts?

I evaluate these areas using the guidelines set forth in Hedberg & Westby, 1993 “Analyzing Storytelling Skills” book, which offers a comprehensive guide to narrative analysis. After I finish, I include the transcription and a detailed write-up in the body of my language report, so parents and professionals could see it and understand the nature of the child’s errors/weaknesses. Another reason why I include the transcription in the body of my report is because it allows parents and professionals to compare past results with future gains and to see in which specific areas (e.g., story complexity, sentence structure, perspective taking, use of details etc) progress was made following therapeutic intervention. Because storytelling encompasses a number of higher-level language and cognitive skills (Paul et al, 1996), I believe that it should be an integral component of every language evaluation since no other assessment yields as much information about the child’s language abilities as the narrative one. Now that I’ve told you how I perform narrative assessments, I’d love to know how you perform them and what formal/informal instruments do you use? Helpful Smart Speech Therapy Resources:

• Narrative Assessment Bundle
• Narrative Assessments of Preschool and School Aged Children
• Vocabulary Development: Working With Disadvantaged Populations
• Understanding Complex Sentences

Bio: Tatyana Elleseff MA CCC-SLP is a bilingual SLP with a full time hospital affiliation as well as private practice in Central, NJ. She specializes in working with multicultural, internationally and domestically adopted as well as at-risk children with complex communication disorders. For more information visit her BLOG, STORE, or follow her Facebook page. References:

1. Allen, M, Ukrainetz, T & Carswell, A (2012) The narrative language performance of three types of at-risk first-grade readers. Language, Speech, and Hearing Services in Schools, 43(2), 205-221.
2. Baxter, A. & Van Lingen, G. (2005). Test review of the Test of Narrative Language. In Spies, R.A. and B. S. Plake (Eds.), The sixteenth mental measurement yearbook (pp. 1040-1045). Lincoln, NE: Buros Institute of Mental Measurements.
3. Bishop, D. V. M., & Edmundson, A. (1987). Language impaired 4-year-olds: Distinguishing transient from persistent impairment. Journal of Speech and Hearing Disorders, 52, 156–173.
4. Bliss, L. & McCabe, A (2012, Oct) Personal Narratives: Assessment and Intervention. Perspectives on Language Learning and Education. 19:130-138
5. Feegans, L.,& Appelbaum, M (1986). Validation of language subtypes in learning disabled children. Journal of Educational Psychology, 78, 358–364.
6. Goldstein, B. A. (2006). Clinical implications of research on language development and disorders in bilingual children. Topics in Language Disorders, 26(4), 305–321.
7. Hayward, D. V., Stewart, G. E., Phillips, L. M., Norris, S. P., & Lovell, M. A. (2008). Test review: Test of narrative language (TNL). Language, Phonological Awareness, and Reading Test Directory (pp. 1-12). Edmonton, AB: Canadian Centre for Research on Literacy. Retrieved 9/2/13 from http://www.ualberta.ca/~lphillip/documents/The%20Test%20of%20Narrative%20Language%20(TNL).doc
8. Hedberg, N. & Westby, C. (1993). Analyzing story-telling skills: Theory to Practice. Tucson, AZ: Communication Skill Builders.
9. Hoffman, L. M. (2009a). Narrative language intervention intensity and dosage: Telling the whole story. Topics in Language Disorders, 29, 329–343.
10. Hughes, D., (2001, Oct) Assessment of Narrative Abilities in Preschool and School-Age Children. Perspectives in Language Learning and Education. pp. 7-11.
11. Justice, L. M., Bowles, R. P., Kaderavek, J. N., Ukrainetz, T. A., Eisenberg, S. L., & Gillam, R. B. (2006). The index of narrative microstructure: A clinical tool for analyzing school-age children's narrative performance. American Journal of Speech-Language Pathology, 15, 177–191.
12. McCabe, A., & Rollins, P. R. (1994). Assessment of preschool narrative skills. American Journal of Speech-Language Pathology, 3(1), 45–56
13. Miller, J et al (2006). Oral language and reading in bilingual children. Learning Disabilities Research and Practice, 21, 30–43.
14. Paul R, Hernandez R, Taylor L, Johnson K. (1996) Narrative development in late talkers: early school age. Journal of Speech and Hearing Research, 39(6):1295–1303
15. Rojas, R. & Iglesias, A. (2009, March 03). Making a Case for Language Sampling : Assessment and Intervention With (Spanish-English) Second Language Learners. The ASHA Leader.
16. Stein, N., & Glenn, C. (1979). An analysis of story comprehension in elementary school children. In R. O. Freedle (Ed.), New directions in discourse processing (Vol. 2, pp. 53-120). Norwood, NJ: Ablex.
17. Ukrainetz, T. A. (2006b). Teaching narrative structure: Coherence, cohesion, and captivation. In T. A. Ukrainetz (Ed.), Contextualized language intervention: Scaffolding PreK–12 literacy achievement (pp. 195–246). Austin, TX: Pro-Ed.

    ------ Tatyana Elleseff MA CCC-SLP Speech Language Pathologist Rutgers University Behavioral Healthcare Smart Speech Therapy LLC 732-235-5829 office 917-916-7487 mobile www.smartspeechtherapy.com/blog/ http://www.smartspeechtherapy.com/shop/ https://www.facebook.com/SmartSpeechTherapyLlc