Why standardized tests might not be enough:
A number of articles and perspective pieces have been published that have focused on the importance of using language sample analysis (LSA) as a diagnostic tool (Melanie Schuele’s (2010) article is a great place to start). Despite this, we also know that many SLPs still see LSA as being too time consuming and too difficult, and therefore still tend to over-rely on standardized tests. But, the thing is, norm-reference, standardized tests really aren’t enough. So, let’s first just quickly review some of the key reasons why it’s SO important that we include LSA in our battery of assessments…
First, standardized tests provide just a snapshot of a child’s actual linguistic abilities. They remove the context, social cues, and linguistic demands that a child is confronted with when having to use their language skills in everyday life.
Standardized assessments also place a high level of behavioral demands on the child, which often makes them inappropriate for clients with attention and behavioral difficulties. For instance, the format of standardized tests is also often not appropriate for clients with autism, those who have limited verbal abilities, and children who require AAC. Going along with this, standardized tests are also often invalid for diverse populations (while LSA has been shown to be just the opposite!). We’ve written some recent reviews on some related topics, including how to collect language samples from children who use AAC, LSA as a measure of language development in children with severe speech sound disorders, and using LSA to diagnose language disorders in bilingual English-Spanish speakers.
For all of these reasons, best practice states that we should be using multiple methods of assessment to be able to provide a complete description of a child’s language abilities. This means that we need to go beyond standardized testing to be able to fully describe a child’s strengths and weaknesses, and to be able to write appropriate, functional goals. Language samples address the weaknesses of standardized tests by providing information about a child’s ability to use language in real-world situations.
A number of articles and perspective pieces have been published that have focused on the importance of using language sample analysis (LSA) as a diagnostic tool (Melanie Schuele’s (2010) article is a great place to start). Despite this, we also know that many SLPs still see LSA as being too time consuming and too difficult, and therefore still tend to over-rely on standardized tests. But, the thing is, norm-reference, standardized tests really aren’t enough. So, let’s first just quickly review some of the key reasons why it’s SO important that we include LSA in our battery of assessments…
First, standardized tests provide just a snapshot of a child’s actual linguistic abilities. They remove the context, social cues, and linguistic demands that a child is confronted with when having to use their language skills in everyday life.
Standardized assessments also place a high level of behavioral demands on the child, which often makes them inappropriate for clients with attention and behavioral difficulties. For instance, the format of standardized tests is also often not appropriate for clients with autism, those who have limited verbal abilities, and children who require AAC. Going along with this, standardized tests are also often invalid for diverse populations (while LSA has been shown to be just the opposite!). We’ve written some recent reviews on some related topics, including how to collect language samples from children who use AAC, LSA as a measure of language development in children with severe speech sound disorders, and using LSA to diagnose language disorders in bilingual English-Spanish speakers.
For all of these reasons, best practice states that we should be using multiple methods of assessment to be able to provide a complete description of a child’s language abilities. This means that we need to go beyond standardized testing to be able to fully describe a child’s strengths and weaknesses, and to be able to write appropriate, functional goals. Language samples address the weaknesses of standardized tests by providing information about a child’s ability to use language in real-world situations.
How to go beyond standardized testing:
What analyses can I do? What norms do I use?
Standard LSA includes MLU, NDW, TNW, and TTR. If you remember these terms from your undergrad language development class but can’t remember exactly what they mean, this quick little review is for you:
MLU: Mean Length of Utterance
In addition to MLU, LSA can be used to calculate other measures of semantic diversity, including:
NDW: Number of Different Words
TNW: Total Number of Words
And, from these two measures, we can calculate the:
TTR: Type Token Ratio
In addition to performing these analyses by hand, you also have the option of learning (or, possibly relearning) how to use one of the programs that will do the work for you. The most well-known is SALT (Systematic Analysis of Language Transcripts). If you’re interested in learning more about SALT or just re-familiarizing yourself with the program after not using it for a while, you’re in luck because the website is full of free information, links to short You Tube Videos that walk you through using the program, and self-paced online courses. And, not only that, but almost all of the courses can be completed for (also, free!) ASHA CEUs.
In addition to information about coding conventions and using the program, the courses and website also contain free resources and information about the different contexts that we can use to collect a language sample. You can also access free scripts for the different elicitation contexts so that you’re consistent when you collect your language samples. The website also includes free audio files of stories and graphic organizers that can be used along with the scripts for narrative, expository, and persuasive samples. And, you can find the age and grade levels of each in the norms database on the SALT website here. And, speaking of the database, this article provides a general overview of how it was established, and then discusses how they can be used to identify children with language impairments.
You have the option of downloading a trial version of SALT on the website. This is a good place to start so that you can use it to complete the practice activities in the courses before you decide if you actually want to buy the software. If you do decide to buy the software, it comes with a pdf version of the clinician’s guide to LSA, which is really handy resource for language sampling in general and coding and analysis in SALT.
SUGAR (Sampling Utterances and Grammatical Analysis Revised) is a newer method, and is a procedure rather than software. It is a streamlined method of collecting, transcribing, coding, and analyzing language samples elicited from conversation. Here are some links that will help familiarize you with the program:
The other alternative to SALT and SUGAR is the free, research-based software called Computerized Language Analysis (CLAN). CLAN is like SALT in that it's a software program. Pros are that it's free to download and has excellent data on languages other than English; the con is that is has a bit of a learning curve. Our recent review of Ratner and MacWhinney’s (2016) article regarding the use of CLAN also includes links to download the free software and clinician’s book, and links to articles that focus on the use of CLAN and other related utilities that can be used to further analyze your language samples.
MLU: Mean Length of Utterance
- Total number of morphemes divided by total number of utterances
- Important!!! There are different ways to calculate MLU. The norms that you use need to match your method of calculation and also match the age of the child that you’re evaluating. You have some options when it comes to which norms to use to analyze your sample. If you’re calculating MLU by hand, your first option would be to use Brown’s Stages for interpretation. If you need a quick refresher on Brown’s Stages, you can access a chart that summarizes the information on Caroline Bowen’s website here. The norms for MLU established by Miller and Chapman (1981) are also commonly referenced by SLPs. And, Mabel Rice and colleagues (2010) provided updated norms for kids between ages 3 and 9 with and without language impairments that can be referenced. Both of these articles include a description of how the MLU was calculated so you can replicate their method to ensure that it makes sense to compare your client’s MLU to their norms
In addition to MLU, LSA can be used to calculate other measures of semantic diversity, including:
NDW: Number of Different Words
TNW: Total Number of Words
And, from these two measures, we can calculate the:
TTR: Type Token Ratio
- Total number of different words divided by total number of words
- Templin’s early (1957) work tends to be the most frequently cited as far as guidelines for counting total words (tokens) and number of different words (types). And, this website will calculate number of words and number of different words for you; all you have to do is copy and paste the child’s utterances (minus other speakers’ utterances and any kind of numbering).
- Miller (1981) and Fletcher’s (1985) early interpretations of Templin’s work tend to be the only sources cited as far as interpretation of TTRs. They suggested that typically developing 3 to 8 year olds should have a TTR of about .5. So, TTRs below that level suggest that the child is using the same words over and over again. But, definitely use caution when interpreting TTRs in general because they’re situationally variable (meaning, in some situations, it’s perfectly understandable that a child would use the same word or phrase over and over) and vary with sample size.
In addition to performing these analyses by hand, you also have the option of learning (or, possibly relearning) how to use one of the programs that will do the work for you. The most well-known is SALT (Systematic Analysis of Language Transcripts). If you’re interested in learning more about SALT or just re-familiarizing yourself with the program after not using it for a while, you’re in luck because the website is full of free information, links to short You Tube Videos that walk you through using the program, and self-paced online courses. And, not only that, but almost all of the courses can be completed for (also, free!) ASHA CEUs.
In addition to information about coding conventions and using the program, the courses and website also contain free resources and information about the different contexts that we can use to collect a language sample. You can also access free scripts for the different elicitation contexts so that you’re consistent when you collect your language samples. The website also includes free audio files of stories and graphic organizers that can be used along with the scripts for narrative, expository, and persuasive samples. And, you can find the age and grade levels of each in the norms database on the SALT website here. And, speaking of the database, this article provides a general overview of how it was established, and then discusses how they can be used to identify children with language impairments.
You have the option of downloading a trial version of SALT on the website. This is a good place to start so that you can use it to complete the practice activities in the courses before you decide if you actually want to buy the software. If you do decide to buy the software, it comes with a pdf version of the clinician’s guide to LSA, which is really handy resource for language sampling in general and coding and analysis in SALT.
SUGAR (Sampling Utterances and Grammatical Analysis Revised) is a newer method, and is a procedure rather than software. It is a streamlined method of collecting, transcribing, coding, and analyzing language samples elicited from conversation. Here are some links that will help familiarize you with the program:
- Pavelko and Owens (2017) recent article discussed use of SUGAR to analyze a language sample, and provided evidence to suggest that the analysis documented age-related changes in children’s language development. We provided a review of it here, including a general description of the protocol used for SUGAR.
- The SUGAR website includes free informational videos that walk you through collecting, transcribing, and analyzing a sample, and then comparing the results to the SUGAR norms. There are also a ton of other, free references and forms available on the website that include: references for collecting language samples, a guide to counting morphemes and calculating MLU, a description and corresponding forms to guide you through a sub-analysis of grammar based on your client’s age, and age-based norms. And, perhaps the best part: there are fantastic, free intervention resources available to target specific grammatical structures on the website!!! These include a description of the developmental sequence, specific populations that tend to demonstrate a deficit in that area, research-based intervention methods, and a list of recommended materials. So, so useful!
- Note: Not everybody is ready to use SUGAR. Making the LSA process as quick and easy as possible is definitely a step in the right direction, though, so we're looking forward to further refinement and analysis of this process!
The other alternative to SALT and SUGAR is the free, research-based software called Computerized Language Analysis (CLAN). CLAN is like SALT in that it's a software program. Pros are that it's free to download and has excellent data on languages other than English; the con is that is has a bit of a learning curve. Our recent review of Ratner and MacWhinney’s (2016) article regarding the use of CLAN also includes links to download the free software and clinician’s book, and links to articles that focus on the use of CLAN and other related utilities that can be used to further analyze your language samples.
September 2018