Podcast Transcript - Episode 46
SLP Toolkit Podcast, Episode 46, Transcript
Click here to download the PDF version of the transcript
Sarah (00:36):
Well – hey, lisa!
Lisa (00:37):
What’s up, sarah?
Sarah (00:38):
Um, not much. What about you?
Lisa (00:41):
Well, you just had somebody leave your house yesterday, didn’t you?
Sarah (00:46):
This whole week has just been a blur – because my son came to town after being gone for what? Three months? He came in town for Thanksgiving week from college so that was awesome and I took him to the airport last night–he had a red eye out of here, which was so sad–but I was just telling Trent I felt like I hardly saw him because between work and–oh! We should talk about our new project–so that’s, like, we’re super busy with something else, unrelated to Toolkit. And then the holidays–it was just the craziest freaking week. So I feel like he's gone and I feel kind of sad, but he’ll be back in three weeks.
Lisa (01:29):
I used to come home, I always–I moved out when I was seventeen and I always would come back every few months. And it's a portion of hanging out with your family, but then you've got friends, you know? You're at that age where the social relations are, you know, important.
Sarah (01:46):
Yes, he was with his friends the whole time, which I was so glad because, unfortunately, I didn't have that much time for him anyways. We did go to top golf last night–
Lisa (01:53):
That’s fun.
Sarah (01:53):
–that's the one we said, what do you want to do? Is there one thing you didn't get to do this week? Because that's the other thing too–I think he loves the college life and he's actually doing much better than I thought.
Lisa (02:01):
Yay!
Sarah (02:01):
Him, leaving home and going clear on the other side of the country, but he's in a very tiny town in Virginia–about an hour outside of Roanoke. And, um, it's–it's boring. To me, it's like my dream quaintness, most beautiful–he gets to see fall for the first time in his life. But for him, he has to drive 50 minutes to get to a Chipotle. (Lisa: [Laughs]) And that in and of itself is a deal breaker. So he came home and was just like all the things: every restaurant, bowling, movies, whatever he could do. (Lisa: Aw…) So he was busy. It was good. It was good.
Lisa (02:36):
And then you said we have another project goin on, what's that?
Sarah (02:39):
Well, um. I was trying to explain this to somebody the other day, because I do think–I mean, if this is just how you and I work now? We got a hair up our ass, what? Like a month ago? Well, while we've talked–for a while we had talked about having our own space. So for those–I talk about this all the time, but if this is new information for anybody, I am a sixth generation native of Mesa, Arizona. One of my greats–what? So that'd be my great, great, great grandfather? Was one of the founders of this town. One of the five–I think there's five, anyway. So I have such a love of the city that we're in. And my grandpa served in the city. His building is where our office is located right now on Main Street in downtown Mesa. So for a long, long time, we kept saying, we should get a little piece of property here, maybe we should get our own space. Um, and then kind of looked around. Well, things are popping off in Mesa. Like for those of you guys, who've never been to Arizona, you have no idea. Like we've got darling towns, like Gilbert, Chandler, Scottsdale that are–their downtowns are amazing. And then Mesa is like [sad sound effect], but it's coming. It's coming. We've got–ASU’s bringing to campus downtown, all this exciting innovation is happening, and so we were like, it's now or never because we’ll get priced out real fast. And so we–we bought a property and it's–it's not quite what I thought. I was thinking, we'd just get like a little building, but we got almost an acre.
Lisa (04:04):
[Laughing] We got three little buildings!
Sarah (04:04):
Yeah! The main home was built in 1912 and that's where toolkit headquarters is going to go. So we had been gutting these buildings and thank goodness for handy husbands. And it's not–it's me, Lisa and another female partner who also–we each have our own strengths, but we're going to do this on a budget. So it's going to take us a while, but–
Lisa (04:30):
That’s my strength, the budget part. The part where I'm like “no ladies, we don't have the money.” and then you guys will say “Overruled! This is cute. We'll figure out the money later.”
Sarah (04:38):
That's like–I was looking at appliances and we want to stay true to the, um, timing? The time period, you know, of really keeping these houses in the early 1900s. So of course I'm looking at, like, vintage appliances.
Lisa (04:51):
Why don't we just get a fan and a block of ice for air conditioning?
Sarah (04:56):
We should’ve, because this oven I’m looking at is like $12,000 for the oven. And I’m all–
Lisa (04:59):
No.
Sarah (05:00):
I’m all “I don’t know if that’s a good use of money…”
Lisa (05:02):
No.
Sarah (05:03):
So then I went back to the drawing board and I just was able to order all the kitchen appliances for far, far less than that. So, we're doing it but it's going to be slow and steady. It's going to be amazing. And then: here's the cool news. The building–the main house will be our office, but then we have two [of] the cutest little cottages for Airbnb. So whoever wants to come on a visit to Mesa Arizona, you can rent our little cottages and hang out with us.
Lisa (05:29)
We'll add–we'll add actually the link. There is no such link at the time of this recording but, by spring training, there should be. So we can go back in and add that link in the show notes. Um, in a few months.
Sarah (05:42):
Yes. And if you want to follow along, I am documenting this whole thing because I do actually think this is insane. This isn't as insane as it was when you and I decided we were going to create an app like–who the hell did we think we were doing that? And then we think because we have done it, and then we were able to do Bright–the professional development side of our business–that now I think we're like, we can do anything, which is foolish, probably, and insane, but we're just like go big or go home.
Lisa (06:10):
It’s different hurdles though! I do think the first time we started a business, we didn't even know how to form an LLC. We, like–it, in the state of Arizona, is super easy. You just fill out some paperwork and pay like 50 bucks and it's not a big deal, but we didn't know anything about anything. So there's a huge learning curve when we started. So now I feel like we've got the fundamentals now it's more out, not over stretching ourselves because that's where we get in trouble where like, this is now going to be business number three.
Sarah (06:42):
Right. [Unintelligible] But also the money, like, we got really, you know, we really strapped all of that together in the early days. And people ask all the time, like, “Can you get small business loans?” And, and yeah, I think there are opportunities out there. We couldn't find them. So we had to max out credit cards, and I borrowed for 401ks and life insurance policies. So even this opportunity was different because we had investors! And I would have known the first thing about trying to get people to invest in a project. First of all, I don't think anybody would have the first time, but I think they trust us now that we're serious. You know, when we commit to something, we take it seriously and, anyway. So it's very, very cool, but obviously insane. But if you want to follow along, it is Act On First Street. I'll put the handle in the show notes too. That is where all of the shenanigans of us trying to renovate a very old property–
Lisa (07:35):
On Instagram!
Sarah (07:37):
Just on Instagram. Yeah. Yeah, yeah, yea, yeah. Okay. Sorry. That was like a giant tangent, but it's because all these things are just constantly on top of mind. But what is really happening this time of year, Lisa?
Lisa (07:50):
Well, it’s one of those times of year that everybody… I think it's a mixture of excitement and fatigue and crying and overwhelm because you're excited. You're about to come up on winter break and actually get some R&R, spend some time with some family, just relax. But these last few weeks leading up into that break are just brutal because everybody's favorite! … Progress monitoring. Progress monitoring is the worst. I love–our friend, Anne Page, Beautiful Speech Life, had compared at once to laundry; like just when you feel like you got caught up on it, it's time to do it again.
Sarah (08:30):
And it happens at the worst times, like you said. It's usually before a fall break and then it's before winter break and then it's before a spring break and then it's the end of the year. It is the timing–the timing of progress reports in and of itself sucks, but then the stress of progress reports. It’s tough for somebody who already started on their progress reports.
Lisa (08:53):
I used to try to start this time of year, I would try to start that first week of December. Yeah.
Sarah (08:58):
Wow. I'm the queen of procrastination. I did it the day before, they were due!
Lisa (09:03):
I always I feel like I did procrastinate and then I would back it up a little bit, just due to the fact of… kids are absent too. They start leaving. And especially like in our region? We had a lot of families that took off for like a month to go to Mexico. And so they would miss like that week before winter break and the week after winter break, because they would leave for kind of an extended time period? So I just got to the point where I was like, I need to start pulling them sooner because I would stress out that I didn't have the data, I need to complete the damn things. And when we have–you know, my average caseload at an elementary school, when I work full-time at an elementary school, my average caseload was about 60 students. And on average, my kids had anywhere from about two, two-to-three goals sometimes, um, more if I was breaking out some–some articulation kind of goals? But that really led to, I had to come up with scores for a progress report for a couple of hundred goals! It's insane! And what are some of the–you know, I know it's overwhelming just the sheer number–but what are some of the things that kind of tripped you up in the process in general? You know, waiting until the last minute?
Sarah (10:18):
Yeah, yeah. One: the mad brush scramble–especially when you're on a team, you know, where we–where the student has a team. And so they're all waiting for you to get your part in? You know, thasucks, you know, and they're like “Sarah, we wanted to send these home today.” and I'm like “I'll do it! I'll put them all, send them in their backpacks later, whatever.”. So yes, always timing because I waited too long, but then it always came down to data for me that–there is–that's a real crappy feeling when, um, you have to scramble and make some smart guesses? Throw to a dart board?
Lisa (10:52):
Well there’s two reasons–
Sarah (10:53):
How do I add data on that progress report, right? I can't just say they're doing awesome, two thumbs up! That doesn’t count, does it?
Lisa (10:59):
Well, my Medicaid billing went one of two ways. It was either played–played game? Making progress? Or, it was a percentage– a random percentage. Like even something as simple as an artic goal. Maybe the goal is “R in conversation 90%”. And maybe I was working at the word level. And so I would just put “R words 20%” or whatever it was, but then I'd be like, oh crap. I didn't write where they were in the R words. I don't know what they're at as far as how the goal is, I imagined–so this is where the smart guests as well, if they're only 20% of the word level, they've gotta be at least what, like 0% in conversation? So let's just write a zero and call it a day. I mean, that was–
Sarah (11:43):
That still trips me up because I know that when we're running our progress report, obviously we have to right where they are at, the way the goal was written. So you could be at 0% all year!
Lisa (11:57):
Yes. I used to warn my–
Sarah (12:00):
Right. So, but then like, I would put in there 0% at conversation level, 20% at initial position of words. Like, I would put that in there because otherwise it's like just zero forever. And that's embarrassing.
Lisa (12:13):
I think the only goal that really–it was zero for a long, long, long period of time, was that stupid, R sound. It's so hard to get in and establish and start to carry over. So even when we met for the IEPs, I would warn families like you're going to see zero on this for a while! But in our software you still would put zero for the score, but there was always a comment section. So in the comment section is where I'd add wherever they were at in that kind of hierarchy of–of getting there.
Sarah (12:38):
And I will tell you, as much as I complain about writing objectives–and we were only in our district, we were only required to write objectives for students who were on an alternative assessment–um, thinking about this now, like we do the way we write it, you know, we were so focused on how to write really quality objective goals and, and all of those things? I got to tell you, I think the greatest thing–service you can do for not only your students, but yourself is to write objectives. Because then it would be.. the annual goal is that they would be at whatever percentage in conversation at the end of that year. But then I would have those objectives to really break down: how am I going to get them there? And then you would see progress on the objectives.
Lisa (13:19):
Well, and I think we do that even if we only do it mentally for Artic, because we do work on that. It's a very kind of hierarchy sort of based skill that you're–you're establishing in isolation or phrases or words or phrases or whatever, early on, hoping to get them to conversation eventually. But what about synonyms?
Sarah:
Right.
Lisa:
What about using context clues and vocabulary? We just don't think about, we know that eventually that's what we want them to do, but that's where it can get us tripped up.
Sarah (13:48):
Right. So if you took the time to break that down, what does it look like? Where are they at right now and how am I going to get them to that? And I really broke it down–I just think, I mean, that in and of itself would help guide everything you need to do that quarter? And then I probably would have had the data I needed at the end! But I didn't ever take the time to really think through this long-term plan. I would just, you know–student can’t do synonyms and antonyms, I'm gonna give him a goal for that, then we're just going to go and I'm going to focus on my teaching. And then the progress report time would come and I would be like “Oh, shit”.
[Unintelligible overlap]
Sarah (14:28):
Maybe one data point?
Lisa (14:30):
It would help knowing when you're off track too. I think that's one of the cCtch 22s we get ourselves in is… I know I've got to work on this goal and then I know I've got to complete a progress report and then come that next IEP–if you have 0/0/0/0/0? Then you're–you are making yourself, um, in a position that, you know, parents would question that–I would question that! Even that's myself, I've had a couple of students where I've looked and I'm like “Oh crap, why did I not stop this earlier in the year?”. And I think it's because of that sort of frenetic, like we're just working at things and then, oh, I need to complete a progress report–oh, where's he at? He's at this? All right. That's the report, check. I did what I had to do. I had to complete a progress report. But then I really wasn't doing that student justice.
Sarah (15:16):
Right. I think in terms of data, we talk about it very often in this field, obviously we're trained in grad school to take it, but we talk about data in this sense of like the data you take during a therapy session? And I don't think I ever had a good grasp on the fact that we need three different types of data throughout that child's treatment, right? So you need the data to determine their strengths and needs. So that's the information we're writing in their IEP, but then we need the data to determine treatment efficacy, which is what you just talked about. So therapy data, and then we need data to show progress. So that's progress reports, where in our minds, I think, because it's all just–we don't even think about present level assessment or present level data as data. We don't even think about that until we go to write that IEP. Um, but we always think of data just as like, there's just this one type of data. It's the little data collection notes. I have (Overlapping begins with Lisa saying the same thing as Sarah, agreeing) pluses and minuses, pluses and minuses, (Overlapping ends) but that isn't true. And so when I really sat back and thought about how can I make progress parts less painful? First and foremost, I’m gonna stop relying on those pluses and minuses on my little datasheets.
Lisa (16:26):
And looking at what is the kind of data specifically I need. So when you talk about strengths and needs, I've got to do some sort of screening assessment as far as–and not assessment like evaluation–but to get some different data points, to know a global kind of picture of how the kid is doing. In therapy, I already know what my goals are. So I've got to have data that's specific to wherever we're at working on that goal to keep pushing them along that continuum. But then, like you said, on progress reports? My data has to be specific to how that goal is written and that's where–it can even get tricky, I think of something like a story retell kind of goal that maybe we are early on in the goal. And we're just working on identifying characters and sequencing, like knowing the beginning, middle and end of a story. So they're not even putting it together in a retell yet we're working on some of those kind of foundational skills. So that's where all of my day-to-day therapy data is to keep driving my next session but then that's where I get in trouble. When I go to see how they're doing on retell. And I'm like, oh crap. I don't know. We haven't even done a retell yet this quarter because we've been so focused on the individual pieces of that. So, that's where I know we became huge proponents for purposes of just having the data we need, but then also just efficiency of data for progress reports of using criteria and reference tests and rubrics. And for us criteria reference tests where something, both of us were introduced to in a district that we worked in, in Mesa, Arizona, and it was just for articulation. But there were criteria and reference to us that were provided to us that it would have say the S sound. We would have it in beginning, middle and end positions. We had data–these sheets that we would use to give the assessment to see how the student was doing in–on that articulation sound. And then we also had one rubric that was just kind of a generic rubric that looked at, you know, if you're working on something like social skills, it's not usually a plus and minus kind of skill it's, well, what level of queuing do they need? And how do they look kind of compared to their peers? And so it's a more dynamic kind of skill, so we need more dynamic information. So it teased our brains that when we were first even developing Toolkit, this was kind of the springboard for the whole app. It was this idea of making progress monitoring less painful, making it more efficient. And so we have hundreds of criteria reference tests built in at this point. And you can actually, you can create your own, you can share them with other users, but we have hundreds built into the software. But they really do make it really nice in order to not just get a baseline when you write a goal. But then you can focus on that therapeutic data. And when it's time to do the progress report, take the–give them either that criteria and reference test or pull out that rubric and reflect on your, therapeutic data to fill in that rubric. And, um, you always know–
Sarah (19:24):
Can you explain a little bit about specifically the term “criterion reference test” just for those who are unfamiliar? Like, it was news to me at first time I ever heard that because it's, to me, it's something like teachers do. But what does it mean?
Lisa (19:36):
So a criteria and reference test is where you, as the SLP, determine the criteria. So you are measuring a student's performance against a predetermined criteria that you select or, if you're using a pre-made criteria and reference test, that we are, um, there might be the stimulus items on there, but you're still saying that I want this student, on this test, to get X score. And that's how I'm measuring their baseline data. That's how I'm retesting the student's performance, each grading period in order to monitor their progress. So the benefit of that is that you always will have a way to measure progress. It will be consistent, because I know you and I have talked about too, that even when you have a goal, like synonyms, if you're just relying on your therapeutic data and one day you notice they're at a hundred percent, because you were asking for a synonym for big, versus another session, they were at 20%... And when you look back at your stimulus items, maybe you were looking for a synonym for foliage. You know? Those are two different words, different expectations. And so you're going to have random scores as a result, which makes it really hard to interpret, you know, from that kind of global perspective. So, that's the idea for criterion reference test is you measure a student's progress over time using the same set of criteria and you're looking really at how they are taking these skills they're learning in therapy and applying it to this test. So you're not teaching the test, it's not meant to teach me these items.
Sarah (20:06):
No feedback, right?
Sarah (20:08):
Right, no feedback. You put it away for the rest of the grading period until you need to use it again the next time. But they are best for those kinds of skills that are your plus and minus data. So they're great for articulation, they're great for grammar, they're great for “wh” questions. (Sarah offers “Vocabulary”) Yes. Vocabulary, comprehension question. So anything that you would normally measure with a plus and a minus, that is where criterion reference tests will allow you to kind of zoom back out. So you're in your sessions, you can be real zoomed in and focused on what are we doing today and what do we want to accomplish next time? And then zoom out.
Sarah (21:46):
Here's the part, it takes away, um, any of the thought required for what do I need in order to determine if the student's making progress? Because even your example you gave about synonyms is–is 1) I would've had to take the time to analyze that, like you–like you said, you know, one session, he's got a hundred, the next session he's got 20%--wait, it was an issue with my stimulus times? How the hell would I know you think I remember what items I used the day he got 20%? No, I probably pulled out my super duper fun deck, um, and so there's so much interpretation required for your session data. And you've got the–we talk about this a lot and that's a whole other conversation we're talking about data, is, if you're smart, you're gonna take the time to analyze that right then in that session, or at least prior to the next session, right? Like while you're in that, uh, teaching time and, um, you know, working on the goal, that's when you really want to look and go wait, shoot, he was a hundred last time, why was he at 20% today? Well, crap, I need to–and do you want to do that? What happens with progress reports is, if I'm just looking back over the last quarter's worth of data and I see those data points, you think I'm going to remember any of that? No, I'm not. So now I just eliminated any contextual problems with data, or did they have bad day, one day or, um, you know, [Unintelligible overlapping] Did the SLPA teach that day. Did that like, yeah–do I have any data to look at? So not only do the criterion referenced tests, like, make everything so much quicker because I literally grab it off the shelf and administer it, I'm only–for purposes of reporting progress, it is more effective in my humble opinion, because I don't have to think about all of those reasons the data might be all over the place.
Lisa (23:43):
And time efficient. Because you can do this so quickly, you have a process in place that, you know, when it's time to do progress reports, you already have these, um, tests established for the students because you've used them for the baseline for their IEP. So it's there, it's assigned to the student. You just have to redo it at each progress report time. Versus again, if I'm looking at and trying to analyze the data in my files? I mean, I have to pray that I even have data! I have–if I do have it, I have to figure out, well, am I averaging the whole quarter? Or am I averaging–
Sarah (24:18):
My favorite is the goals were, I would say, um, student will be able to identify a synonym, or whatever, you know, with 80% accuracy over three consecutive sessions. What did I just–what kind of failure did I just set myself up for? Because then, I need to have data from the last three sessions I saw with that student prior to that progress report period. You think I did that? No!
Lisa (24:43):
So that's the kind of trouble I think that we get ourselves into. And that's why that's part of what makes it so stressful. The other part is what I'd kind of referenced before is that not all goals are black and white. So I literally remember not wanting to write goals like social skills goals, or any kind of goal that I couldn't think of “well, darn how do I get that plus and minus?”. Um, I would just kind of avoid them and I would put in the IEP something about like, well, this is an area of need and we're going to work on this within other activities because I'm like, oh, okay, we're still working on it, but I have no idea how to measure it. So rubrics are a really great way to–to alleviate some of that anxiety. If rubrics are new to you, um, you may, again, these are tools that teachers have used forever. I mean, I even think of the writing rubric, the six traits writing rubric that they often use and they use it in–in lots of different ways, but the idea of a rubric, again, you can create these! You can usually find these as well, but you're looking at a skill. You're looking at where on the hierarchy, are they on that skill? You're looking at where are they using that skill? What kind of support do they need to use to use that skill? So you're making it a really dynamic collection of data that I think makes a lot more sense to parents as well, because I always think of even, think of a basic kind of social skills goal, where you want a student to initiate a conversation. You'll see those with 80% accuracy. And then you're like, okay, but great. He can initiate with, um, his one-on-one aid, he can't initiate with his teacher, he can't initiate with his peer unless the peers on the playground, and then he really is–and it's a motivating kind of activity for him, then he'll initiate that he wants to play or somebody. So there's all of these moving pieces that go into [Unintelligible].
Sarah (26:36):
And then am I supposed to follow him around for 30 minutes? Tallying every time he did it?
Lisa (26:41):
Right, so that's to get a percentage. So with the rubric, you don't have to do that. You are literally taking your qualitative data–and it could be data also that you collect from other stakeholders: the teacher, the parents, if the student has an instructional assistant, that's assigned specifically to them… You're able to–to compile all of that, to get a really robust, um, answer as far as progress that makes sense to the team. And then on top of that, I always think too, for, even, for some of our students that are–that–that make gains that are huge gains for them, but maybe smaller, um–or take more time if we were comparing it to another student, that rubrics are great because you can really pick out those nuances, like, hey, maybe our score didn't change. Maybe they're still at zero for being able to do it, but look, we moved from them needing full assistance to now we just have to give one cue! Or maybe we moved them from where they can only do it if it was a super structured activity, but now we've started to move where he could do it, um, in another setting or whatever the case is. So you can show progress in different ways, which again, I think for parents is so meaningful and for us as providers, it helps to really hone in on, okay, we've got it here, what's next? And that–
Sarah (27:58):
Gives us so much hard context that 80%, 20%.
Lisa (28:01):
It connects into what you were saying though to that, I think, it's almost like if we're taking that goal and we're basically giving it objective, short-term little benchmarks to reach. Once you get to one benchmark, you move to the next, and that's why it becomes so powerful for you as a therapist to drive your–your–your next decisions in treatment. And then also to document them.
Sarah (28:25):
Yeah, so the rubrics are probably my absolute favorite thing ever. One, because like you said, we had to task analyze a skill, um, in order to figure out how to even create a rubric. Um, and so we spent so much time researching and doing that? Um, but it–so that in itself helps to guide my therapy? But then two, I get so much more bang for my buck because if I am, um, doing something like a story retell, like–you give an example. My old goal might've said something like the student will retell a story currently on three to four out of opportunities tracked. Okay, so what? They're just going to retell me the story and I just say yay or nay. Right? There's so many factors involved in story retell! But one of them can even be grammar or word choice, vocabulary. Like there's so many things that go into being a really good story reteller or a storyteller that I can target so many skills in this one goal. So now the goal looks like, you know, um, Johnny will, um, retell a story scoring 14 out of 16 on the story you're retelling rubric. Period. Then I attach that rubric so that everybody on the team has context to what the ultimate goal is for him and what he has to do, but it is–it's not just, can he do it, yes or no, but where is he getting tripped up? Is it that he doesn't understand story grammar markers? Or is it, um, that he doesn't have, uh, the sequencing in a logical order? Or is it his grammar is just so, um, uh–what's the word I'm trying to say?
Lisa (30:04):
Terrible? (Laughs)
Sarah (30:06):
(Laughs) Well, yeah! I mean, at that impact–that impacts your ability, right, significantly to tell a story effectively. So, anyway, I think that the rubrics are just, like, the coolest thing ever.
Lisa (30:19):
Well, they take goals that tend to be subjective and make them objective. Because like you had said for–I had inherited a goal once from somebody that was exactly like that. Like we'll retell a story, whatever it was. Four out of five times. And when I looked at the last progress report, they were at like, I think four out of five . And whenI tested the student, I'm like, uh, I'm at zero out of five. So either I failed–which is how you start to feel–which in–I was really big on I'm like, I don't know what they were measuring, there's nothing in the comments, I have no clue what they use, but how I perceive a story retell to be considered “yes, we did this” is clearly very different than the previous therapist.
Sarah (30:58):
Exactly right. And that's the, that's the problem. That's–it, it makes me think of even when we–so when we first got the idea to, to create a SLP Toolkit, it was–it was solely for progress monitoring. ‘Cause progress reports were so freaking painful? Um, but then it did–it led us to think: wait, but, um, I don't even know if I'm writing really great goals that are helping my progress report? It
Lisa (31:18):
It was as a domino effect.
Sarah (31:21):
I don't even know how, like how do we determine which goals to give a student? I need a better way to assess. Like it all just kind of like, and it, that again, the luxury that we had to sit down and really think this through, and then kind of scaffold that, um, into a tool that like helped everybody else, who I assumed was feeling the same kind of angst and struggle I was when it came to goal selection, goal writing, progress reports–they all go together and it's all about data. And so I do, I don't think there's a better way to do it. So, um, for those of you who have not been using something like criterion reference tests, or even a checklist or a language sample or a rubric, any other means than SLP data, then you really, really are doing yourself a disservice because it just it's–it's game changing. So, um, like right now, this is about–we're probably about three weeks in, uh, are we have about three weeks left before school goes out. So tell me kind of your process of, um, what you would be doing now to get ready for your progress reports.
Lisa (32:28):
Well, before Toolkit or?
Sarah (32:30):
No, no, no, no, no! Now that we know–now that we've convinced everybody not to use SLP data as their criterion for their goals.
Lisa (32:39):
Thank you for clarifying. Um, no. So here's the deal: I did use criteria and reference test in rubrics prior to the creation of Toolkit, but they were in paper form. So you can–these are tools you can create yourself, I used to keep them in student files, so part of my process would be that I would pull out all of the student files for my filing cabinet and I would, um, get that criterion and reference test and I'd have it ready and I'd pull the students in and usually how, how I tested–and I know this can be different because you tested yours still within their–within groups, So we can talk about that too–but for me personally, I tended to tell teachers that okay, for the week of, whatever it is, December 7th, whatever week I was doing the testing, um, I'm going to pull your students one-on-one so I can get their progress monitoring data completed. It will be around their regularly scheduled speech time, but it–they'll come for about 10 minutes each. And so, just know we might be a little bit off that day. So I pulled them in, give them their test, and then sadly, because it wasn't digital, their folder would go on my back counter and then I'd still have a lot of kind of time afterwards that I'd have to score everything and then get it in the system. So the beauty of having these in a kind of digital format is it–you can get rid of all of that kind of extra process where all I have to do now is focus on getting the kid in there, pulling them up, clicking on their name, the test is waiting for me and I administer it, and it scores it for me. So that's beautiful! And then I know, I remember when we first, first launched, you were like, oh my gosh, I literally was able to complete my progress reports with this group before the next group even came in. Because you can have your tab open for toolkit on one tab of your browser, and in the next tab, have your IEP software open where you fill in those scores and then they're done!
Sarah (34:34):
Done. It was so freaking magical. I think I would start maybe like a week or two before progress reports were due, probably two if I was smart. Um, and I did, I kept all their sessions the same. They would come in, but I had an activity for them. And I figured, you know, what, what the hell, it’ss one session? You know, I'm just going to give them something like–not like fluffy, fluffy, but like something fun that they can independently work on, whether they're a kindergartener or a sixth grader. Um, and so they would come in independently all work, I would pull one over to my desk, do–administer his, um, CRTs–’cause the rubrics I'm doing those on my own, but he had, if he had say four goals, three of which are being measured by a CRT. I administer the CRTs next kid. And then I would just rotate through them. And then literally as soon as I had that score, I would just go throw it into the–into that IEP software onto the progress report and done, done, done. It was the most amazing feeling in the world to just like, I–I'm not kidding. I used to take a good–probably two days of pulling on all my binders and like diving through any notes that I hope I made or data points and trends and patterns over time and panicking because I was missing it. And it's too late, by the way, they're all home because I waited. So I can't get it. So then once in a while though, if I really didn't feel like I could make a smart guess, I would just have to say, I'm going to send Johnny's progress report home after the break, and then I'd have to pull them in. But again, then it–pulling them in and then what? Randomly just like pulling things out of thin air to assess? That’s not going to do it. So, um, having this, like, consistent measurement that I know I'm giving every single time and then throw that score in there? And I'm just, I'm done. And that's what we came up with. We did for this for a presentation once–I got to find it and I'll attach it to the show notes, I'll also be attaching an example of a CRT in a rubric. So that if you go to those of you who want to start creating these yourself, you can have a template to do it. Um, but I'll find this too. We even came up with a list of common things you might want to–’cause you know, sometimes you want to add the notes, if you have that on your published report, you don't want to just put 80%? Like sometimes you want to say something in addition to–so remember all of those things we brainstorm. So I'll add that document too. So sometimes I would add a little things, you know, like: Johnny is starting to show some more self-awareness! Or, you know, sometimes it was just even like, um, I'm so proud of everything Johnny's done this quarter, have a great break! Or, you know, whatever it was, but just, I would kind of have some of those things ready to say that were more like anecdotal? But the fact that the score, I just like threw that in there and I was done?
Lisa (37:04):
Amazing. Well, and that's where going back to the timing of when progress reports are done, what ends up happening, if you don't do them before, is: you either a) bring them home and b) either work on them over your break, or don't work on them and stress about them your entire break. So it's nice to be able to just leave work at work, take your well-earned, well-deserved break is so good.
Sarah (37:30):
And feel confident that the information you just found the progress report was not darts to a darn dartboard! Like that's a great feeling to be like, cool, I'm using that same thing. Now–oh, I was going to say, I'm using the same thing I get my baseline on so I know that the data is going to be consistent. Um, what if you are new to using CRTs and rubrics–let me rephrase. Can you use CRTs or rubrics, um, after the goal’s been written, when you didn't use it as your–your measurement tool in the goal, um, and you didn't use it for your baseline data?
Lisa (38:02):
Yes and no. Yes. For criterion reference tests, no for rubrics. Rubrics are very specific measurement tools. So that's why even you had referenced attaching it to your IEP. Those rubric scores are meaningless without the rubric. So, you could use it for your own purposes of kind of driving your treatment? And even having, um, more information that you could put in the anecdotal section? But no, you've got to kind of stick to how you've written it. The beauty of criterion reference tests is they are therapy data. You're giving them in your assessment, your ongoing assessment that you're doing for students on your caseload. So that is not evaluation. You don't need permission to test you doing ongoing assessment. It is part of your, your SLP data. And so I would just start using that. I think it will be eye opening because, again, if you're–if you're not used to using something consistent or you were working on quote unquote smart guesses, your score might be way off. Or, sometimes you surprise yourself where like maybe you left in October and the kid was at 20% and then you give a criteria reference test and you're like, oh, whoa! They do a lot better than I thought they did. Now that I've put it in this context–
Sarah (39:11):
Right, now that you have data–
Lisa (39:14):
–You do you have this skill! Or maybe you're just a really remarkable SLP that got them to a hundred percent in two months. But yes. So yes, to criteria reference tests: I would start using them now. Rubrics? Start thinking about your next IEP and if you have any of those tricky goals that are more, um, complex and maybe more dynamic in nature that you would want to start using those and then use it for your baseline and keep going from there.
Sarah (39:42):
Yeah. I love it. I think that's perfect. Like if you have not started, you know, analyzing your data for progress reports, use the CRTs now. So if you have the goal that the student will produce, you know, the “TH” sound, pull out one of these criteria reference tests–oh, you can sign up for free for the first month for Toolkit!
Lisa (40:05):
Yeah, I was going to say, so if the–if this is new to you and you want to give it a go this time? I am a big fan, like you had already said–we just sort of jump into things. I'm all for anything that helps me, I will give it a good college try. So sign up for an account. If you have never created an account before you get your first three 30 days for free! So you can access all of the tests and rubrics that we've created and loved.
Sarah (40:29):
Click on that student tab to get–so you have to create the students. So you put Johnny's name in there, then you go to the student tab and then you'll see the progress monitoring section, and when you click on it, it takes you to a library that has all of our builds and tests. So you can search by keyword or by, um, content area and try and see if there's already something there created. And then the cool thing is, is, you know, again, we tried to literally think of every single goal we've ever, ever worked on with every child on this planet, but that's impossible. And because IEPs have to be truly individualized, there–we couldn't have thought of everything. So there is a rubric builder and a criterion test maker, um, in there too. So you can either like copy one, but then edit it and tweak it so that it works better for you or you can start from scratch and make your own. Um, but anyway, definitely give it a try this next progress report! I swear it is going to make your lives so much easier. Um, and what a gift to you to be able to get those done, knock them out, go home, and enjoy your insanely well-deserved break.
Lisa (41:35):
Word! I don't think, I don't think we can end on a better note.
Sarah (41:38):
Right, right. And email us if there's any questions. I feel like you guys–I know I talk so fast, I'm so sorry, and I always should probably say this at the beginning of every podcast episode, you know, you can change the speed, right? Just go down, go down like a half, and then maybe I'll sound not like I'm–do you remember the micro mini machine guy? That ages me. Nobody else would remember that. The one, they sound like an auctioneer? Maybe it’s commercials. No? That's what I sound like. Anyway. So, if there was anything that you felt like we just kind of skipped over too quick or you want a deeper dive. I think we'll definitely do another episode just specifically about session data and goal writing? Um, because again, these all go hand in hand, and once you are confident with the goal selection, how you write your goals and the measurements for taking data, it's going to be–it's going to rock your world man. So email us. You can for Toolkit related stuff, you can email us at helloandslptoolkit.com. Um, and then for, like, more things you want us to go, um, confess to on this podcast that's podcast@SLPtook it.com.
Lisa (42:45):
All right, guys, we hope you enjoy rake. And, and, uh, let us know if you get, if you have actually given us a try, I would love for you–you to email us, let us know how it went. And like Sarah said, if you have any questions, we're here to help.
Sarah (43:00):
Love you guys.
Lisa (43:01):
Bye!
Sarah (43:02):
We'll see you next year!