Podcast Transcript - Episode 13
SLP Toolkit Podcast, Episode 13, Transcript
Click here to download the PDF version of the transcript
Sarah (00:36):
Well, hi Lisa!
Lisa (00:37):
What up Sarah?
Sarah (00:38):
Why do I feel like we haven't done an episode in a long time?
Lisa (00:41):
We haven't done one with just the two of us. I've been missing us.
Sarah (00:44):
We had guests, which changes the dynamic a little bit. In all the right ways, but it is different.
Lisa (00:50):
I'm not crammed next to somebody like usual. When we have a third person in the confessional, I get the other person, not you.
Sarah (00:56):
You do!
Lisa (00:57):
So it can be awkward.
Sarah (00:58):
You have to sit really close next to somebody. And I just take all this room on the other side of the table. It's because I'm dealing with the computer and technology. And because I'm the boss.
Lisa (01:08):
Of?
Sarah (01:09):
Everything! Um, so I'm excited about this episode because it's everybody's favorite topic. It's like super glamorous, but before we get into it, I did just want to talk really briefly about this new Facebook group.
Lisa (01:23):
Oh! I was like, what new--did I miss something?
Sarah (01:27):
Yeah. You didn't know about it, it’s secret.
Lisa (01:29):
Even though I'm an admin and I've been in on all the conversations. I forget sometimes it's a secret group, but um, I'll send you an invite. No, the pediatric SLP group. It's been amazing. It's been only like what? A week and a half?
Sarah (01:43):
Yeah, barely a week. It's got over 2000 members already and the conversations… It literally–it's such a breath of fresh air. It's everything I hoped for in a community. Do you think?
Lisa (01:55):
I do. I think that what I was impressed with is that even within our first 24 hours, we had a thousand members, but then people were actually even regularly posting really thoughtful questions.
Sarah (02:08):
Yeah.
Lisa (02:08):
So it's not like a new group with no direction. No people are using it already.
Sarah (02:13):
I kind of expected people to be like, okay, I'm going to join it. Cause I'm curious, what is it? But then sit back, you know, how you just sit back and wait for someone else.
Lisa (02:20):
Right.
Sarah (02:20):
Nope. People started posting questions right away. And what I loved the most, uh, people were posting questions. I don't see any other groups being posted because it's those kind of–I always think, especially when it's specific to therapy, I know I hesitate to ask because I always in the back of my mind think, should I ask this or should I just know?
Lisa (02:37):
Right.
Sarah (02:37):
The answer? Um, and so anyway, great questions. And it's so respectful. The responses have been helpful, but really kind. And even there's been a couple of those topics that are like based off topics of other groups that people have talked about. And I'm thinking specifically with–
Lisa (02:55):
The frenulum.
Sarah (02:55):
And the tongue tie one, that's a hot topic. And sometimes it's a no-no you don't even bring it up. Have you–the dialogue–
Lisa (03:03):
If, but here's the thing like, because I think in other groups they call them what? Like garbage–dumpster fires or whatever it's called? Like, I feel like the members in this group, there's a really diverse mix. And the norms for the group to begin with are: everybody has the right to an opinion. Everybody has a different perspective. So be respectful in sharing that. But then, um, these are authentic questions because even though some conversations can go sideways in, you know, maybe the wrong context, they are still real questions. Sitting in a school, you get parents that ask you about clipping the frenulum that their doctor's recommending it or whatever. What are your thoughts? And if I don't really have resources or have had a thoughtful discussion in any other context, like what better way to kind of get everybody's different viewpoint on that then in a group with a bunch of different SLPs
Sarah(03:53):
Because people are doing it respectfully and, and feel safe to give their experiences. Um, I have been, I don't know if persuaded is the word–but I have–it's making me take a step back and look at both sides. And I think it's done that for a lot of people. So you might have gone into that conversation with this very clear “I'll never, I will never, there is no research to support it”, but then when you start hearing people's real life experiences. Again, clinical experience is part of EBP. So when you start hearing these people’s experiences, individuals' experiences, it makes you think for a second, maybe there is more to this and maybe we should consider other things. And maybe even just, I think it more than anything–itcame down to maybe it's just okay to have conversations about things that are hard.
Lisa (04:36):
Right! You might not change your mind, might not, but you've got at least some perspective from other people. And that's what I think is cool about this group too, is that we are–it is a pediatric SLP group. So it's not just school-based, it's also including clinic, but I've noticed a lot of people from other countries. So it's not just the US.
Sarah (04:52):
Right? Which is so great.
Lisa (04:53):
And we've got a mixture of even researchers, students, lots of different kinds of–
Sarah (04:59):
Business owners.
Lisa (05:00):
–Of people in there. It's amazing. Which again brings different perspectives, yeah.
Sarah (05:03):
And that's what we wanted. We have for a long time talked about–we want a Facebook group where people feel safe. And I think that comes from being out and about and talking to so many SLPs and they didn't feel like there was a platform where they could ask honest questions that they had things that they were truly struggling with because they were worried about, um, either this, you know, a dumpster fire situation where people go crazy on you or to like maybe feeling dumb that they should know that why are they asking it? Um, and so we've talked about it for a long time and there was no action. We did nothing. And then we joined up with some of our friends and Chanel was one of them. She's one of the admins of the group. And like within three minutes the group existed and–
Lisa (05:43):
then we're creating norms and rules, and very few of them
Sarah (05:44):
That was my big, big hope: minimize the rules. I don't think that this is, um, helpful when admins have control. I think this is a group that belongs to members and our only job is to maybe guide conversations. I don't ever want to tell somebody you can't say that, that's not okay. But I do think that our role is to make sure people stay nice. Obviously ethics–
Lisa (06:12):
no spam,
Sarah (06:12):
No spam, yeah. That kind of stuff.
Lisa (06:15):
Other than that, I totally agree, it's about people are asking questions because they are curious about it and want other people's opinion about it.
Sarah (06:23):
And we have so much to learn from each other. So anyway, if you have not yet joined it, it's called Pediatric SLPs. And um, again, [it’s been up] just a weekend and I couldn't be more happy with how it's going so far and I hope it will continue to grow and—and really be a helpful resource for people. So topic at hand, we've got a lot of, um, data collection on the mind right now. We've done some posts about it, our newsletter was just about it. And so I thought this is a perfect time to talk about data. And I want to start by saying, um, don't turn off this podcast.
Lisa (06:58):
But I, you know, it's funny. I was just visiting with--I'm mentoring a CF this year and I just visited with her two days ago--and a lot of the questions that she had were all directed to data. It really all comes down to data when, even in terms of, um–some of the questions about dealing with difficult parents or dealing with difficult teams or things like that. And I was like, when you make it about data and not your “I think”s and “I want”s and “I believe”s it makes the conversation a lot less confrontational because it's not personal anymore. And then you have that information to choose if your recommendations are maybe contrary to what somebody else thinks or wants or believes, you can put the focus on the data and not that individual person and make it this battle. So it was kind of interesting. I was trying to tell her that, you know, this is hard to figure out your first year, but if you can get into some really good habits of using data to drive your decisions, I mean–not like that's a novel concept, but in practice, sometimes it can be. We get data thrown down our throats saying that we have to collect it, but don't always necessarily know why and how that applies to real life other than–
Sarah (08:07):
And that's what I was going to admit to. I was always someone until fairly recently, I was always someone who took data because somebody told me I had to. I took it in grad school because there was a supervisor who required it. I took it in the school setting because I had to bill for Medicaid. Um, and so I did it because I knew–now, that being said, I know the best practice. I know I need data to make decisions. But my problem with it was I never knew what kind of data to take and what to do with it. So then I had no respect for it. It wasn't doing anything for me. So then there was no value there. And so it really took us once we started focusing on, um, the resource that we built, um, that we really started to look at data beyond just this paper with pluses and minuses on it, that it really was coming down to confidence and to making data-driven decisions and in no time greater than right now, is that critically important to our jobs. And so it–I had a huge mind shift [note: she misspoke and said “Mind Shit”, causing a new running joke for the podcast] at–
Lisa (09:10):
That sounds real messy.
Sarah (09:13):
Shift! A mind shift about data! And I love data now. So I hope you haven't turned this off yet because we want–we do want to share some solutions that I think are beneficial. And I know it's not glamorous, but it's so important because they're just a shift in our brain and that's real fun to listen. Watch.
Lisa (09:34):
No, I was just thinking, even if you think about back when we first started developing SLP Toolkit, it wasn't out of respect for data either. It was just, it solved a problem that we had where we knew we had to do progress reports and report on data and all of that kind of stuff. But it was the–it was still not out of respect for data.
Sarah (09:52):
Nope.
Lisa (09:53):
It was a journey for both of us. I think to really kind of–when we started looking, the collection of data became easier and we saw it represented in different ways, whether that be visually or whatever, that we started having these real, like a real yeah, aha moments and relationship with this data is meaningful. It's–I'm not collecting this because the school district tells me I have to, I'm collecting this because it's helping me not spin my wheels. It saves me time. I mean, there's so many benefits to it. Once you kind of allow yourself to go there and…
Sarah (10:26):
I'm sure that the obstacle I was facing is: I have always been this way my whole life, since I was little, when you tell me I have to do something, um, I, it, it, I need buy-in, right? Like I'm not going to do something just because you said, it's a good idea. I need to have that buy-in. And so it took me figuring out what it was, what was really, truly doing for me. And so going back to what you said about how, when we were building progress, the idea for progress monitoring, really, it came from a place of 1) progress reports take a long time. How can we streamline the process? So it's faster, but 2) I was real sick of not having data to report on progress reports. I was really sick of that feeling of crap, I forgot about that goal or, um, shoot, I haven't taken data on that goal in five sessions. How can I confidently report how that student's doing on that goal? That's a terrible, terrible feeling. And this, you know, we, I, we talked about this a lot, a lot where I actually, for a long time, I thought I was the only one that was struggling with that. And then when we start to talk to people and it's just a real problem, because we have case loads of 60 plus, and sometimes students with four to five goals are going to get missed. It didn't make me a bad therapist, just a real one. And so that was that solution of how do we progress? How do we monitor in a way where we collect sufficient data to feel confident about what we're reporting on? And so that, the solution, was our progress monitoring tools. Um, and then like you said, once we realized we've got this solution for progress monitoring, what’s the data for writing present levels? When I think of data, that's not even where I thought of data, right? I thought of data specific session, session data: here's the goal. Can they do this skill or not? It never crossed my mind that I also needed data to support the strengths and needs section in an IEP so that I could even develop goals.
Lisa (12:12):
Well, and that's where, again, if you think about any meeting that you've been in that's been uncomfortable, it's usually been because you haven't had sufficient data or the opposite to when–I've been in some of those meetings where advocates or attorneys push for something in an IEP and they don't have the data either but, you know, they know you don't have it. So they are usually, you know, louder and more persistent in what they want. And so sometimes districts can bend towards their will because there's no data on either side. So once I got into the system of really collecting some great data for my present levels, that gave me the confidence to then say again, well, show me where–if you're recommending something crazy–like, I had a family once that wanted a purple leather rocking chair, but show me where in the data that we have right now supports a purple other rocking chair. How was that going to provide the student access to their curriculum? This is the data I have. This is the data you have. Let's keep it focused on that. So it makes some of those uncomfortable situations, more powerful for you where, you know, again, you don't have that feeling of please don't ask me anything in this meeting, don't target speech and language in this meeting because I don't have anything to really talk about.
Sarah (13:23):
Right. Don't ask me why I picked this goal. Right. Cause I don't even know. I just thought like it's pretty, it's a fun goal to write. Um, so yes. So let's talk about specifically, um, I--and I think what I was going to say is, is the data that we need to be collecting and how we should be focusing on this and, and talk about solutions. But also there is this component where–where it is required. Most districts are billing Medicaid and Medicaid requires data. And so I, a couple of things I think I want to be real clear about is–-is that what is required for us to be collecting? Um, is it quantitative data, every single session? Every single one, every single goal? Or is qualitative data sufficient? When should we be taking which types of data? Um, and again, we need this information for Medicaid, but that's not why we're taking the data. So to me, if you're figuring out a system that works for you and you're taking really great data, period, end all, it doesn't matter where the information is going to go. That's all that matters. And then guess what? You have the data you need for Medicaid, right? So I know everybody hates Medicaid. If I could make it go away, I probably would, you know, I'd be the first one to vote for that to go away. But Medicaid is not the problem here. It's our lack of data. And the fact that systems, lack of systems, for the data that we're collecting and then trying to put it into a Medicaid billing software is a nightmare, right? So we probably won't spend a lot of time talking about it except for maybe a solution of how to make it a little easier of a process. We just want to type, take a bit to talk about taking data because it's what's right for your students and it's what's gonna make your life easier.
Lisa (15:03):
So let's talk about some of the things that didn't make it easy. I know before I ever ventured into digital data, I had paper data sheets. I had big binders that would sit on my lap or sit on the table and I'd organize my paper data sheets by groups. And they would always look really awesome at the start of the year because I had a little extra time before he started seeing students to get everything organized for the year. But, you know, inevitably goals change, kids change and so my data sheets became such crap after a while that, well, and not only–I may not even have the current goals at the top, but then I would even run out of space where even if I had enough room to write 10 weeks worth of notes on one piece of paper, then I not have made it to the copier to get another copy so that I'm writing on the back or I'm writing on the bottom and creating things and writing up the side. So that was problematic.
Sarah (15:55):
Yeah, I do--I know you said this I've always wanted my goals on the data sheet. Did you, you always did put goals on there cause I've seen the sheets without goals, but then I don't know. How do you keep track of everything that everyone's working on?
Lisa (16:05):
That was for me, one of the main purposes was so I could keep track of it. But even back in the day before I did any kind of digital Medicaid billing, we did paper logs for Medicaid. And so that was one of the components you had to have the goals on today, the paper log and have the goals on it. But then again, when the goal is changed, all I would do is either I would forget, like I would literally be working on old goals for a month. And then I'm like, oh yeah, we did an IEP meeting, well, let me go ahead and update that. Or I would remember that they had changed the goals and I would scratch over the old goals with ink and try to hand write in the new goals. So they just looked messy and it overall–
Sarah (16:42):
They never had enough copies, right?
Lisa (16:44):
If a system is not efficient, then you're not efficient. You know, then it becomes more about the process and not the product. And that's really, I think what we're getting with here is that data should be a product. It should be meaningful. It should be giving you information that you need to streamline what you're doing and also make what you're doing with students. The best it can be. But when it becomes all focused on just that process of I have my paper, data sheets, and I'm using whatever I can grab, that's where you start to have problems. And then also that lack of respect of even while you're taking it to begin with, because then it becomes a burden. It's a burden, it becomes a chore. And anything that becomes a chore becomes something that we put on the back burner and don't give any importance.
Sarah (17:24):
Yeah. And not only that, the problem with papers, and I think you said it, it’s about the binders, is that system. I couldn't even ever figure out how to work, make it work for me. Do I have one binder with all data sheets? Are my data sheets individual students? Or are they a group? Is one sheet the whole group? Do I put them in individual files? I had hanging files at one time by group. I mean, even that part changed the organization of it. And then the worst part would be when you have to do an IEP at home or you're on another set on another campus, cause you travel, you never had them with you when you needed them. Right? So there were so many issues with paper in and of itself. I talk about this all the time, but I was the queen of writing whenever I could get my hands on. And so it would, could have been a gum wrapper and I was taking data on it. Um, and so then you lose them and that becomes a whole mess, but more than anything, even if avoid, if you got a work around for all those issues we just talked about and you've got amazing systems and a great data sheet, you still have this issue of you've got your pluses and minuses and some little, maybe some basic notes because a, the boxes are small. If you're going to have more than one session on a piece of paper and two, it's probably chicken scratch to people. It's not going to be like if I handed somebody my data sheet, if they asked for it, I have evidence. I took data. So check I guess, but I don't know how meaningful it was to anyone.
Lisa (18:47):
Well in my chicken scratch–unfortunately, even I can't read after a while. Right? Because we're, I've gotten to the point–I don't know if you're like this too, but my signature is horrible. My handwriting is horrible because it became such a rush. It didn't become about the mechanics of writing and having this beautiful, like printed, you know, handwriting or cursive handwriting even. But then it got to the point of: I'm taking data. I have to write quickly. And so it morphed my whole, you know, if I sit there and think, can I write well and can I write in a beautiful way? Yes. But in my everyday life, I still use a lot of chicken scratch.
Sarah (19:22):
Yeah, I do too. And so that, that's–a huge problem is, you know, the data isn't just for us. I mean, we have to do something. It's not just for us to look at. I mean, we have to do something with it. So when I'm looking at a data sheet, now I've got to tally all of those pluses and minuses. I need to get averages. I need to have enough information to see trends, right? In–in the data that I'm seeing. So I need to do some analysis of that information and having that on a piece of paper makes it really difficult to do that. That's a lot of steps. And so I think that was a huge hurdle to get past, finding a system that gives you information that you need. And we weren't going to build the daily data feature in SLP Toolkit because we knew that 1) everybody's got systems that they use and like, it's a very personal thing, but also 2) we didn't want to do anything redundant where you were going to have to put information in SLP Toolkit and then go put it in another system. Right? And so that was kind of the things that we were trying to work out. And so it took us a while to build it. And I'm really proud of what we came up with. Um, and I think first and foremost, um, and maybe that's, I–I had already seen a shift in the reason I wanted to take better data, but then actually having a system that allowed me to do that was game changing for me because 1) I realized I was–I had room to write really discreet, descriptive notes. I was writing everything from language samples to, um, anecdotal notes about what was working and what wasn't as far as visuals. I was writing information about personal things that they were sharing with me or, or things that could have impacted their session that day. Um, so there was this beautiful, big open note box where I could write all of that. But 2) I started lesson planning. That's a contract. We'll do a whole podcast on that. I mean, I know for me anyway, most of my sessions happen on the fly right in the moment. But the fact that now I've got this system? That was allowing me to look at the data from a session and then go throw something into the next week session. So I'd have a plan. So that changed things. And then too, the graphs. I think the graphs were game changing for me to be able to see that data plotted out on a graph in a way that was meaningful. So I could see if there were ebbs and flows and trends and things like that.
Lisa (21:28):
I love -- even the big aha for me too -- like you had mentioned, is that room to write. Because a lot of times we think things, but then we either just don't jot it down on paper. Or we were thinking like in the moment, Oh, we'll remember this next time. You don't remember it so much time has passed between when you see a kid. I think of two groups that come in after a group two groups ago? I don't remember what happened. So to be able to do things in the moment and then having all of that space to record, it kind of makes me think of like a fish in a fishbowl analogy. You put it in a little tiny tank, then the fish stays really small, but once you open it up to a bigger size, the fish grows to match the size. And I experienced that with my data. You experienced that with your data, that once I had this easy way to collect it in tons of room to write notes, I did use it. And it doesn't mean that I was writing paragraphs. It just means that I was writing meaningful information. I wasn't restricting myself to this little teeny box that I could only write a couple of words and take pluses and minuses.
Sarah (22:28):
Exactly. I remember some really cool Ahas early on was when I was working in the preschool last year, um, which–taking data on a device I hesitated to do, because I thought, how in the world do you do this in a preschool setting? When you have 15 students that are chasing and I did it on my iPad and it worked great. And that hurdle I got over real quickly, but I remember writing things down and like feeling so happy about it. It was really weird, but it was like for the first time ever, I noticed that when this one student worked with another student, he was more willing to–to participate in the activity at very limited language. But I saw more engagement with one student over another and the activity and interest increased with, with this one student, he modeled this one student, I wrote all that down, but that's data. And duh, like, because I've got such specific things, guess what I did next week, I pulled that same student that this kid was really successful with. And so it's that kind of a thing that I never took time to write, reflect and reflect on things that were driving therapy, because that's what the data is about.
Lisa (23:33):
Nobody requires you to reflect, they require you to collect, but not reflect. So that's kind of on you to do that. And then I think that part about lesson planning too, when you do it in the moment. So with that student, you might in the moment be like, oh, I think, you know, Jose's working really well with Michael today, but if you don't write it down, then that information gets lost. And so I even think of it, you might write it down in that session, but that would be part of my lesson plan for the next–
Sarah (23:59):
I took that information and posted it into next week.
Lisa (24:02):
Exactly. Because that's the kind of stuff. Again, you–you think you're going to remember and you don't, or what happens is you start your next week session. If you haven't written it down and it takes you about 15 minutes into your 30 minute session to be like, Oh yeah, I was going to have Jose work with Michael today. And I totally forgot.
Sarah (24:20):
That's exactly right.
Lisa (24:21):
So that's the kind of stuff that we talk about, that it doesn't take extra time. It actually saves you time. And it all goes towards that, you know, idea of what's best for the student, what helps them move through and, and acquire their goals more quickly. And just, it just makes so much sense. But until you kind of go through that and, and flounder a little bit, that's where I think you're like, okay, I see now where this really does help me, where before what I was doing was not helping there wasn't one.
Sarah (24:51):
No, it wasn't helping anybody. So I do–I think I loved that. Part of it is really seeing how data could drive my instruction and shows evidence of treatment efficacy, um, was really kind of a light bulb moment for me. Um, and I don't write really long lesson plans, but again, it's such a relief to walk into a session the following week and pick up right where I had left off.
Lisa (25:17):
And not have to recreate what the heck was it for every group. That's what I'm saying is like, you think you're going to remember, but you don't.
Sarah (25:23):
Right? No, it really was. The other thing I was gonna say is, um, I think the reason I miss goals occasionally, um, only like once or twice for progress reports is because on the data sheets, it was really hard to see which goals I had targeted and the data that I had on them, you know? Unless you had a really great system where you could see a big blink, right. Somehow you know and so this system is awesome because I just click on that little graph and it will show me the last data points that were taken on that goal. So I can see that oh shoot, I haven't taken data on that goal in two weeks.
Lisa (25:54):
Or two months.
Sarah (25:55):
Or two months! Ha, definitely not. And so those kinds of things are really game changing in the fact of, again, collecting the information that we need so that we have sufficient data, um, to, to feel confident. But there was one thing I was just thinking of, oh, what should we be writing down? So I feel like we just went over to the system and I know we feel really happy about the one that we're using, but what should I be recording? So we talked a little bit about that qualitative data, but don't, I–to me that's sometimes more critical even than that quantitative piece.
Lisa (26:25):
I think maybe a combo of both? And it depends on what you're looking for. So most SLPs are really comfortable taking the quantitative piece because we do that and we know that if we take qualitative data, it tends to be like a quick little blurb on something like "used cues" or, you know, there might be a couple of words, but there's not really much else. So when you're thinking of qualitative data, it's that kind of data that is not concrete, that maybe what you observe is important. If you were looking at the same session and, um, taking data on a goal, that's quantitative, we'll probably get similar scores, but for the qualitative piece, it's all of the extra stuff that maybe you pick up through your clinical lens, and then mine could look different than that because it's what I pick up through my clinical lens. So it is important information, especially when you're working with kids, because as you mentioned, we're recording something like: two students working together improves overall outcomes. That's qualitative data and that's important to drive future instruction. Or it might be if I'm taking data on something like cues, I want to know specifically what cues worked, but then also what didn't work, because then that next session, again, I'm using this data to drive my instruction.
Sarah (27:38):
Not using that method that didn't work.
Lisa (27:40):
If I know a model works and a visual cue does not work, then what I want to note is that moment, reflect on that, make sure it's part of my ongoing lesson plan next week use models, because that's part of that spinning your wheels thing, that if you get going with the next session and you just kind of go based on what you do for everyone, I use a multitude of cues for everyone and I support nice scaffold. But if I know that visual cues are not working for this student, then why would I offer them again the next week? Why don't I figure out what is working and do that each time? And then I have all of those cues in my kind of toolbox of things that I use with students. But if this student works well with this, that's how we're individualizing our treatment. And that's how what we're doing is specially designed instruction and makes it different than what you do for every other student.
Sarah (28:31):
Yeah, and I have heard, and I actually think it could have been the case in our district too, that, um, we have to have quantitative measurements. We have to have that number, that percentage on the data. And I will be the first to argue that that's not always necessary for every single session for every single goal, because there are going to be sessions where I am doing instruction. And so is it helpful for me to say we used, um, you know, X book or game and, um, the players took turns. No, that's not data. I mean, I want to know the activity, but I'm saying that's not going to drive my instruction. But if I were to say, we did this activity, here's how it went. We focused on teaching green group on the EET and they really struggled with this type of categorization, but they were able to do it with blah, blah, blah, blah, blah. That's data.
Lisa (29:22):
I think if you go back to like, even thinking from a district perspective, if you don't give sort of overall directives, then it can become sort of like a Wild Wild West situation. Where if you tell people that they don't have to take quantitative data, then they are just writing, you know, the book name.
Sarah (29:39):
Yep.
Lisa (29:39):
And that is the only data that have. So, and if you think of those paper sheets, most people aren't reflecting to the extent that you just gave the example. So there has to be some sort of criteria. I think that districts give for a one size fits all kind of response. But on the flip side, as a practicing therapist, if I do have data like you outlined, that would be where I would want to start as far as if I got questioned, why did you take quantitative data? That would be my response. I have all of this other information that is important, right? But if all of my data sheets say, book, name, book, name, game, whatever then that's where I think that those kinds of policies–they get implemented for a reason. I mean, I even think of when you go through the McDonald's drive through and order a cup of coffee, and it says, the contents of this cup are hot and may burn you really? Cause that's usually how coffee comes, but they implement kind of overall rules for a reason. There was–there was a purpose for that. I would just say whenever, you know, if you ever are in a kind of a district that has those sort of restrictions and you get questioned on it, just make sure that you have your argument outlined as far as this is why I collected this, the thought behind it.
Sarah (30:52):
And like you said, the big red flag is going to be every single week. All I have is the name of the book. Right? So that's huge. And so that's what I was trying to say is there needs to be some balance, right? It is okay to do a session where it is only instruction, just make sure that you are writing down information that's driving instruction. And so then maybe the next week though, for that goal–because we just talked, we just did do some instruction in the last session, green groups. So maybe I get some–I want to take them to annotate a date on that one. So to have some balance there, the other big thing for me is no, I don't target every single goal, every single session, impossible. Sometimes I can get more than one goal, sometimes I can't, but I do try to take that quantitative data at the very beginning of the session and have it just, I look and see which one I have not taken data on. Or like I said, I maybe want to take data on the one for what we worked on last week, take the, the very beginning of the session. And then I don't have to stress about it, the rest of the session. So try it. It's about balance. And again, it's about what we need to, to try. Again, I feel like I've just said driven into driving instruction 17 times during this podcast, but that's what it's coming down to.
Lisa (31:57):
Well, and I think it comes down to systems and what makes sense for what you're doing, because I know in the past, even with paper data, I had more of that format of just taking it at the beginning. But I found last year when I was using our digital data, I was actually taking data the whole session because I was working with older students and we incorporated the idea of data collection into, like, kind of their meta awareness of why they're coming to speech. So we used it to set goals for the session. They would eve– I had, um, one group of Artic students that I would mark like the–I would take live data with the calculator. And if I marked the minus sign, one of the kids was like, wait, wait, wait, why did you mark you that? And I'm like, that was on you, dude. Like you didn't say the sound. So if you want more pluses… So we used it actually as part of their instruction. And then they became really competitive with each other. Even I use–I would hear them echoing back some of the words that I would use–that if something–if I did notice and I would have have that aha moment in the session of this was really successful, we don't want to forget that. I would just say that was super awesome, let's write that down so we don't forget it for next time. So even when I left that campus, it was so cute because one of the students that I was working with was like, now is this new speech person–is she going to have all of your notes? Because I want to make sure that, you know, she got everything that we wrote down so she doesn't forget to use those. And it was just like, data can be super powerful. And we, I think we forget that our students are surrounded by that in their classrooms. There are charts, there are goals, data, right? So if they can come out of your room.
Sarah (33:32):
And they have students taking their own data,
Lisa (33:34):
Right and If they can come out of your speech room and have more information, then I worked on a game because we did use games and we used apps and we use other things as our kind of generic reinforcers, but their focus was not on the game. That was just like an okay, that's cool. But it was, they were actually getting that reinforcement intrinsically by working with their data. It was real weird. It was a very new concept that they actually knew what they were working on. Right? It was just super cool because I think, again, we think, we assume as adults sometimes like that this will be distracting or this won't work or this whatever. And you do, when you try some different things and then you do kind of figure out this system works for it. Maybe even within your caseload that might work for some groups and not for others. I might have to do a mixture of both. If I have students that get really distracted by data collection, I might just do it at the beginning of the session. And then right as they're leaving. So you just need to be flexible.
Sarah (34:26):
Flexible, being flexible is key for sure. Okay. So I feel like we've talked about the kind of data and how to take it in a system for it. Um, this is pretty much this, this part has been about for treatment efficacy purposes and lesson planning and, and thinking about where you want to go in each session. Um, but then we've got this big moment that happens once a year, sometimes more where we have to write present levels for an IEP. And so even if now we're taking this killer data, right, we've got all this information. Certainly I should be able to write a present levels based on all the data I've taken.
Lisa (35:03):
Well, one of my favorite expressions is you can't see the forest for the trees. So I think sometimes if we're so hyper-focused on the IEP goals from the previous IEP, we forget to zoom out and kind of see the big picture on a kid. Right? And so that's where I used to struggle, not so much for those concrete sort of skills that were developed in a hierarchy like articulation. If I knew that we were at the phrase level, then I would think, okay, in a year I could probably get them up to reading or structure conversation or something that was real easy to do sort of in the moment. And I wouldn't to zoom out, but language, I struggled each and every time for my older kids, when I had developmental norms that I was working with where I'm like, okay, I know they're supposed to be doing X, Y, and Z, but then after about first grade, I struggled with figuring out what I want to treat next.
Sarah (35:54):
Can I tell you how relieved I was to hear that? When we talked about this initially I thought I literally was the only person on the planet who didn't know how to pick goals for my students. I remember talking to another SLP that I was brave enough to ask because she was a good friend of mine and I had said to her, but how do you know what they should be doing? Like, where are you getting this information from? And she said, well, I use developmental milestones as far as I can and then I, you know, try to compile some information from some other screeners. And she had shared some things with me. Um, and I–I thought, okay. Because I never knew, like, if I've got a third grade language student and I'm writing his present levels, what is it I should be looking for? If I have a student in a self-contained program with complex communication needs, what should I be looking for? I literally thought I was the only person on the planet who struggled with that. And so it's been great talking to other people because I realized, how on earth could we know all of that? We work with way too many diverse needs to be able to think of every single thing we should consider when we're writing a student's present levels and determining targets for treatment.
Lisa (36:55):
So I know sometimes I would, you know, everybody inherits stuff, right. I'm sure you inherited a ton of things from your–throughout your career. Yeah. So one of the things I inherited early on was a screener. And I remember thinking, well, you know, this is a K6 screener, and it's the same items for every student that you administer it to. No matter if they're in kindergarten and sixth grade. And that sat funny with me because I'm like, well, clearly there's a little bit of a difference between a sixth grader and a kindergartner, but when you don't have anything else, I still used it. I still tried to figure out if it could give me information
Sarah (37:29):
For our system. Yes. And that's what it all comes down to. Right? Something–a way to guide your thinking, a concrete system like we do for the evaluation process, you know, what tests you're going to administer when you do an evaluation. And so that's exactly why we had created those present level assessments, we knew that we needed a system, um, and a way to be able to quickly assess a student and gather information on strengths and needs. And so we've had, we just had that conversation with Doug about AAC. Um, and there was another topic I thought of it where he had said, there's, you can't really have an assessment for AAC because it's so variable. Praxia one, Praxia two, you know, there's not an assessment out there, cause there's no way I don't necessarily need this norm referenced assessment. I need a guide. I need direction. I need things to consider that I might otherwise not know to be considering.
Lisa (38:21):
And have it in a systematic way. So every time I go to work with a student, there might always be variables within that. But if I have this set procedure down of these are the things I'm looking for, it helps to guide your thinking. Like you said, there's no--no worse feeling than, uh, you know, floundering. And I think of like, even the feeling that people get, I don't have it anymore because I use our present level assessments. But when you'd have that transfer kid that would come in and you're like, oh crap, I need to test the student and I want to put the fifth evaluation because just because they transferred to your caseload and had an IEP that was due, like, you know, the next week or two weeks ago,
Sarah (38:59):
I'm supposed to collect that data quickly.
Lisa (39:00):
Right. Right. And so before, I think, that's, that was the first thought is: a lot of people would say, well, clearly we need to do an evaluation. I don't have the data I need. So I need to give some standardized tests and pull the kid in to really get that information. And then along the way, um, that was one of the things I feel like I changed with too, where I'm like, but that's not what standardized tests are designed for. And so we, and I think you feel that when you inherit an IEP, that has goals that are directly from standardized tests and subtests. Yes. And I'm always like, well, what the heck is that? How a–how do you even work on that? Why is that meaningful for the student to work on that? And that's where you come to the sea of like, oh, well maybe it's not. Because that was really just designed to see how students of a certain age perform on that specific task. And if students perform poorly, it may be indicative that they have a language impairment but, really, the things that give you the most bang for your buck during evaluation is not even that. To me, it's the classroom observations, it's the language sampling. And so whether you're having them in a regular conversation or for the younger kids, I do a lot of narrative retell. For older kids, I'll do an expository task. I like to even do picture description scenes. I think that gives me a lot of information about the language that they're using and the breakdowns that they're having. But then that becomes, I think, when we're thinking of writing IEPs, really tricky because when we say the word language sample, the first thoughts that come to our mind are screw you, I'm not doing SALT, I'm not doing a language sample. And so even, um, that was one of the things when I was visiting with my CF earlier this week, I said, you know, you need this information, but it's a little different, your process might be more in depth and thorough when you're doing an evaluation versus an IEP. You do have the time to just talk to a kid.
Sarah (40:52):
It's so much information. First, I mean, besides the fact that it's such an authentic way to gather information, you can, in a conversation sample, you can hear articulation, grammar, vocabulary, watch for pragmatic organization. Yeah. So there's so much you get from it. But I think we get so hung up on the fact that I have to transcribe every single word he says and analyze it with a tool like SALT. Ain't nobody got time for them. No, they don't. So no I'm going to do this conversation sample. And then I've got a guide that's helping me think of what should I be listening for and in a way to take notes, to gather that information. So, um, the present level assessments that we built in SLP toolkit, that is how I look at them. They're a great way to, and scaffold your thinking. They do not replace your clinical judgment. They're not normal. They're not referenced. They just help you gather data to be able to determine goals. And so we called them present level assessments because the idea was to get, you know, have some kind of an assessment tool to give you information for the present level section of an IEP. And I was talking to somebody once and they said, why did you, you know, that, that takes some explanation, maybe that title's kind of confusing on what these are designed to do. Why didn't you just call them goal selectors? And I was like, ah, dang, that is so true. I think I got too fancy on that one. We are super fans, but that's what they're–that's what they really allow you to do is to select meaningful goals individualized to that student's needs. I don't have to write synonyms for every single student.
Lisa (42:19):
Well, and you know what I think they free you up from too is–and I know, I think you've had some CFs that you worked with last year that had this–this panic moment of, of either not having the present levels or using the present levels from last year to try to create a current present levels. And we've all done this, right? where you look at the IEP from last year and you try to tweak a couple of things, but that's not really accurate. I mean, it needs to be in this moment that might've been accurate in that moment in time, but we really need information for right now. And so I think the reason that gets overlooked is because we don't have a great system for getting that information for right now. So we'll just kind of use the old information, add a couple other things that we know about the student and hope that that's okay. And that I think that whole concept too comes into play. When we think about even the respect, I now have for an IEP document, as far as it being, um, a story for that student. And so that was one of the things we talked–I talked with my CF about also, because she was working with a kid that it took them about a month, month and a half to figure out how this kid worked. There were a lot of behavior issues and things going on. And I said, well, how great would it have been if you wouldn't have been able to read the present levels and have some of these strategies of things that specifically work for this child versus you spinning your wheels for six weeks having to figure that out as a team. So do your next team a solid, because it may be you, but what if that kid moves again? Make sure that you write down these things that work for the student. Cause that's what it's meant to be as a story for that student and that's, and it's meant to be sort of a paper advocate for that student of all the things he or she needs in order to access curriculum, just like gen ed peers, gen ed peers don't need the same things. That's why they don't qualify for special education, but this kid needs these thing.s
Sarah (44:12):
Right? So we need to be really thorough in our documentation and have good data to be able to do that
Lisa (44:18):
And not just make it a checklist of, sweet, I wrote the IEP because I had to. And so it's that whole thing I know that's kind of been our theme for today of just that mind shift change in how we look at the data we're collecting and how we're housing it. So I, you know, it's that IEP is meaningful.
Sarah (44:36):
And that having this, um, this mind shift in data and IEPs are what's best for our students. And if we do a really great job at both of those, then we're going to feel more confident, right? It's that mind shift set? And I always said–I almost said mind shit again. I don't know. I think that's a whole other story, but anyway, um, I was thinking too, I love our I heart IEP t-shirt it gets a lot of, um–
Lisa (45:01):
Physical reactions, and polarizing opinions on it?
Sarah (45:04):
But the joke is that we're trying to save the poor IEP’s reputation, one IEP at a time. We know the paperwork's a nightmare and there's all of these I's to dot and T's to cross. And it's a very overwhelming process. Um, but again, if you realize what exactly it's doing for students, um, then maybe, you know, it won't be–seem like such a burden.
Lisa (45:25):
And indirectly, it's what it does for you too, because I think, you know, it is, it's an amazing tool for students and it is important, but then also that confidence piece that we referenced earlier. When you have great data, you have great peace of mind because I don't care who asked me what, I will have some information to reference what my decisions were made on, versus if I'm just rehashing information that I inherited from a year ago, I have no clue where they got that. I don't know why they picked synonym speaking, and I don't want to just say well, just because he worked on it last year and we're kind of connecting into that. Like we really–[unintelligible overlap].
Sarah (46:03):
That's my, here's the deal. He might need to work on synonyms, but isn't the most meaningful thing that he needs right now to access curriculum. And that's, our job is to determine, based on the data, what does this child need to be successful? Um, and so–
Lisa (46:19):
And our data, you kind of, you have to have that data too, because I think of the conversations that come up in meetings, whether it's an IEP meeting or even a Met meeting, if all you've done is standardized assessment for evaluation and you don't have any other data, it becomes really hard to filter the other team members’ information. So even think of like when you have, um–something pretty common that comes up as like the student's having difficulty following directions. Well, if you don't have data to know that, you know, maybe it's attention related, maybe it's memory related, maybe it's vocabulary related–
Sarah (46:48):
He may need to have temporal concepts.
Lisa (46:50):
Exactly. And so that helps you filter where the immediate response to things like that as oh, we need a goal for falling attention. I mean, falling attention, following directions, but maybe, if it is more attention related, that if you just shorten the steps or if you get the student's attention before. Those are accommodations, those are not, that's not a goal because if you bring a kid in, I mean, think of, and I know everyone has done this, you inherit a goal like, or are kind of coerced into writing a goal for following directions. You bring the kid into the speech room, you start working on following directions and they do it perfectly. And so you have to think, well, why is that? They've got the concept. So am I going to work on this for a year? And if I don't have data that shows otherwise, that's where I think we kind of get cornered into some of those sorts of adding things. You know, you feel like, okay, well I'll be a team player. And just add that. But if I had the data, I think you're being a better team player to say, well, this is what I show. I don't think it's related to this because we have evidence that he can follow temporal concepts in this. And so what we might want to try instead is chunking up the information or gaining his attention, or, you know, other strategies give a visual reminder, whatever the case is, right? Those are accommodations out of goal. I don't want to spend my–I want to work with the student on things that are meaningful, that he can't do with just an accommodation. Things that require instruction that I can support and, and do what's best for them.
Sarah (48:17):
And you wouldn't make that determination without data. Right. So, okay. So I know, I feel like this is a long one, but there's so much to go over because we just, we talked–started by talking about, we need data for treatment efficacy. Then we just went into, we need data for current levels of performance. And then the last piece is going to be, we need data to determine outcome on goals. So that's where progress monitoring is the greatest invention on the planet. Teachers do this all the time. I've always been envious of a teacher who teaches a skill. And then at week's end gives a test to determine whether or not the student has actually learned what was taught in class. And I thought, you know, I know where what we're doing is different in a way, there's not a set curriculum that I can grab these assessment pieces, you know, to look and see whether or not they're learning that part of the curriculum. But when we first–when I first started working in Mesa with, um, Lisa, they had what they called criteria reference tests in the area of articulation. And so it was a binder that had all of the speech sounds and the initial medial and final positions of a word. And if the student was working on one of those sounds, you could give this test and then based on how they did that could be what you reported on your progress report. And so in talking with Lisa, she had developed some criterion reference tests and other areas outside of articulation and things like grammar and vocabulary. Um, and so the idea of how teachers progress monitor really was the founding reason we created SLP Toolkit. We wanted assessments to be able to do that same thing for us.
Lisa (49:45):
Well, and wanting a system. So even if I go back to that, that was the first time I was introduced to criterion reference test as well, was that articulation binder with those in there. And I liked the idea of having a system for this because before I didn't, it's like you said, you know, you, you would use your data half the time. I didn't average out the data half the time. I would figure out that I didn't have the data that I needed to actually report how the goal was written. And so it was just like it, I don't like feeling unsettled. I like to create systems for many things in my life, because it gives me a go-to of knowing what to do. And of course there will always be variables and I might, um, have to change it up for some of the times, but when I first kind of wrapped my head around this would be awesome to have in language areas too, that's when I started to develop those. And even I would inherit IEPs sometimes that would have mega goals where it was like, um, student will produce S, F, T, H, and R. And so I would create a test that had words with all of those sounds in them, because then I didn't feel like I had to progress monitor for eight weeks to get all of that information. I could just create this quick test on the fly and match it to the student's goal. So if criterion reference tests are a new concept to anyone who's listening right now, kind of the idea of them for all four of you actually exist, four people–actually, somebody emailed, I think we're up to five. Okay. So, um, but actually the idea of a criterion reference tests is it's not norm reference. It's not like the standardized test you give. I think people get tripped up with that word test. So they get nervous about even writing in and, and a goal that, you know, I need permission. It's just more data. So, um, you develop a list of stimulus items, put it, we use 10, we like the number 10 is a nice round number. It gives you enough information, but you test the student against themselves. So you give the test to get a baseline on your IEP. And then you pull out that same test to readminister every grading period and see how the student performs on that test over time. So if it's something like grammar and maybe you're testing plural Ss, then you would have that list, your therapy. You're not teaching items on this test. Your therapy is just teaching plural S as in general. And then you want to see if at the time of grading periods, they can generalize those strategies that they're learning in your therapy to this test. So it becomes super freeing because then when I know I need to write my progress reports, I don't have to comb through all of my data in files. I just pull out that test and give it to the student and boom, have my score done.
Sarah (52:20):
Done. It is game-changing. I typically progress monitor about one to two weeks before progress reports are due. I use the session time to progress monitor. So I find an activity that the students can be working on. I pull one student over, I give them the tests, I finish with the group, they leave, done before they have walked out the door. I have my IEP software open in another window, and I literally just record however they did on each of those tests. And I'm done. So I have not completed progress reports after hours in two years now, two and a half years.
Lisa (52:51):
So, and that's what I think was cool about when we developed it in this kind of digital platform. I had created some that were in a paper format, but the problem with that is I literally–I had the Manila folders in a filing cabinet for all of my students. I'd have to pull them out. I, even if I had, you know, hopefully I would have that test in the Manila folder, but then I would have to give the student the test, all of those folders would then go on my back counter because that meant I had to score them. And then once they were scored, they'd go on my desk because that means I have to input them into my IEP software for the progress report. So that whole process, when you have 60 students, took a ton of time. It gave me a system, but it took a lot of time when, once we moved over to this kind of digital format and created a very robust set of comprehensive tests, literally like you said, I could administer the test. It scores it for me. And as they're walking out, I just report that score, report it done.
Sarah (53:51):
Right? So not only huge time saver, I think it's–it definitely improves. I mean, we talked to people who took two days to do progress reports, who can get progress reports done in a couple of hours. So a huge time saver, but also really great, consistent data. I feel confident that what I'm reporting on is accurate to how that student is performing. And so, you know, it really has changed the way I have, um, done my progress reports in–in so many ways. But I think I love that I don't hesitate to send home a progress report anymore. Right? I feel good about it. If the parents want to call me and ask me about it, I can explain it well.
Lisa (54:31):
And do you send home the graph too? I love the graphs.
Sarah (54:34):
Yeah. I'm not going to lie. There were times where, Oh man, I would report progress and it was kind of this like, like throwing darts, so embarrassing. Oh my gosh. I'm so embarrassed.
Lisa (54:46):
I think that's a 50, right?
Sarah (54:48):
I think he's about here based on what I did collect and what I think I know, you know, and we're pretty good guessers. I would have to say, I don't think I was always far off. It's only–it was a stretch, but smart guesses. But those don't get you far in life. We need concrete evidence that's supporting these things because, again, I'm not doing anyone any favors by making up data.
Lisa (55:08):
Yeah. Well, and guesses are subjective. So your guess is–it might be different than my guesses. So if I inherit your kid and you have 70% and I have 10%, that's it, you know, it's a problem, right?
Sarah (55:19):
So I know it's, again, a reoccurring theme, but I hope if anything, we have convinced you that these are not hoops we jumped through because somebody says we have to do these things. And I know it is scary because we are living in a litigious time and parents are very savvy, they know their rights
Lisa (55:35):
And they want special education.
Sarah (55:38):
I hear advocates coming into meetings more often now than ever before. And so we're not saying do all of this so that you can feel comfortable in a meeting with advocates. We really hope that you're realizing it's what's best for you and your students. And it's going to really dramatically change your therapy in the school setting.
Lisa (55:57):
Data is your homie people.
Sarah (56:01):
We could talk about this forever and ever, but I hope the big takeaways are that finding systems that work for you for taking session data is critically important. The types of data we should be taking during a session, finding a systematic way to collect information you need for present levels, as well as an efficient and consistent method for progress monitoring. I love it. Did I cover it all.
Lisa (56:23):
I think so.
Sarah (56:24):
Boom. There you go. Mic drop.
Lisa (56:26):
We out, everyone have a good one you guys.