validity-McNamara & Nelson-Barber

Validity: testing the test\McNamara Ch. 5
The topic of validity at first brings to mind my logics and ethics class I took as an undergraduate. The terminology seemed confusing and I felt like I was going to fail the class. I remembered that there are degrees of validity based on the logic and that’s what threw mw off. Now, in this reading- the subject is easier to understand in this context of assessment. There are definitely a lot of things to consider and I wonder how a teacher can remember all of it when creating assessments for their class. Of course having a critical pedagogy most likely would help.
The example given of content validity being that tests don’t capture the whole person was delight to see in text, because it is a fact that there are people do not do well with paper-pencil tests but have the oral fluency in the language. This is definitely the case out here in our area. I believe this is one of the reasons why many of our students who graduate just stay home, because the tests they’ve taken label them negatively, to where they do not have the confidence to go out into the world on their own. So what could work then for our students here is what I would like to be on a quest for.
It was surprising to see that, “the more complex the content of performance, the more there is to jeopardize the validity of the ratings (52).”
Speculation and empiricism were a surprise to brought up finally in the conclusion of the chapter. Speculation- “reasoning and logical analysis about the nature of languages and language use, and of the nature of performance,” (as outlined in chapter 2.?hmmmm.) “Empiricism means subjectinbg such theorizing and specific implications of particular twesting practices to examination in the light of data from test trials and operational test administrations (54).”


Nelson-Barber, S. & Trumbull, E. (2007). Making assessment practices valid for indigenous American students. Journal of American Indian Education (3, 4), 132-147.
This article reminded me of the fact that there is a lot of research done on indigenous people, but nothing is done about it. In fact sometimes I think they do it just to degrade us and highlight the negatives instead of the positives. And it seems like they don't get real informed consents like the IRB has us doing for our research. Anyways, this article luckily came from an insider (technical name temporarily forgotten at this time) who proved to be well informed and has the ability to reinforce what the indigenous people's awareness of these standardized testing situations effects on the children and their communities to the public in this medium. I wonder if the author read the Maori articles?
I liked that they shared a school’s assessment procedures of using two scoring guides- one for the funds of knowledge and one for the academic knowledge. I definitely would love to have time to create those kind, especially now, since I’m doing my research on this topic, and I am supposed to be doing action research to compare the effects but this way sounds better- to incorporate both kinds of knowledge into the assessment.! Why didn’t I think of that????
Oh- when I read about the executive orders I got so frusterated and blamed the state of Alaska. When I was living on a reservation in Washington state, I learned that the tribe didn’t have to answer to the state, only to the feds, but that the state would find ways to impede or interfere with the tribes progress toward self-sufficiency- so this reminds me of the word hegemony. I think or wouldn’t be surprised if Alaska does the same thing as Washington state did and does when it comes to indigenous rights.
Besides asking if the test developers know who they’re dealing with, the other question is whether or not the state education departments really know and understand what the tests are about that they are requiring our students to take- besides it’s economic cost, let alone it’s emotional costs. Here I am concerned about others when I should be relating this stuff to me.... and I will need to work on that.

Solano- Flores, G. (2008). Who is given tests in what language by whom, when, and where? The need for probabilistic views of language in the testing o

I found this article to have a lot of new vocabulary amongst a couple of surprises...
First in the abstract were the words: testing of ELLs is a random process, and deterministic views.... that caught my attention. Then the phrases/statements that caught my attention were:
*erroneous assumptions about the capacity of assessment systems to serve ELL's
*affects of the dependability of academic achievement measures
The new vocabulary that I appreciated is: mapping sentence. A descriptors tool- an awesome tool provided. It looks like a rubric and I see that it is referred to as a guide. It looks very useful.
The six components (who, tests, language, by whom, when, and where) of the process of ELL testing were analyzed for their affects, which I found beneficial and helpful. I just ended up feeling so sold by this article.... I was wondering to my self, where was the author all these years? And will something be done now that this is out? Will the government take this information into consideration or action?
I want to remember that ELL's are from the WHO section: rarely tested comprehensively enough; neglected as a source of information .... because of basing evaluation on L2 development!, and inaccurately classified because of assessment of their linguistic proficiencies. From the TESTS section: Underrepresented in pilot tests and that accomodations may not be appropriate or beneficial at all.
From the LANGUAGE section, I was totally surprised, almost blown away to see that word dialect and the serious issues involved with it. The topic of translations and translator technicalities gave me something to think about. In the By WHOM section, I thought it was a very good question to ask, (and wondered why I didn’t think of that before) when the author asked, “What are the developers qualifications?”
I was thankful for the reminder of the bilingual and foreign language teachers’ job differences. The question about whether or not the author of the tests are familiar with the target population's communicative style? I appreciated that the author said, “the process of ELL testing suggests that the ability of assessment systems to communicate with ELLs needs to be examined.” I think that the word communicate in this sentence the key to test development.
In the WHEN section, the information about the instability of performance across testing occasions for ELLs, was a concern even ten years ago I remember.
In the WHERE section, linguistic simplification, was good information to think about as well as the information on test localization, both of which I assumed would not be factors. In the part where the author said, “Improved approaches to ELL testing should be based on research that examines the effect of context on the effectiveness of testing strategies for ELLs.” I realized that I think this is where I find myself stuck about assessing students on things that require real-life performance when we’ve only been practicing amongst ourselves.
The information under the heading: Probabilistic views of language on the process of ELL testing, was enlightening. The whole paragraph that begins, “In an ideal world.....,” I want in my thesis as a quote.
I see that there’s a new buzz word: fidelity. It was brought up at our in-service.
It was a relief to see and nice to finally know there's a way to deal with this issue when the author shared that effective research approaches can be devised through the G theory! Thank God- so why isn't it in use???
For the Conclusion section, it was very smart of the author to remind me of the fact that "Valid testing for ELLs cannot be achieved of we focus solely on the proficiency of the ELLs in English but fail to examine linguistic factors involved in the development, adaptation, administration, and scoring of tests."
All in all, I'm sold on the G theory even though I don't understand how it works, I see what it does.

Shohamy 10/7

Shohamy, E. & Inbar, O. The language assessment process: A “multiplism” perspective.
Tel Aviv University.

This article made me wish we had this class along side our research methods class the first summer… but that’s ok, better late than never… or not???. Anyways- I like this article for it’s fluent flow and especially that the authors provided some of the definitions, from their origins which I thought was nice (and smart because this almost felt like a review and almost boring). The authors also provided the definitions according to the original creators which was beneficial, they helped put the assessment/teaching/politics indefinate puzzle/picture together.
Obviously I got hung up on definitions and couldn’t wait to read up more on what the “tools” were…. It seemed like that was a mysterious word, and I so wanted to see what its definition would be in this article…. And I missed it…. For a little while… so I was hooked… on “tools” which seemed to be used interchangeably with instruments. Anyways…
After reading up to half way, I began to wish I created a flow chart, then began wondering if these authors did, then I felt like they should have and provided that instead along with their little text boxes.
The three questions suggested to consider before designing the instrument (tool?) was at first glance appearing easy enough to answer and thus was the case with the first question-easy to answer about what is the purpose(s) of the assessment procedure?- of course these were already thought of for us to be in our 6th- 7th year of standards based design… levels. The second question: How is the language knowledge to be assessed defined?- in which the term valid arrives in the explanation…. Then the 3rd question was: What instruments or assessment procedures will be chosen to elicit the required language knowledge? I like this question because I’ve wanted to ask something like this for a while
The section on the teaching approaches, language knowledge and testing methods was finally able to provide me with what they meant by tools, which was a relief to see, because we do require these in our district… just when I was beginning to wonder if our administrative people really knew what they were getting themselves into, because I’ve never heard them refer to theories or methods or anything “professional” like that at our inservices- so when I go to them and then read this stuff, I see parts of us in this whole SLATE program as far as the classes… just that I wonder who else knows this stuff in this district that we can begin to collaborate with using this information we’re gaining here to really have these authentic assessments that are genuine and effective enough for the state to accept for those needing alternative assessments so as not to be left behind/ out of the job market.
I have seemed to use the words assessment s and tests interchangeable but now, I am more aware of their specific differences, to where I feel confident I can use them appropriately and proficiently in context.
Oh- there’s more but this will do for now.

9/30 rdg responses-NCLB

McNamara Ch. 2 Communication and the design of language tests

I found myself skimming through the history section of testing when I realized that they seemed to coincide with the reading readiness and emergent literacy periods. It seems like they also coincide with the SLA theories of the time also (I think).
The author seemed to imply the politics of it all, which I naturally read deeper into and allowed myself to get frustrated. Still, I liked that McNamara said that the complexities of communication competence and performance measurements cause difficulties for standardization.
I took a great interest in the section, “Models of communicative ability”, particularly the strategic competence information.
The information provided on page 19 about strategic competence and the test designs and complexities were of great interest to me. I found myself agreeing with a lot of the statements he mentioned because I realized that I’ve wondered about some of those things before (which made me feel ok about having those thoughts, and glad that they were stated more clearly than I could have ever done.) For instance,
I’m aware of the factors, and am having a hard time thinking of authentic assessments to create, but I don’t feel that I’m reluctant like the author states the assessment field to be.
The author said it best, that it definitely is an “eclectic scene in language testing (p. 21- conclusion).” (I’m biased because eclectic is how my classroom syllabuses are.)


Abedi, J. (2004). The No Child Left Behind Act and English language learners:
Assessment and accountability issues. Educational Researcher, 33, 4-14.

This article presents very important information about the annual yearly progress (AYP) reporting issues. The No Child Left Behind Act (NCLBA) has four subgroups of children not to leave behind. These subgroups are based on a student’s financial standing, major racial & ethnic groups, disabled, or/and Limited English Proficiency status. The requirement to report the status of these four subgroups presents serious issues.
Of these four subgroups, the author focuses on the complicated and serious issues involving the LEP students. The author pointed out that many LEP students also fit in to one of the other subgroup categories. I suppose the not so many group would consist of stereotyped groups of people/students like say, for instance, the immigrants, like maybe Russians. Where they would fit in? What happens to them? Are they considered a part of the unstable group? I’m assuming that they’re not poor, disabled, but I guess they’re from a major racial & ethnic group. Anyways,…that was my little brainstorm. I now share the author’s identified four critical issues concerning the LEP students within states, counties, school districts, schools, classrooms, and they are:
1. Classification inconsistency
2. Sparse LEP population
3. lack of LEP subgroup stability
4. Measurement quality of AYP instruments for LEP students
On top of that, it should be remembered, as we were reminded by the author, that “state academic achievement tests have lower reliability and validity for LEP population that are constructed and normed for Native English speakers.” It doesn’t look like Alaska is taking that advice? Is it because a lot of the schools are funded by the state?
The information about the cut off points of compensatory aseessments that allow test scores to compensate for each other is of the past, and that that government now has the nation under the conjunctive model which is known as the NCLBA, which does not allow any one subject to be less than proficient for positive AYP results to occur, despite your subgroup category and it’s issues.

I needed more time to understand, but finally read their interpretations and it took a while for me to see the point of the tables because it was explaining accountability. I am glad for the high expectations, but I remember when this word first came flying around when we were abruptly told that our district was now a standards based model and teachers got so frusterated and turned the accountability word around and left it in the hands of the students with little guidance and direction, either in retaiation against change or maybe that was their interpretation of accountability, literally to mean that the students were now/ finally accountable for their own learning and took no responsibility for their learning after that for the rest of the year. My son got caught in this change over adaptation and it was not cool.

Finally, the information on the LEP baseline scores section about not meeting AYP for four (again) years in a row are considered a school in “crisis” which is supposed to get overhauled as they say, hasn’t happened yet to a school I know and this is their fifth year in crisis. This real life situation is not just a scenario, it’s about student’s lives that are going to affected by this NCLBA’s AYP expectations. It goes to prove that invalid assessments and unreliable reporting can determine if a student gets a diploma or not to be able to make a decent living.

Chapters 1

The language assessment chapter was definitely a general introduction to assessment. I almost got myself stuck on the politics of it even more so than the author did. I noticed that I was expecting more new vocabulary than the author provided also. The information about the test-criterion relationship got me wondering if it's something like Bloom's Taxonomy that we need to be utilizing. Finally, the requirements for the testers that were laid out in the conclusion got me wondering about our practices in KSD. We do not have official testers and the training that we're provided doesn't touch on looking for signs and the theory(ies) behind them. All in all, the chapter was definitely general and I can't wait to read more to see what I'll learn.

The authentic assessment chapter caused me to feel sad, mad, and even guilty (about my philosophy). It was disappointing to see that districts have trouble making effective decisions about testing for English Language learners (because of the politics). In the end, I was glad for the useful information this chapter provided. I also ended up thinking about the Maori article and figured that the characteristics of the performance assessment they had were numbers 1, 3, 4, and maybe 5. I also thought about our district's standards and the GLE's and saw that we seem to be following these guidelines. I kept thinking though that they are not very applicable to the grade I'm teaching- kindergarten, even though this chapter includes K-12, and it even mentioned including pre-K in these instruction and assessment procedures. Finally, I am actually looking forward to more information on reliability and validity.

LING612 9/16 rdgs

Teacher Magazine. April 22, 2008, by the Associated Press 2007. Teacher suspended for refusing to give state test

A 60 year old teacher refusing to administer the WASL test to his students should be expected. I feel like if I was that old and still teaching, then I’d probably do some “radical” stuff myself. I’ve been to a couple of conferences where I’ve seen some of the teachers be quite opinionated and political that I wish more teachers and the public for that matter would ban together and refuse to give the standardized tests; to stand up against the federal government for all the trouble they’re causing by requiring archaic methods of assessment for their accountability sake.
This Seattle teacher did, give a good point about offered monies to him to instead donate their money to an alternative assessment “research” organization that is assessing the state’s education system. Now, this was nice to see, but is there really such an organization?
In the end, this teacher’s demand for justice and equality, (as I believe the boycott will be), was a win-win situation as the he saw it, because he “didn’t get fired.” I wonder if he’d have taken it like Rosa Parks, etc… those people that were into civil rights long ago were able to contribute to the culminating result of their end goal. So, when will it happen in education? Will it ever?
Finally, the Seattle teacher suggests simplistically (it seems) that we have to, “work as hard as we can with these kids and their families.” I had to read this line several times and ended up having more questions from wondering if this statement is as simple as black and white or is it something else? I wish I was there to interview him myself.



Pierce, L.V. (2006). Assessment. In Ovando, C.J., Combs, M.C. & Collier, V. (Eds) Bilingual and ESL Classrooms, Boston: McGraw Hill.

The information that pertained to the politics got me so frustrated, I wished we could all boycott the lawmakers in the federal government like the 60 year old Seattle teacher who refused to give the state standardized test to his students and got suspended, not fired, for a couple of weeks. That, I believe is what needs to be planned by us as a cohort to start this kind of ban before elections this year! Especially, now that we are aware of this assessment politics, we can’t just do what we can in our own classroom. That’s not what the government wants, but they would have to accept our authentic assessments if we boycotted them right now! What do you say about that!?
Anyways, I was surprised to see the authors provide test taking skills information to us in this part of the chapter (Coping with Standardized Tests p.348. After thinking about this information, I realized that although I have access to these test taking strategies, I find that I do not utilize them neither as a student or a teacher (I wonder why?).
Finally, the suggestion to provide ten days of inservice was beautiful to see suggested, and would be beautiful if it actually could happen. Maybe, there’s a way. It was so easy to get worked up reading this selection, I have to work hard to stay positive about assessments that are conducted on our students who should be assessed the way the Maori’s are assessed (Rameka 2007).
Rameka, L. (2007). Maori Approaches to Assessment. Canadian Journal of Native Education: Edmonton, 30, 126-146.

The choice of words used to describe this, as said by the author, overaching philosophical framework was amazing. My vocabulary knowledge was refined. As I read this article, I kept on relating it to our indigenous people here in America. It was amazing to see how carefully detailed the New Zealand’s Indigenous people’s way of life and learning was explained.
The perspectives shared about the children of the Maori people was inspiring. I liked the statement that, “Children’s learning was, therefore, not left to chance (126).”
Of course the information about the government thinking they’re conceiving a conspiracy against them by asserting their rights as the people they are is typical government practice in order to have an excuse to control and subject the people. Luckily, private and public organizations are available resources that can contribute greatly to a cause.
The author (I assume) is an insider to this culture for this article to be as comprehensive about the Maori’s perspectives and lives.
I was just so amazed at the cooperation and teamwork and details that were considered for this framework. I’m going to assume that if the Alaska Native Knowledge Network and their affiliates were to write something up about their work, it might be something like this.
All in all, this was a beautiful article to read. I could imagine the beauty/ serenity in this type of learning environment. I say this, because my Kindergartners come into school and learn a different way to learn that is not natural to them and I find myself not feeling comfortable with molding them to do so- knowing that it is not natural for then to sit and write and listen for many hours a day when all these years they’ve been learning through natural situations that do occur spontaneously. My critical pedagogy is being tested here. I want to move to New Zealand and work in this school that has this awesome authentic assessment information and access.

Triadic scaffolds

Meskill, C. (2005). Triadic scaffolds: Tools for teaching English language learners with computers. Language Learning & Technology, 9, 46-59.

The computer, teacher and a pair of students make the triadic in triadic scaffolds. The teacher supports the computer software's language through her conversations with the beginning ELL students. The "computer-supported classroom discourse" through the use of instructional conversation with the children provide the students with models of what's needed for participation in school.
The scaffolds are the words the teacher uses as models for school participation. The teacher provides scaffolds for the ELLs during their one period a day intensive English class during their computer time through modeling of the different types of language used in the mainstream classroom (and from the computer program). The students were said to be using the language they hear from the teacher and computer activities they used within a couple of days.

I could see how this can be applied to my kindergarten classroom by modeling the language commands during their time on the computer and making sure that they understand them, instead of just assuming that they should already know. Providing this scaffold through modeling the English language's forms and functions of school ("words") language in context/ during the situation, while their attention is high was a good point to mention and take note of.