Tuesday, August 25, 2009

Back to the Outcomes Ass-essment Borg: An Apoplectic Fit

I'm in foul mood today because I have to get dressed, drive up to the self-proclaimed "main campus" (they aren't, they just like to think they are) and sit in on one of those hideous Outcomes Assessment meetings run by the OA Borg, a group of True Believers who get paid a lot of money NOT to teach.

I feel so horribly unethical and insulted in those meetings, like I'm contributing to the downfall of American education. This is, at our college at least, essentially a very obvious creep toward No Child Left Behind at the college level because that worked so well at K-12. I feel as if, by participating, I'm complicit in the problem, that I'm adding to the ignorance of Americans, which means that I'm adding to the evils of the world done by ignorant Americans. I'm not being hyperbolic, either. I feel that, if I were a moral and ethical person, I would refuse to participate.

Of course, if I refuse to participate, then I would also be an unemployed person, being untenured at an institution that does not grant tenure. Still, shouldn't someone, somewhere along the line just say "NO"? The higher up and the greater in numbers, the more effective this "NO" would be; but, as it plays out, it's like the corrupt system of speculation in Little Dorrit: nobody's fault. Everyone just participates in this system that everybody knows is a sham, and everyone says that we have to do it because "THEY" say we that have to and because "THEY say it coming whether we like it or not so we might as well accept it."

This is all obviously a sore point for me. Remember, I grew up in Texas. As an adult I lived in the very district where the No Child Left Behind business was hatched. I knew teachers -- talented, enthusiastic teachers, who quit within a few years because of No Child Left Behind. From the very start, everyone tasked with implementing the program -- that is, the people actually engaged in the actual education of actual students -- knew that it was bullshit and detrimental to education.

Meanwhile, at our college, in our department, we all settled on a truce. Do what they ask, generate the data and hand it over with as little disruption to our own teaching as possible. After all, the OA Borg kept telling us, "You are the professionals. You know your subject. We trust you to come up with the most effective assessment instrument. We will accept what you come up with." If we didn't comply, then, "THEY will come in and create one for you."

Someone actually told that to me yesterday. I wanted to tell her, "c'mon! You are far too old to believe that, if we are good little professors, and do exactly what is expected of us, then THEY are going to leave us alone." I did tell her, "THEY are going to take it over if THEY want to no matter what we do." She has become assimilated. She honestly believes that she can limit the impact of the system by becoming part of it. Our pity for her prevents us from holding her in contempt.

THEY are actually already taking it over. All of that "we trust you" and "you are the professionals" and "we will accept what you come up with" is just smoke. You see, we came up with ours, and they kept sending it back to us. At first, it was just tweaking the language. "Students will understand the causes of the American Revolution," had to be "Students will demonstrate an understanding of the causes of the American Revolution." That sort of thing. Then, their revisions became more detailed. "How does this question show that students are demonstrating the causes of the American Revolution?" they wanted to know.

Ultimately, what they wanted from us was an essay-based exam. Ultimately, we refuse to give it to them. Understanding that, really, all the Borg really wants are numbers to plug into spreadsheets to generate charts and graphs that demonstrate "learning," we decided to create a multiple choice "instrument" that covered basic questions that you can't get through a history class without knowing. They aren't quite "who's buried in Grant's Tomb" questions. They are more like "Which of the following was NOT a cause of the Civil War," followed by a list of items that incorporates several different interpretations with one very wrong answer like "the invasion of Poland." That way, if any of us did not emphasize an interpretation that another did -- and we have to be aware that the adjuncts have their own interpretations, too, since they aren't part of this process -- then the student could at least intuit the correct answer.

We would all give this "instrument" as a quiz of some sort. Then, we could just plug the numbers into the software at the end of the semester and feed the numbers to the Borg. The Borg is fed with minimal impact on the way that we run our own classes. That was what we agreed was the best way to approach this process.

We see a huge difference between feeding numbers to the Borg and education. We test education with our own assignments and exams, which are based on writing and through which we can see if students are improving their thought processes. We feed numbers to the Borg with this "instrument" thing and evaluate our students based on our own thing.

We are fully aware that ours is a joke of an "instrument," but we felt that anything more would begin to dictate what we teach in class. While we all know that essays are the best way to test understanding of material, if we all gave the same essay question using the same rubric for grading those essays, then we would essentially be creating a common exam, and we all agreed that a common exam would mean that we would end up teaching to the test.

Make no mistake, the Borg wants it all to be EXACTLY the same. You can't just have a rubric with general parts like "opposition to taxation." You have to have parts that say, "Stamp Act," and "Boston Tea Party," and so forth. While the Borg insists that they are not advocating standardized testing or common exams, they really are.

The OA Borg becomes more and more intrusive with more and more forms and more and more rejection of our own "assessment tools." They say, "we let you create your own tool because we trust that you know what you are doing." Then, when we do, they send it back saying "this isn't good enough." The process repeats until they are satisfied, which means that they do have requirements for these "instruments," (please! They are "tests"!) but to keep up the mendacity of "you create the instrument yourselves," they have to coerce us into figuring out what it is and giving it to them. To keep up the lie that "we aren't asking for a standardized or common exam" they have to get us to decide that a standardized and common exam is the best option.

Clearly, they do have to coerce our department because we don't buy it and we have no respect for their process. They want us to give them honest-to-god exams that demonstrate education. We believe that we already do, they just aren't the same exams approaching the questions of the course in the exact same way. They don't accept that method because, if their numbers are going to mean anything, they need sameness. To achieve that sameness, they want us to give the same exam.

We rebel against that because we see that as standardized testing with common exams. We see that as not only an infringement on our freedom in the classroom but also the source of our students being untrained and even frightened to think on their own after 12 years of similar standardized testing. We teach in the humanities. Education in the humanities cannot be quantified in the same way as, say, business productivity. Yet, the way that the Borg describes their ideal education, you and I and the professors at Harvard or even the Sorbonne should all be giving the same exam with the same exact rubric so that that THEY can prove that education is happening. In fact, I often wonder if they expect the students to turn in the same exact answers.

Who, by the way, are THEY? Because THEY should be resisted.

NOTE: I vent here so that I will not be this pissed off at the meeting. As great as 99% of my colleagues are, and as great a place as my institution is to work in general, as with most places, the administrative culture does not appreciate speaking truth to power. If something has been implemented, it is 100% good, and there will be no complaints.


Susan said...

I'm sorry, Clio. Your outcomes stuff is much worse than ours, and I don't blame you for being frustrated. We have Program Learning OUtcomes (PLOs) which are things like "Majors will know how to analyze primary and secondary sources". Then we design outcomes for each course that connect to the PLOs.
We can evaluate the PLOs any way we want, but we've decided to just collect all student papers electronically, and use them as the basis for assessing one outcomes a year.

The sociology dept has outcomes like "Understand the causes of inequality" and in any course where it's relevant -- i.e. gender, race, ethnicity etc. -- they add a question to the final exam tied explicitly to the outcome, and then just need to assess that.

Clio Bluestocking said...

See, that is part of what gets us so frustrated. We already have outcomes listed for the class. We already test those outcomes in our exams. They just aren't the exact same exams. This OA thing is meant to create sameness, which is nigh on impossible. It's also supposed to help us become "better teachers." Yet, we are already evaluated by the chair, the dean, a peer, and students. We also turn in a self-evaluation. Yet, none of this is good enough because it doesn't generate numbers. Generating numbers is what it's all about.

I begin to wonder at the budget for this whole process -- along with the hours and resources used by the college departments. How much does all of this cost?

Now I'm getting worked up again!

Roxie Smith Lindemann said...

Oh, Clio, the English profs and LOA haters of Roxie's World feel your pain! We, too, have marveled at how sheep-like colleges and universities have been toward something that is at best a sham and at worst a serious erosion of academic freedom, not to mention an astonishing waste of faculty time and energy. Top it all off with news this morning that the state of Maryland is planning furloughs of up to TEN days for state employees, and you've got a serious case of the back-to-school blues taking hold. Good thing one of my profs has the year off this year!

Notorious Ph.D. said...

OY. Assessment. I think you should replace your word "instrument" with its synonym "tool", which is pretty descriptive of how we all feel every time we give coerced consent to this B.S.

Ann said...

. . . and speaking of "tools," enter Historiann!

I seriously feel your pain. I think your analysis is accurate and not at all too cynical. The whole concept of assessment is cynical. Why can't these asses see that?

Sandwichman said...

Don't worry, Clio, academic freedom is alive and well at the Harvard Kennedy School of Government! I know because I just received an email from the Dean there affirming the fundamental tenets of academic freedom.

"Dear Tom," wrote the Dean, "One of the fundamental tenets of the Kennedy School and all American universities is academic freedom. Scholars introduce ideas into the public arena where they can be energetically discussed and debated, and that is what Professor X has done with the publication of his op-ed. This free exchange of ideas and opinions is one of the lynchpins of a democracy.

"As a matter of academic policy, the Kennedy School does not restrict, interfere with, or take a position on the research conclusions reached by individual faculty members."

You see, I had written to the Dean pointing out egregious factual errors and misrepresentations in Professor X's op-ed and asking to have my response to X's op-ed published on the Kennedy School website in conjunction with Professor X's. I sought to debate, not to censor him.

The statements in question, by the way, were not the "conclusions of research" conducted by Professor X but simply the rote repetition of a very old and thoroughly discredited lump of textbook lore. I have written two published articles documenting the historical, epistemological and empirical illegitimacy of the claim recited by Professor X as gospel truth. Professor X did not reply to my direct request to debate the matter.

The Dean very deftly ignored my request for debate and instead re-framed it as an attempt to abridge academic freedom. You see, Clio, "academic freedom" is one of those concepts like freedom of the press, which is guaranteed, A.J. Liebling pointed out, "only to those who own one."

Digger said...

Ouch @ Clio. How do you not put your head through a wall? The numbers thing is maddening, but the icing is the very poor attempt at invisible manipulation. Come off it, borg-people, they're on to you! Just make it so, already! Then again, if they just TOLD you they wanted standardized tests and that's what you had to cough up, then they'd lose their plausible deniability. Very passive aggressive-like, in a butt-covering sort of way.

bitternsweet said...

Thank you, Clio, for clarifying for me why everything about assessment makes my skin crawl, makes me nauseous, makes me want to run away and hide. THEY are the BORG! I just knew it!

But seriously, your explanation of how "going along with" the mechanics of assessment in an attempt to just "get along" makes one complicit with an erosion of American education ... well, it's both chilling and, I fear, accurate. I'm completely guilty of this -- because I don't have the strength or status (like you, I'm untenured) to fight against the tide, I've been just going along with the stupid steps I've been required to implement, dragging my feet a little bit, doing it badly or sloppily, but still doing it.

My conclusion: Assessment sucks.I really don't understand how seemingly intelligent people -- most who have been good teachers at some point -- buy their own BS. They have truly been assimilated.

Thanks for putting it out there.

Ubab said...

Less outcome, more income I always say.

Another Damned Medievalist said...

I sort of vehemently and rabidly disagreed over at Historiann's, because you really pushed one of my buttons. I work in a SACS place, and they are one of the most intrusive assessment agencies. And number-wise, there is nothing wrong with saying that we think we are managing to do our jobs if 60% or our students are getting a C or better -- as long as we are really clear about what we assess. And I have to deal with this, "it's the BORG, and it's all a right-wing conspiracy to take our freeeeeeeedom" stuff all the time. And I spend so much time having to deal with that, just to try to get faculty to take some ownership of the process, and actually get them to articulate what it is that they think is important for students to learn, and how it is they know when students learn it, KNOWING that after all that time, they are going to piss off and not help at all, and leave the work to a few people. Accreditation is not going away. Accreditation agencies want outcomes assessment. There are two choices, as far as I can see -- take charge of it and set your own standards, or let someone else do it. And they will. I am currently teaching gen ed classes whose outcomes were set by a committee on which there were NO historians, because my colleagues in the department at the time felt as you do, and decided that they weren't going to be bothered, and would just do things their own way anyway.

Assessment is another tool, that's all. It's like smartboards, podcasts, and powerpoints. It can be used for good or for ill, but it's not going away. And honestly? if you are a good and self-reflective faculty member who cares about student learning, it's all about putting what you have always done well into a different dialect. That's all.

CTS said...

Hi, Clio. I came here from IHE, went over to Historiann's a read the post and comments over there.

I think Another Damned Medievalist and others have made some good points, so I won't weigh in on the core of the issues. Rather, I have some suggestions for you and your colleagues:

Get yourselves educated. You can probably access various AAC&U documents that will show you what intelligent and useful learning assessment looks like (maybe your fancy OA can provide the materials). You might get the college to send you and others to one of the many conferences, where you could meet people - not Borgs - who are interested in and savvy about student learning assessment as done by faculty for meanignful aims.

Rethink your multiple choice tool. If tests of that sort are not what you all usually prefer for your courses, why use them for assessment? Of course, essay reading is more difficult, but it probably tells you more of what you want to know about student acheivements. Believe it or not, sitting down with your colleagues and discussing a piece of student work is actually really interesting; it also brings out all kinds of considerations that just do not seem to come up in our passing gab about 'students' in general.

Resist, but do so on the basis of information (as my first point notes) and good sense. Someone over at Historiann's claimed that our teaching has no outcomes. That is inane. If we do not know what we want our students to learn, be able to do, and have as intellectual habits, why the heck should anyone (often their parents) be paying for them to spend time with us?

Anyhow, I think your particular school is going about this badly, from what you write. If the faculty can somehow take this work back/over, you will all be better off.

Best of luck,

Brian D-L said...

Imagine the nerve of some people, expecting that we, as teachers, should be able to explain how we know that our students are learning what we think we are teaching!

Maude Lebowski said...

Clio, this was great!

And timely, too, as I find myself on the cusp of being appointed to such a committee. It is about 90% certain that my chair will assign me to the college's assessment committee. My first committee ever, and I land in this one--most likely.


Keep the faith, sister!

alia said...

seems this day started from troubles:( college essays


Unless noted otherwise, copyright for all written content held by Clio Bluestocking.