Week 3
Chapter 10 discusses evaluation in
instructional design and provides you with two evaluation models, the CIPP and
Kirkpatrick models for evaluation. Search for at least two other models used
for evaluation and summarize these models. Describe how you would use them to
evaluate your instruction.
“Evaluation is the process of
determining the merit, worth and value of things and evaluations are the
products of that process (Reiser 97).” Evaluation models are designed with
specific goals in mind in order to have a profound impact on the evaluation
process. CIPP developed by Stufflebeam stands for context, input, process and
final, product. Each step represents a significant and critical part of the
evaluation process. CIPP looks at how instruction is implemented, how effective
it is, assessing outcomes and what resources are available for the project.
This evaluation model has since become a formal process widely used though
education today.
Kirkpatrick’s model primarily focused
on training evaluation. Within this model are four levels of evaluation. The
levels include information covering reaction, learning, behavior and lastly,
results. The reaction level assesses the attitude the learner has toward their
educational experience. This could be a quick survey or questionnaire document.
The learning level looks at what the participant learned and to what extent.
Learning objectives that were covered should align with any given quiz or test
to get an accurate measure of what was taught. The behavior level looks at
whether the participants/learner changed their behavior due to the training
program. Do they consistently use the skills they were taught? Kirkpatrick
believes the participants should be contacted in order to conduct an
appropriate follow up. The last level is results. This is where we see if the
participant reached the desired goals or outcomes.
Another model I researched was Rossi’s Five Domain Evaluation Model.
Peter Rossi developed this model in the 1970’s. However, it is still commonly
used in the education field today. In this model, each step is uniquely designed
where it is shaped and molded to suitably fit the needs of the program. This
model focuses on evaluation questions such as “who should be involved?” “What level of participation is desired?” and
“Should an internal/external evaluator be used? (Reiser 98)” Each step or level
is broken into five domains.
·
Needs assessment
·
Theory assessment
·
Implementation assessment
·
Impact assessment
·
Efficiency assessment
Needs assessment will ask the question
of “is this type of program necessary?” Theory assessment addresses if the
lesson is designed in an appropriate way that it should work. Implementation
will look to see if the instruction was implemented in suitable way in order
for learning and success to take place. Efficiency is the domain where we look
to see if the program is having an impact and at what cost.
Many campuses across my district have
implemented the use of “intervention.” Intervention provides us an hour of time
to spend on math and reading utilizing small group instruction. Students are
grouped based on data and state assessments. I could see myself using Rossi’s Five Domain Evaluation Model in
order to better assess the progress and instruction during our intervention
time. Following these guidelines of Rossi’s Five Domain Evaluation could change
the way we design and produce the instruction in this hour. This would allow
teachers to see if their instruction is being implemented correctly, are we
having an impact and is what we are doing efficient.
The next model I researched was Patton’s Utilization Focused Evaluation. This
model states that the evaluation process and outcomes are of little value
unless we actually practice them. “The cardinal rule in U-FE is that the
utility of an evaluation is to be judged by the degree to which it is used
(Reiser 102).” This model states that
the evaluation process is carefully examined from beginning to end. Process use is a major concept
throughout this model. This occurs when educators understand the logic of the
evaluation process (Reiser 102). This model follows several stages.
·
Conduct a readiness
assessment
·
Identify the “primary
intended users” and develop a working relationship with them
·
Conduct a situational
analysis
·
Identify the “primary
intended uses”
·
Focus the evaluation
·
Design the evaluation
·
Collect, analyze, and
interpret the evaluation data
·
Continually facilitate
evaluation use
·
Conduct a metaevaluation
“An evaluation that is not used is of
little use to an organization; therefore it is wise to consider the principles
provided in U-FE (Reiser 102).” I think
that I could use this model for when our school conducts Curriculum Based
Assessments (CBA’s) or six weeks test. I think that if I followed these steps
for evaluating data and instruction that we could make significant changes and
perhaps increase test scores. This would be a great way to adjust instruction
in order to re-teach certain student expectations and TEKS.
Reflect on what other questions that
instructional design evaluation should address besides whether the
instructional design leads to comparable amounts of learning and learner
satisfaction as traditional methods. What else would be useful to know?
Throughout my research on Evaluation in Instructional Design I was
able to further develop questions that I think evaluations should address.
“Are some parts of the program working, and
others not?”
“What materials or manipulatives would
best support my instruction in order to lead to a more successful evaluation?”
“Does environment play a significant
role in the evaluation process?”
“How many instructors are used during
the evaluation process?”
“How are we going to monitor and
document progress?”
Chapter's 12 & 13
focus on project management and how to manage projects when resources are scarce.
You have been assigned to develop a series of professional development sessions
focusing on technology use in the classroom for teachers during a time of
economic decline. How will you use Situational Leadership to facilitate this
project and manage scarce resources?
Designing professional development
sessions can be an arduous task. With the proper and skillful leadership one
can develop high quality professional learning. Management is defined as, “…the
process of working with and through individuals and groups and other resources
(equipment, capital, technology) to accomplish organizational goals (Reiser
117).” Paul Hersey developed the
situational leadership model. In his model there are four phases in order to produce
successful management. Phase one says you should be very direct and specific.
Phase two states you move towards a different role where you reward
improvements being made. Phase three is where we focus on the results. Lastly
in phase 4 the leader does not act as a director but more as of a facilitator. In
the situational leadership approach the leader will learn to adapt their
leadership style according to the people you are leading.
When designing a project or staff
development session, it’s important to clearly set a starting and stopping
point as well as your objective for the lesson. However another key aspect in
designing staff development is your budget and resources. Though many school
districts across Texas seem to have increasingly high budgets, many schools
still do not.
If I was to design staff development
sessions using the situational leadership approach, I would first set a goal as
to what I want to accomplish with the technology. I would make sure to
communicate well with the teachers to see what technologies they would find
most effective during their teaching and instruction. I would also make sure to
train the staff if they’re unfamiliar and inexperienced with how to use the
technology or devices. It’s important to purchase technology that will be put
to good use. Budget cuts have had a huge
impact on technology within the classroom. Best Buy is one of many companies
that makes donations for nonprofit organizations that serve children in grades
K-12. More information regarding donations can be found at the link provided
below.
Many high schools in my district this
year have also implemented “Bring your own device.” Students are encouraged to
bring their own technologies or smart phones in order to participate in lessons
and activities. This is a phenomenal way to get technology more involved in our
schools at no cost to the district.