Instructional design and Performance Improvement: You want me to design it and then what?

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency."  Bill Gates

This weeks reflection was a bit of a challenge for me as I tried to search for two other ways to evaluate Instructional Design.  I am sure there were more out there, but I was unable to find any that I felt fit with the topic this week.  The quote above fit well with the idea of Return on Investment and Managing Resources.  It also seemed to sum up Performance Support Systems and Knowledge Managment Systems in Section IV.  

1.  Chapters in Section III discuss evaluation in instructional design and provide three evaluation models:  CIPP, Five-Domain, and Kirkpatrick’s Training Evaluation model.  Search for at least two other models used for evaluation and summarize these models.  Describe how you would use them to evaluate your instruction.

ADDIE Model

This model was first presented in 1975 by Florida State University.  They developed it for the U.S. Army to use in their training.  It has been revised over the years to move from a linear model into a more dynamic and interrelated concept.  ADDIE is an Instructional Systems Design model which is broader in scope than the traditional Instructional Design models which focus mostly on analysis and design.  The ADDIE model focuses on five phases:

• Analysis – basis for the training program asking who will be trained, what needs to be learned, how will the training occur and where.  Simply put it is where the main problem is identified and learning outcomes are set.  Uses a Backwards Planning Model to make sure instruction connects to the organization’s needs, vision, mission, and values.
• Design – driven by products or results of the analysis phase and is completed when a blueprint of the training process to be developed is created.  Must contain five key outputs:  learning objectives, performance test, learning steps, entry behaviors, and structure and sequence of the instructional outline.
• Development – builds on the learning objectives and takes previous content to create learning platform.  This in turn results in the instructional courseware.
• Implementation – instructional courseware is delivered to the organization and presented to learners.  The previous three phases insure that the instructional platform performs as predicted and supports learner’s needs.
• Evaluation – is the determination that the instruction meets the criteria outlined during the analysis and design phase.  Often we think it only occurs at the end of the design process, but in actuality it is on-going from beginning to end.  In the ADDIE model it is both formative and summative in nature. 
  

Since the ADDIE Model is designed to evaluate the Instructional Design throughout the entire process, it is a valid assessment tool.  It is based on Kirkpatrick’s Four Levels of Evaluation, however it flips them to look at the results first.  This creates the Backwards Design Model which many in education use to design instruction in the classroom.  In other words, what do we want the learners to know, so we may create instruction that gets those results?  (Clark, www.nwlink.com)

Kemp Instructional Design Model

The Kemp design model takes a holistic approach to designing instruction.  This model takes into consideration all factors in the learning environment from learner characteristics and learning objectives, to instructional activities and resources needed.  It has a cycle that incorporates nine key elements and continuous implementation and evaluation.  In other words, while the instruction is being designed, evaluators are determining the effectiveness of each component on learning.    The nine key elements of the Kemp ID model are:

1. Identify instructional problems, and specify goals for designing an instructional program.
2. Examine learner characteristics that should receive attention during planning.
3. Identify subject content, and analyze task components related to stated goals and purposes. 
4. State instructional objectives for the learner.
5. Sequence content within each instructional unit for logical learning.
6. Design instructional strategies so that each learner can master the objectives.
7. Plan the instructional message and delivery.
8. Develop evaluation instruments to assess objectives.
9. Select resources to support instruction and learning activities.
   

This model focuses on the learner’s needs and instructional objectives, which make it effective for creating whole units of instruction, as well as, individual lessons. 

Most of the time our units of instruction are based on the Common Based Assessment (CBA) that our district creates to determine if students have mastered the learning objectives for that unit.  With this backward design in mind, I see how the ADDIE ISD model might be useful in creating instruction that gets the results we are looking for.  However, as a teacher, I need a model that helps assess the objectives that I want my students to meet.  The Kemp ID model looks at the learning objectives with learner characteristics to create instruction that meets all my needs.  It takes pedagogy, content, and technology into consideration when designing instruction.  I definitely see myself using this model over the others.  I also agree with many of the websites that I read through that sometimes it is not one Instructional Design Model that works all the time.  By looking at several ID models, we are able to piece together instruction that fits all of our needs. 

2.  Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods.  Should return on investment and management of resources also be considered in evaluations of instructional programs?  What other measures should/could be considered?

In the last two years, our district has been tasked with the challenge of evaluating textbooks to determine if they meet the objectives or TEKS that TEA has set out for Science instruction.  We have looked at the information in the textbooks to make sure they addressed the content needed and learning objectives.  Functionality was another criteria addressed since we have been told that TEA will not allow textbook publishers to print traditional copies for school districts.  With the push for technology use, the textbooks will now need to be online or in a PDF or ebook format.  So, we were asked to test a couple of “textbook” websites to see if they fit the bill.  I looked for a few things when cruising their websites and that was ease-of-use, correct content, connection to real-world topics, and an evaluation component to see if students met the objectives for the unit of instruction.  Ease-of-use is a big thing for instruction that is being used in the classroom.  I may teach Pre-AP Science, but my students still need an instructional resource that is designed with their level of development in mind.  I won’t name the specific publishers that I tested, but will say that one did meet most of my criteria.  I know that I am supposed to be addressing instructional design, but as a teacher, my instruction is designed with the resources we have available in mind.  If the resources our district gives us are not lending to the learning environment, then we need to find something that does.  It also must give us the best return on investment possible.  Let’s face it, Texas school districts are dealing with budget shortfalls just like the state and federal governments.  We are asked to teach more students with less money for supplies and teaching resources.  Our time is very valuable and searching for the best resources to complete an unit of instruction can be very time consuming and costly.  I think that when evaluating instructional design, ROI and management of resources is a big deal, but also whether or not it is effective in meeting the learning objectives.    

3.  Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning.  Not all problems in learning and/or performance require an instructional one.  Many times a non-instructional approach is a more appropriate solution.  Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem.  Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem?

"There can be infinite uses of the computer and of new age technology, but if teachers themselves are not able to bring it into the classroom and make it work, then it fails."  Nancy Kassebaum 

             Since I began teaching five years ago, the recurring argument has been about how to use technology in our classrooms while trying to manage more students than we had before.  I am not sure why this is such a big issue, but it has become the proverbial “broken record” complaint.  When I think of performance problems, this one seems to be one of the biggest.  I say this because it does not necessarily have anything to do with teaching the students and more about personal issues with technology.  The “complainers” all demand more training from administration on how to use the technology we are given.  They think with more training they will miraculously be able to make wonderful lessons using the technology given to us.  The problem is, they really do not know how to use the technology and deep down do not want to use the technology.  Therefore, we have a performance problem that needs to be improved in some way.  While reading the chapter on Human Performance Improvement, one thing I noticed is that the word performance was defined by Gilbert as “the valued accomplishment derived by costly behavior”.  This is an interesting point, because behavior is one of the issues we have.  I don’t mean they have inappropriate behavior, I just think that their negative attitude directly relates to how they perform in the classroom and thus their behavior does not result in an increasing value.  The question now is, how do we address these problems in a way that improves performance and attitude?  I think informal learning has benefitted our campus by allowing some of the “knowledgeable” teachers to mentor the others.  I personally have helped teachers who are not as “into” technology as myself, by finding lessons that are easy to use and then assisting in implementing them in the classroom.  This does not cost me or my peers anything and the district saves money on formal training sessions.  I also believe the knowledge management systems may benefit these teachers, but only if they will access them.  We use OneNote to consolidate training materials, campus forms, and even testing data for our own campus.  Our district uses a website that houses the district forms, and an internal server that allows us to access training videos and podcasts, curriculum from our instructional facilitators, and even updates for programs we use.  These knowledge management systems keep us up-to-date while minimizing the need for formal trainings and meetings.  Our principal also addresses issues through meetings with our department chairs, who pass on the information to each individual department, and emails from the district office that address items we need to change or important details we need to know.  None of these require “formal lessons” and everyone benefits from them. 

Another example of how formal instruction may not benefit is with my husband’s work.  His employment (which I will not name specifically because it is a federal government entity) has many different systems for each component of the organization.  Most of the systems are not available to all employees for one reason or another.  They do not interact with one another often as they are housed in different locations even within the region.  So when an issue arises, they must call each department to obtain the information needed to make a decision.  If the knowledge management systems were all accessible to everyone, then this process might take less time.  They also have issues with some supervisors who like to promote based on people’s personalities (basically they brown-nose their way up the ladder) rather than on their own performance, which as Gilbert outlined in his principles and theorems, encourages incompetent employees (Reiser & Dempsey, 2012 pg. 137).  These employees are able to socialize, but do not have any knowledge of the performance factors that allow them to do their job well.  This also causes issues as these employees are often out to discredit others in the department and spend more time on this endeavor than their jobs.  All the performance improvement support, knowledge management systems, and informal learning cannot overcome the “good -old-buddy system” that occurs in this situation.  

Resources:

Clark, D. (1995, July 13). ADDIE Model. Performance, Learning, Leadership, and Knowledge. Retrieved June 21, 2013, from http://www.nwlink.com/~donclark/history_isd/addie.html#dynamic

File:ADDIE Model of Design.jpg - Wikipedia, the free encyclopedia. (2012, April 18). Wikipedia, the free encyclopedia. Retrieved June 21, 2013, from http://en.wikipedia.org/wiki/File:ADDIE_Model_of_Design.jpg

Instructional Design Models and Methods | Instructional Design Central. (n.d.). Instructional Design Central | Instructional Designer Resources, Information, and Professional Networking Services. Retrieved June 21, 2013, from http://www.instructionaldesigncentral.com/htm/IDC_instructionaldesignmodels.htm

Morrison, G. (n.d.). Kemp Design Model. Instructional Design. Retrieved June 21, 2013, from http://www.instructionaldesign.org/models/kemp_model.html

Schneider, D. K. (2007, June 4). Kemp design model - EduTech Wiki. EduTech Wiki. Retrieved June 21, 2013, from http://edutechwiki.unige.ch/en/Kemp_design_model