Goal 8:

    Carry out a formative and summative evaluation of Project activities.

Objective    /    Outcomes    /    Individuals Responsible    /     Time Line    /    Budget    /  Weekly Update

Weekly Update:

4/13/2000

Donna has completed the "Baseline Technology Analysis" component of our Project.  She sent the resulting report to me a couple of weeks ago, but it has taken me this long to have the report reformated into "html."  To see a copy of hte report simply "click" on:

    "Baseline Technology Analysis"

The resulting information provides an excellent "snap shot" of where we are now with our technology "use", "wants" and "needs."  I would urge everyone to take a few minutes to read over the report and to reflect upon both how far we have come and where we are yet heading.  These are exciting times to be in Deaf Ed. and Donna's work helps to insure we both remember where we have been and to decide where we want to go.

Harold Johnson
 

Weekly report March 10, 2000

I have been monitoring the returns on the baseline surveys and feel that it is now time to stop collecting data and prepare the final report on this activity. We will probably have 102 responses. I will know for certain when I finish the data entry for the last round.
I have conferred with the team leaders to determine their activities with regard to the evaluation needed for their goals. Each team leader will present information based on evaluation data related to their goals, and the syllabi and multimedia teams will collect focus group data in New Orleans.
I also attended the faculty development workshop at Fontbonne College in St. Louis Feb. 25 and 26. I was a participant observer and collected evaluation data on evaluation forms at the end of the workshop as well.
I worked with Karen Dilka and Harold Johnson on the preparation of the catalyst grant evaluation plan.
 

Weekly report Feb. 14, 2000

I sent the baseline survey to the ACEDHH listserve and sent mail surveys to those who had not responded to the first request. I received a total of 29 useable responses, 4 that were returned by the post office as undeliverable and 4 people wrote back saying they were not teaching teachers of the D/HH.. If we subtract those 8 as not being available or appropriate for participation, we have a 62% response rate. I consider that an acceptable level of response for use in decision making and to indicate broad support for the project. I entered the data and analyzed the complete set of responses based on 82 participants.
I prepared and distributed the third draft report of the baseline survey. The results are interesting in that we can see the interest in case studies, the use of technology in syllabi, the desire for additional training, and the barriers to current use. All of this information should be useful for the preparation of the next proposal and for the presentations in New Orleans.
I also contacted all the team leaders to determine their level of activity with regard to the evaluation components associated with their goals. I will follow-up with them this week to determine the status for each goal. I have made plans to go to St. Louis for the faculty development workshop and to begin planning evaluation activities with Harold and Karen re: the a proposal to continue and expand this effort.
 

Weekly Report Jan. 20, 2000

 I entered data from mail respondents in the baseline survey (5) and three
other respondents who printed the survey off the web and mailed their
responses to me, and added these 8 plus 8 more survey respondents who
submitted their responses electronically to the data base.  There are now 53
respondents (38%).  I analyzed the data, created a report, and distributed
it to the goal leaders and all the members of the faculty development group.
 I have also begun discussions re: involvement of other goal leaders in
evaluation activities, the possible use of the ACEDHH meeting venue for
conducting formative evaluation activities, and use of the time at the
meeting for presentation of results of the evaluation.
 

Dec. 22, 1999

First Draft Report of Baseline Survey for:

Technological Innovations & Collaboration for the Field of Deaf/Hard of Hearing Teacher Preparation Project

As of December 22, 1999

Analyzing 37 responses

Submitted by: Dr. Donna M. Mertens, Project Evaluator

Please note: This is a preliminary report. Additional responses will be entered at a later date. This represents all responses that were submitted in electronic format as of this date. Additional responses have been submitted in written form and will be entered at a later date. Additional reminders will be sent to nonrespondents in hopes of raising the number of responses.

I was able to identify 140 ACE-D/HH members. Of those, 111 had email addresses or were members of the list serve. The baseline response form was sent electronically to all 111 of those members. An additional 29 members were sent paper copies of the response form in the mail. None of the mailed responses are included in this preliminary report, however, only one mailed response has been received as of this date.

    Draft survey was prepared for Baseline data collection and shared with Team leaders and Faculty development team for feedback and testing for web-based delivery. Feedback was obtained; survey was revised; bugs were worked out for delivery on the web.

    Baseline Response Form was distributed to 98 listserve participants on Sunday, Dec. 5; a follow-up reminder was sent to listserve on Thursday, Dec. 9.

    Mailing labels were obtained for members not on the listserve. Email addresses were found for 14 members and the form was sent to them on email on Dec. 9. Paper copies were mailed to 29 members without email addresses on Friday, Dec. 10.
 

    Nov. 18, 1999

    I attended the faculty training workshop and faculty development team planning sessions in Northampton, MA Nov. 11-13.  I collected evaluation data, analyzed, and reported the results for the faculty development workshop. I developed a plan for collection of baseline data for all the goals for the project.  The first step in the plan was to identify a web based survey program that could be used for this purpose. The one I have selected is: Perseus Survey Solutions for the Web and can be found at: http://www.perseusdevelopment.com/

    I asked the project directors and the faculty development team for reactions to this choice and the plan for data collection.  Here is what the Perseus program can do: It allows me to send out a survey as an attachment to email. So, people who can and will open attachments can answer that form and their responses will be sent directly to me in an analyzable data base. It can also be saved to a web site and people who prefer and can, can click on the web address and answer the survey that way and again the results will be sent to me.  I requested information from Harold re: putting the survey on ACE-D/HH?s web site as well as information re: how many of ACE-D/HH members do not participate in the listserve and must have a paper copy of the survey sent to them. I will be preparing a draft of the survey itself sometime between today (Th. and Mon.) and will share that for your review and comment.

    11/1/99

    I began the process of asking the grant goal leaders to participate in the determination of need for baseline data for their goals. I asked them to review a survey that Alan Marvelli has used at Smith to determine needs for training in technology.  I realize that the focus of this questionnaire is on faculty development and so closely supports the collection of baseline data for the faculty development goal of the technology grant.  However, as I review the other goals, it seems that baseline data could be useful for the other goal areas also. I would like to have only one survey sent out to the ACE-D/HH members that would cover all the goals.  Therefore, I asked that each team leader look  at the survey to determine issues and/or items that could be added to this survey then we could modify it and send it out over the ACE-D/HH network. I would like to do this be the end of November if possible. I suggested that the team leaders circulate this message to their team members and get feedback to me by Monday next week (Nov. 8).
    Specifically, I asked  Karen to determine how we could address the goal of increasing participation in technology-based training for which we said that we would determine the extent of participation and then identify barriers for nonparticipants. How can that be addressed in the survey?  For Katharine, we said we'd have at least 5 syllabi that integrate technology... Is there a need for any baseline data on this goal?  For Barbara, we said we'd have 2-3 case studies...Is there a need for baseline data here?  For Alan, we said we'd see a 25% increase in number of faculty using technology-based teaching, so we need to know  the number that is out there now.  For David, what baseline data would be helpful for the funding goal to include in grant proposals?  For Harold, what kind of recruitment baseline data would be helpful and is this the right mechanism for getting that?  I will see Karen, Harold, and Alan in MA the end of next week (Nov. 11-13) and that would be a good time to talk. It would be helpful to have feedback from the other team leaders at that time for me to bring into the discussion.  DonnaMMertens Project Evaluator

    10/07/99

   I have made arrangements to attend the team leader planning meeting and workshop the second week of Nov. I have reviewed another possible survey form to collect data base information, one that was used at Smith College. It looks very good. I am thinking we might be able to settle on an instrument   During the first day of the planning meetings, then we could pilot test it with the people who attend the first meeting. Alan and I have plans to talk next week to discuss evaluation as an agenda item for the planning meeting and the evaluation of the first workshop. I requested data on goal 1 from the team leaders to document progress in terms of participation and to use as a sampling source when baseline data are collected. I continue to store weekly progress reports for all goals in a searchable data base.

    9/30/00

   Asked team leaders for feedback on baseline. Am still waiting for response. Agreed to attend meeting second week of Nov. with team leaders and workshop. Will plan to bring copies of possible baseline data collection instruments to meeting for review and discussion. Harold agreed that he could put the survey on the web site if that is the best way to reach respondents. Added data to data base on weekly up-dates for each goal.

    9/23/99

    I sent the following message to team leaders:

I have been doing some research looking for instruments that others have used to determine the extent to which faculty in teacher ed programs use technology in their teaching. I thought I'd make inquiry as to whether any of you had a favorite survey form that you have used or had used on you in the past. If so, I'd appreciate a copy fax 202 651-5710 (voice 202 651 5202), or email as attachment if you have it in electronic form. I came across an interesting report entit  led: Will new teachers be prepared to teach in a digital age? A national survey on information technology in teacher education. The survey was of departments of education and asked the respondents to rate their teacher-training institutions on coursework, faculty capacity and use of technology, facilities, field experience opportunities and graduates' skills. They distributed it via email, HTML, and surface mail. You can view the full report and instruments at the following web site:

     www.milkenexchange.org

Or, if you want to see the report in ERIC, the number is: ED428072.  It seems to touch on many of the areas of interest present in our own endeavor. I would be curious if you have a chance to look at it and if you have any reactions, feedback you would like to share with me. If there is enthusiasm, I could approach the publisher/author to see if they would let us use some or all of the survey. We could distribute it through the listserve or web site or surface mail...But, I thought I'd let you know what I had found so far. We may want to go with something a bit simpler, but I welcome your thoughts on the matter.

In addition, I set up a system to keep a searchable data base of the weekly up-dates that I can use as evaluation data later in the project.  And, I have been in communication with Allan re: logistics for a team leaders meeting in early Nov.

    9/15/99

    I reviewed the proposal for important evaluation activities. The collection of baseline information on faculty use of technology is the first priority. This activity will be useful for all the other goals in terms of measuring progress during the period of the grant. I have reviewed a number of possible surveys that have been used in other research studies of technology use. I have a few more that I want to look at. I plan to contact the team leaders to see if they have knowledge of other surveys that they thought might be appropriate. I am also wondering if it would be best to try to share these with the team leaders via fax or email or if we will have a team leader meeting where we could discuss this approach and the possible surveys.

    9/9/99

    This week I spoke with Harold Johnson on the phone to learn about the beginning activities in the project. He sent me the list of goal leaders with contact information. I made email contact with the goal leaders to introduce myself to them and let them know my plan for the next week. My plan is to review the proposal and identify early evaluation activities that need to be undertaken, and to review possible instruments that could be used to establish baseline information re: faculty use of technology in teaching.  I will send these possible instruments around to goal leaders to get their input into the appropriateness of the information to be collected.

submitted by Donna M. Mertens

Objectives:

    Determine progress towards each of the seven other project goals, including the determination of the extent of involvement of D/HH teacher education programs in the project; document the development and quality of the course syllabi and case studies; determine the needs of faculty for further equipment and training; asses the identification of potential funding sources; evaluate strategies for recruitment into the field; and develop a plan to restructure D/HH education.

Outcomes:

    Timely information to aid in decision making throughout the project for all seven goals, with specific reports on numbers participating, reasons for nonparticipation, quality of syllabi and case studies, sensitivity of recruitment strategies for under-served populations, future needs for faculty development, and production of a plan to restructure D/HH teacher education programs in the U.S.

Note:  See "Evaluation Plan" for details

Individual Responsible:

    Dr. D. Mertens.

Time Line:

    Throughout the project, monthly progress reports; milestone reports as needed.

Budget:

    Personnel:

        Project Evaluator (PE) - Donna Mertens

uploaded by Harold Johnson