Icon: Screen-friendly stylesheet Screen-friendly page

Participate in Our Programs :: Brown Bag Series

Evaluating the IClicker

Cheryl Bullock and Laura Hahn (Center for Teaching Excellence)

In most classrooms, technology is still only providing passive tools (tools that allow instructors to display information) and not tools that are inherently interactive. New wireless and digital technologies, however, are beginning to provide instructors with feedback tools to create active learning environments outside of specialized classrooms or computer labs. Even in large lecture halls, instructors are now able to take advantage of different digital technologies to garner feedback and interaction from their students. One technology developed at the University of Illinois to provide this interaction is the IClicker. With a device similar to a TV remote control, students are able to answer questions, surveys, and polls during a class session that an instructor can access and display in real time. The IClicker is undergoing a beta testing in classes across disciplines on the Illinois campus.

Cheryl Bullock and Laura Hahn, both from the Center for Teaching Excellence (CTE), conducted a comprehensive study of the first semester of beta testing. The study included interviews with over 1000 students and six faculty, as well as a quasi-experiment conducted in a large course with a section that used IClickers and a control section that did not. At their presentation Professors Bullock and Hahn discussed some of their preliminary results and provided best practices for the IClicker.

Presentation Summary

The talk began with Professor Hahn describing how the IClicker works. The IClicker system has an instructor module and individual devices that are given to students. The instructor module is a small receiver that connects to a PC. Software on the PC allows the instructor to modify the system's settings as well as to view and record student results. The student device looks like a very simple TV remote control. It has only five buttons labeled A, B, C, D, and E respectively and a power switch. To use it, the students press one of the five buttons in response to a multiple choice-type question. From the receiver, the instructor gets a record in real time of how many students answered a particular letter. While the device is simple, instructors in the beta test used the technology in a wide variety of ways - some posed multiple choice and true false questions during lectures, some used them to garner results from group work, some used them to take attendance, some used them for quizzes, and one instructor even used it as a gauge to see when students were confused.

After the introduction to the IClicker technology, Professor Bullock discussed the CTE's evaluation of its impact in the classroom. The IClicker evaluation occurred over the Summer and Fall of 2004. The evaluation included a literature review, eight pre-semester faculty interviews, surveying 950 students, direct course observations, and six post-semester faculty interviews. The variety of evaluation methods was paralleled by the variety of courses - they evaluated courses using the IClicker from Crop Sciences, Business Administration, Physics, Geology, Food Sciences, and ACE. The evaluation attempted to gauge the impact on student learning as well as faculty perceptions of the value of the system.

The preliminary analysis of the CTE evaluation indicates that the IClicker improved student learning outcomes in participating classes. In the surveys to classes where all students were using the IClicker, the students reported that they felt they were learning more, coming to class better prepared, and participating more compared to other classes they had taken at the U of I that did not use the IClicker. Students also reported that they felt better prepared for tests and scored better due to the IClicker. For the most part, negative assessments of the IClicker were over both real and perceived technical shortcomings. For example, many students were frustrated with the battery life in the IClicker (an issue that was corrected during the semester). There were also some unfounded worries that the results might be unreliable. There were no reports that students felt the IClicker was interfering with the course or their learning.

The results from the quasi-experiment (where a section using an IClicker was compared to another in the same course that was not), while indicating less encouraging outcomes, were still positive about the system overall. In this experiment, similar questions about student perceptions of learning were asked to the sections that used the IClicker and those that did not. Both types of sections felt that the class offered a superior learning environment compared to other classes they had taken at the U of I. However, a statistically significant greater number of students felt that the IClicker class encouraged them to participate more than other classes.

The preliminary results for instructors using the system were generally as positive as those for students. Instructors found the system generally easy to use and implement. Also, they fit quite well with existing course structures and most instructors did not need to do significant redesigns of their material to use the IClicker. Pedagogically, instructors reported that by using the IClicker, they were better able to judge if students were comprehending the material. They also found it a very useful tool for collaborative and group work. The only area where the IClicker was not seen as successful was as a tool to keep attendance.

While the preliminary results demonstrate that the IClicker is a valuable tool in the classroom, the analysis still needs to be completed to see where the most promising areas are to develop IClicker lessons and what pedagogical strategies and techniques are best suited to the tool. To that end, the CTE team will complete an extensive qualitative analysis of the system, look again at the technical improvements to the system, and conduct more interviews with faculty.

About the Presenters

Cheryl Bullock is an Evaluation Specialist at the Center for Teaching Excellence and has appointments in the departments of Educational Psychology and Educational Organization and Leadership. Laura Hahn is an Education Specialist at the Center for Teaching Excellence and holds an appointment in the department of English as an International Language.

*This PowerPoint has been converted to an accessible web document using the Illinois Accessible Web Publishing Wizard for Microsoft Office.