INDIE meets students: Rembrandt

So how does one determine the overall quality of an INDIE GBS? Rating an individual GBS might include:

• Appeal to students. Students need to find the GBS engaging and
  interesting--they should like to use it and want to use it.

• Teach students something. Students should feel they learned something.

• Be as good or better than traditional instruction in the subject matter. At the very least, students should prefer using an INDIE GBS to attending a lecture.

With Rembrandt's deployment in an art history class, Rembrandt authors had students optionally fill out questionnaires on INDIE. They received 130 responses. The relevant results to INDIE (as opposed to the design of Rembrandt in particular) are summarized in Table 6.3.

 

Table 6.3: Survey results from Is it a Rembrandt? Authors received roughly 130 responses to each question.

Response	%
Is it a Rembrandt? was fun to use.
Strongly disagree	3
Disagree	5
Agree somewhat	38
Agree	35
Strongly agree	10
Is it a Rembrandt? provides a good introduction to the issues of attribution and authenticity.
Strongly disagree	0
Disagree	2
Agree somewhat	20
Agree	44
Strongly agree	35
Is it a Rembrandt? provides a good introduction to the art of Rembrandt.
Strongly disagree	0
Disagree	12
Agree somewhat	21
Agree	45
Strongly agree	21
Compared to a class lecture on the same subject area, how effective a learning tool was Is it a Rembrandt?
Much less effective	4
Less effective	21
As effective	28
More effective	30
Much more effective	17
If you didn't work on all 3 paintings, please indicate why:
I didn't have enough time	72
I was bored	13
I kept having technical problems	11
Other	4

Both from these results and the rest of the survey results, it appears that students generally liked it, had fun, found they learned something, and preferred it over a lecture. Many of the complaints in the written section centered around unfamiliarity with GBSes, such as ``Somehow get rid of the videos, especially Culture Beat [the wrap-up video]. A waste of space.'' Others had suggestions for better interface, such as ``Have a `map' of where you've been and where you're going.''

A fair number of the negative comments suggest that students found problems that were unrelated to the architecture. Students had to use INDIE in a shared laboratory that was crowded, busy, and inconvenient. Also, the machines that Rembrandt ran on were slow and many had technical problems like spotty sound and broken mice. The INDIE software, too, could have used more robust testing in combination with the networked software it was using. Just the fact that 72% of students would have liked to have more time to use it implies that we were reaching a large portion of our audience but they were having problems getting a chance to use it.

There are obviously more rigorous ways we could analyze the results, most of which would use a control group hearing a lecture and and experiment groups running different variations on the GBS. Unfortunately, such an investigation would be difficult and results would be difficult to separate from background noise; students are only using Rembrandt for a few hours, and it is hard to show large differences in behavior or learning with so little intervention. However, from this pilot test we can see that INDIE GBSes have good potential for creating fun, motivating, and educational projects.