next up previous contents
Next: Financial Report Analyst Up: Advise Previous: INDIE and Advise: Focus

INDIE and Advise: Critiquing

Advise critiques student arguments in two ways: general critiques and custom critiquers. General critiques are built-in critiquing that the engine provides. For most of the claims, each advisor has their own rating of the claim and a list of evidence they see as vital to consider. Thus, advisors react when the claim a student has made is different than theirs and/or they have not considered (i.e., placed in either BECAUSE or DESPITE) their favored evidence. Their responses are customizable, but when they fire isn't.

Custom critiques are structured in a similar way to INDIE's rules--they have conditions and subconditions, and these conditions can be composed with AND and OR to make any sort of arbitrary logic authors need. The overall report critiquer is also a list of conditions.

Earlier versions of the Advise tool were almost entirely dependent on general critiques for effective critiquing, and the report critiquer rules were hard-coded and identical from project to project. Authors spent a lot of time writing effective introductions and preparing the ASK system and much less time focused on providing complete and robust critiquing. This lead to students having repetitive and frustrating interaction with advisors.

Advise developers also over-represented their critiquing domain in earlier versions of the tool, as did INDIE 1.0. They were working from an abstract model of argumentation provided by [Toulmin 1958] that provided difficult and arcane categories for custom critiques. The new rule-like critiquer has shown more promise to be closer to authors' needs.

Still, Advise critiquing and INDIE critiquing are not too far from one another, partially because they share the same task: they find something to say about an unordered list of points students have assembled in support or rejection of a claim. The general critiques still exist in Advise, however, partially because of the exponential increase in the number of possible responses. For example, a typical Advise system with four advisors with four plans and four goals leads to 64 possible Advisor opinions. Each Advisor doesn't always have an opinion on each plan/goal combination, but nonetheless this leads to a lot of critiquing which needs the leverage of generalized critiquing rules. INDIE systems tend to have only one advisor (although ASK system sizes and numbers of evidence points tend to be comparable).

Overall, evolution of the two systems has converged on very general functionality for critiquing, which appears to affirm the proposition suggested earlier: authors don't want strong representation if it gets in their way. It will be interesting to see in the future if Advise authors trying to ship complete systems run into the same problems our authors did with their rule-like critiquer.


next up previous contents
Next: Financial Report Analyst Up: Advise Previous: INDIE and Advise: Focus
Wolff Dobson
1998-07-28