CMSC 691B Discussion Questions
Learning and Order

Cohen et al., "Learning to order things"

• What are the main claims of the paper? Does the paper present theoretical and/or empirical evidence for these claims?
• Does the paper position itself thoroughly and accurately with respect to related work?
• Specific technical questions:
• What are the two stages in their "two-stage approach"?
• What are the advantages and disadvantages in separating the problem into these two stages? Is one of the stages more thoroughly worked out/addressed in this paper?
• What is an online learning algorithm? Why do they use one for this problem?
• What are the ranking experts that are used in the first phase? How would you develop a set of ranking experts for a new problem? What ranking experts do they use for their results?
• How does the Hedge algorithm work? What is a loss function, and how is the loss function used by this algorithm defined? Do yout hink this is a good loss function to use?
• What are the differences between the alternative approximately-optimal-ordering algorithms?
• What's being evaluated by the experiments presented in Section 5? Are these results convincing? Why or why not?
• What are the key results of the paper?
• What do you think are the most important/interesting next steps/future work for this paper?
• What  references would you follow up on if you were to do further reading on this topic?
• What were the good and bad aspects of the presentation (organization, writing quality, clarity) of this paper? Is the paper self-contained, or would you need to do further background reading (or correspond with the authors ) to understand the details of the methods and results?

Kamishima and Akaho, "Learning from order examples"

What are the main claims of the paper? Does the paper present theoretical and/or empirical evidence for these claims?

Does the paper position itself thoroughly and accurately with respect to related work?

Specific technical questions:

What does this paper use as the loss function for measuring the effectiveness of the algorithm? Do you think this is a good loss function to use? How does it differ from the loss function used by Cohen et al.?

What is the difference between a "classification-based" and a "regression-based" approach for learning ordering functions?

What is the "Bayesian estimator with Dirichlet prior" (p. 646) used for?

What is the difference between the SumClass and ProductClass sorting strategies?  How woudl one decide which to use?

What is the "Type I quantification method" used for the regression approach?

What is being evaluated by the experiments in Section 5? Are these results convincing? Why or why not? Are the conclusions drawn by the authors supported by the results?

What are the key results of the paper?

What do you think are the most important/interesting next steps/future work for this paper?

What  references would you follow up on if you were to do further reading on this topic?

What were the good and bad aspects of the presentation (organization, writing quality, clarity) of this paper? Is the paper self-contained, or would you need to do further background reading (or correspond with the authors ) to understand the details of the methods and results?

Comparing the papers

• One of these papers is a conference paper; the other is a journal paper. How are the differences between these types of publications apparent?
• The Kamishima and Akaho paper cites the Cohen et al. paper. Could this be seen as an extension of that work? Is their analysis of the earlier paper, and their positioning of their work relative to the earlier work, correct/useful?
• Cohen et al. give several reasons not to use item-set orderings as input to the learning process. What are these reasons? Conversely, Kamishima and Akaho indicate that their use of item-set orderings as input is one of the key differences/advantages of their approach. Is the latter claim supported by the paper? Do you find one of these approaches more convincing?
• Does the later paper represent a significant improvement or extension over earlier work?