Ian Soboroff

PLEASE NOTE, As of June 2001, I have joined the Retrieval Group at the National Institute of Standards and Technology (NIST). I am no longer at UMBC full-time, although I do teach a course now and then. If you are interested in doing information storage and retrieval work at UMBC, contact Charles Nicholas.

My current research interests are

Information retrieval
Measuring the effectiveness of retrieval and other language technologies
Information filtering
User interfaces

I helped to organize a Workshop on Recommender Systems at SIGIR'99 in Berkeley, CA. The workshop papers are archived at that site. Additionally, a summary of the workshop appeared in SIGIR Forum.

I have a blog which links to some random software I've written for the Palm.

I have another blog which I actually update now and then.

Teaching

Past courses

Fall 2002: CMSC 491I/691I, Information Retrieval
Spring 2001: CMSC 491I/691I, Information Retrieval
Spring 1997 (TA): CMSC 437, Graphical User Interface Programming
Fall 1996 (TA): CMSC 437/691C, Graphical User Interface Programming

Publications

(see the TREC Proceedings for TREC track reports I've been involved with:)

Web	2009
Blog	2009	2008	2007	2006
Enterprise	2008	2007	2006	2005
Terabyte	2006	2005	2004
Novelty	2004	2003
Filtering	2002	2001

Ian Soboroff, Dean McCullough, Jimmy Lin, Craig Macdonald, Iadh Ounis, Richard McCreadie (2012): On Building a Reusable Twitter Corpus
Proceedings of the 35th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2012), Portland, OR.
Rodrygo L. T. Santos, Craig Macdonald, Richard McCreadie, Iadh Ounis, Ian Soboroff (2012): Information Retrieval in the Blogosphere
Foundations and Trends in Information Retrieval 6(1)
Ian Soboroff, Dean McCullough, Jimmy Lin, Craig Macdonald, Iadh Ounis, Richard McCreadie: Evaluating Real-Time Search over Tweets
Proceedings of the 2012 AAAI International Conference on Weblogs and Social Media (ICWSM 2012)
Charles Clarke, Nick Craswell, Ian Soboroff, Azin Ashkan (2011): A comparative analysis of cascade measures for novelty and diversity
Proceedings of the Fourth Conference on Web Search and Data Mining (WSDM 2012)
Craig Macdonald, Rodrygo Santos, Iadh Ounis, Ian Soboroff (2010): Blog track research at TREC
ACM SIGIR Forum 44(1), pp. 58-75
Ben Carterette and Ian Soboroff (July 2010): The effect of assessor error on IR system evaluation
Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2010), Boston, MA.
Ian Soboroff (2010): Test collection diagnosis and treatment
Proceedings of the 2010 Workshop on Evaluation of Information Access (EVIA 2010), Tokyo, Japan.
Ian Soboroff (2010): A guide to the RIA workshop data archive
Information Retrieval 12(6), pp. 642-651
Craid Macdonald, Iadh Ounis, and Ian Soboroff (July 2009): Is spam an issue for opinionated blog post search?
Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2009), Boston, MA.
Craig Macdonald, Ben He, Iadh Ounis and Ian Soboroff (July 2008): Limits of opinion-finding baseline systems
Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2008), Singapore.
Peter Bailey, Nick Craswell, Ian Soboroff, Paul Thomas, Arjen P. de Vries, and Emine Yilmaz (July 2008): Relevance assessment: are judges exchangeable and does it matter?
Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2008), Singapore.
Iadh Ounis, Craig Macdonald, and Ian Soboroff (April 2008): On the TREC Blog Track Proceedings of the International Conference on Weblogs and Social Media (ICWSM 2008), Seattle, WA.
Stefan Büttcher, Charles L. A. Clarke, Peter C. K. Yeung, and Ian Soboroff (July 2007): Reliable Information Retrieval Evaluation with Incomplete and Biased Judgments
Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), Amsterdam, the Netherlands.
Ian Soboroff (July 2007): A Comparison of Pooled and Sampled Relevance Judgments
Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), Amsterdam, the Netherlands.
Mark Sanderson and Ian Soboroff (July 2007): Problems with Kendall's Tau
Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), Amsterdam, the Netherlands.
Chris Buckley, Darrin Dimmick, Ian Soboroff, and Ellen Voorhees (2007): Bias and the Limits of Pooling for Large Collections
Information Retrieval, vol. 10, no. 6, pp. 491-508.
Peter Bailey, Nick Craswell, Ian Soboroff, and Arjen P. de Vries (December 2007): The CSIRO Enterprise Search Test Collection
SIGIR Forum, vol. 41, no. 2, pp. 42-45.
Ian Soboroff (August 2006): Dynamic Test Collections: Measuring Search Effectiveness on the Live Web
Proceedings of the 29th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2006), Seattle, WA.
Chris Buckley, Darrin Dimmick, Ian Soboroff, and Ellen Voorhees (August 2006): Bias and the Limits of Pooling (Poster)
Proceedings of the 29th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2006), Seattle, WA.
Ian Soboroff (May 2006): A Comparison of Pooled and Sampled Relevance Judgments in the TREC 2006 Terabyte Track (Invited Paper)
Proceedings of the First International Workshop on Evaluating Information Access (EVIA 2007), Tokyo, Japan.
Ian Soboroff and Donna Harman (October 2005): Novelty Detection: The TREC Experience
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP 2005), Vancouver, British Columbia, Canada.
Ian Soboroff (July 2004): On Evaluating Web Search With Very Few Relevant Documents (Poster)
Proceedings of the 27th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2004), Sheffield, UK.
Ian Soboroff, Ellen Voorhees, and Nick Craswell (September 2003): Summary of the SIGIR 2003 workshop on defining evaluation methodologies for terabyte-scale test collections
SIGIR Forum, vol. 37, no. 2 (Fall 2003)
Ian Soboroff and Stephen Robertson (July 2003): Building a Filtering Test Collection for TREC 2002
Proceedings of the 26th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2003), Toronto, Ontario, Canada.
Ian Soboroff (August 2002): Does WT10g Look Like the Web? (Poster)
Proceedings of the 25th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2002), Tampere, Finland.
An extended version which includes an analysis of the Gov18g collection appeared in SIGIR Forum, vol. 36, no. 2
Abdur Chowdhury and Ian Soboroff (August 2002): Automatic Evaluation of World Wide Web Search Services (Poster)
Proceedings of the 25th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2002), Tampere, Finland.
Ian Soboroff and Charles Nicholas (2002): Related, but not Relevant: Content-based Collaborative Filtering in TREC-8
Information Retrieval, vol. 5, nos. 2/3, April-July 2002, pp. 189-208.
Ian Soboroff, Charles Nicholas, and Patrick Cahan (September 2001): Ranking Retrieval Systems without Relevance Judgments
Proceedings of the 24th Annual International Conference on Research and Development in Information Retrieval (SIGIR 2001), New Orleans, LA.
Ian Soboroff and Charles Nicholas (August 2000): Collaborative Filtering and the Generalized Vector Space Model (Poster)
Proceedings of the 23rd Annual International Conference on Research and Development in Information Retrieval (SIGIR 2000), Athens, Greece.
Douglas W. Oard, Jianqiang Wang, Dekang Lin, and Ian Soboroff (November 1999): TREC-8 Experiments at Maryland: CLIR, QA, and Routing
Working notes of the Eighth Text Retrieval Conference, Gaithersburg MD; also presented as a poster.
Ian Soboroff and Charles Nicholas (August 1999): Combining Content and Collaboration in Text Filtering
Proceedings of the IJCAI'99 Workshop on Machine Learning for Information Filtering, Stockholm, Sweden.
Christopher D. Shaw, James M. Kukla, Ian Soboroff, David S. Ebert, Charles K. Nicholas, Amen Zwa, Ethan L. Miller, and D. Aaron Roberts. (1999): Interactive Volumetric Information Visualization for Document Corpus Management
International Journal on Digital Libraries, vol. 2, issue 2/3, pp. 144-156.
Ian M. Soboroff (May 1998): Collaborative Filtering with LSI: Experiments with Cranfield
UMBC CSEE Technical Report CS-TR-98-01
R. Scott Cost, Tim Finin, Yannis Labrou, Xiaocheng Luan, Yun Peng, Ian Soboroff, James Mayfield, and Akram Boughannam (July 1998): Jackal: a Java-based Tool for Agent Development
Working Papers of the AAAI-98 Workshop on Software Tools for Developing Agents, Madison, WI.
Ian M. Soboroff, Charles K. Nicholas, James M. Kukla, David S. Ebert. (November 1997): Visualizing Document Authorship Using N-grams and Latent Semantic Indexing [Gzipped Postscript w/ b&w images] [First color image] [Second color image]
Proceedings of the Workshop on New Paradigms in Information Visualization and Manipulation (NPIVM '97), Las Vegas, NV. ACM Press, 1998.
David S. Ebert, James M. Kukla, Christopher D. Shaw, Amen Zwa, Ian Soboroff, and D. Aaron Roberts. (October 1997): Automatic Shape Interpolation for Glyph-based Information Visualization
IEEE Visualization '97, Late Breaking Hot Topics, Phoenix, AZ.
R. Scott Cost, Ian Soboroff, Jeegar Lakhani, Tim Finin, Ethan Miller, Charles Nicholas (July 1997): TKQML: A Scripting Tool for Building Agents [Gzipped Postscript]
Proceedings of the 1997 Conference on Agent Theories, Architectures, and Langauges (ATAL '97), Providence, RI.
Published as Intelligent Agents IV, Munindar P. Singh, Anand S. Rao, and Michael J. Woolridge, Eds., Lecture Notes in Artificial Intelligence Vol. 1365. (Springer-Verlag, Feb 1998).
An extended version is available as UMBC CSEE Technical Report CS TR-97-04.
R. Scott Cost, Ian Soboroff, Jeegar Lakhani, Tim Finin, Ethan Miller, Charles Nicholas (July 1997): Agent Development Support for Tcl [Gzipped Postscript]
Poster and extended abstract appearing in Processings of the Fifth Tcl/Tk Workshop, Boston, MA.
Russell Turner, Enrico Gobbetti, Ian Soboroff (1996): Head-Tracked Stereo Viewing with Two-Handed 3D Interaction for Animated Character Construction. [Paper: 2.2MB] [Abstract]
Computer Graphics Forum 15(3), Blackwell.
Special Issue on Proceedings EUROGRAPHICS Conference, Poitier, France.