Daniel M. Dunlavy

[Home]  [Contact]  [CV]  [Research]  [Education]  [Awards]  [Publications]  [Presentations]  [Service]  [QCS] 

QCS - Query, Cluster, Summarize

Abstract

Conducting scientific research most often involves a search through existing literature in order to avoid repeating research efforts, review methods already developed for solving a problem, gain a better understanding of a problem, etc. Typically, this search is performed using the Internet, which is a convenient portal to various databases of books, journal articles, technical reports, preprints, etc.

The Query, Cluster, Summarize (QCS) information retrieval system is presented in an attempt to improve efficiency in these literature searches. Given a query, QCS retrieve documents relevant to the query, separates the retrieved documents into topic clusters, and creates a single summary for each of topic clusters. Latent Semantic Indexing is used retrieval, generalized spherical k-means (gmeans) is used for the document clustering, and a hidden Markov model coupled with a pivoted QR decomposition is used to create a single extract summary for each topic cluster.

This project was developed for Advanced Scientific Computation (AMSC 663-4), the graduate qualifying sequence for the Scientific Computation Concentration of the Applied Mathematics and Scientific Computation (AMSC) Program  at the University of Maryland, College Park.

Advisor: Dianne O'Leary
Course Instructors: Dave Levermore, Bill Dorland

Software

• QCS v1.0
• Screenshots:   1   2  

Publications

• Daniel M. Dunlavy, John M. Conroy, Timothy J. O'Leary and Dianne P. O'Leary
  Clustering and Summarizing Medline Abstracts
  Proceedings of Digital Biology: The Emerging Paradigm
  National Institutes of Health, Bethesda, MD, November 2003.
  
  
• Daniel M. Dunlavy, John P. Conroy, and Dianne P. O'Leary
  QCS: A Tool for Querying, Clustering, and Summarizing Documents
  Proceedings of the HLT-NAACL Conference
  Edmonton, Canada, May 2003.
  

Technical Reports

• Final Report (05/16/2003);   Slides (04/29/03)
  
• Progress Report #1 (12/18/2002);   Slides (12/16/02)
  
• Project Proposal (09/30/02);   Slides (09/30/02)
  

Packages/Tools Used in QCS

• Q: Querying
  • Reference: Using Linear Algebra for Intelligent Information Retrieval
    Authors: Michael Berry, Susan Dumais, and Gavin W. O'Brien
  • Code: General Text Parser (GTP)
    Authors: Michael Berry, S. Howard, H. Tang, D. Martin, and J. Giles
• C: Clustering
  • Reference: Concept Decompositions for Large Sparse Text Data using Clustering
    Authors: Inderjit S. Dhillon and Dharmendra S. Modha
  • Code: Generalized Spherical K-Means (gmeans)
    Author: Yuqiang Guan
• S: Summarizing
  • Reference: Performance of a Three-Stage System for Multi-Document Summarization
    Authors: D.M. Dunlavy, J.M. Conroy, J.D. Schlesinger, S.A. Goodman, M.E. Okurowski, D.P. O'Leary, and H. van Halteren
  • Code: HMM+QR M-files converted to C using the Mathworks Matlab Compiler
    Authors: John Conroy, Dianne O'Leary, and Daniel Dunlavy


• Document Preprocessing
  • Perl wrapper around LT TTT v1.0
  • HTML-Parser
• Servers
  • Apache HTTP Server (httpd)
  • Apache Jakarta Java Servlet Container (tomcat)
• Parallel Code Development
  • MPICH
• Code Maintenance
  • CVS
• Data Storage/Manipulation
  • Harwell-Boeing Matrix I/O

Comments

Comments and suggestions on anything related to QCS or this information page are certainly welcome. You can email me at ddunlavy@cs.umd.edu. Thanks in advance.