Objective

This project explores the Topical Multi-Document Summarization problem and tries to suggest a new summarizing algorithm for it. Topical Multi-Document Summarization is, given an initiated topic, generating a comprehensive summary from a large collection of documents based on the topic-related content of part of or all the documents.

Reports

1. Project proposal: html, pdf
2. Final paper: pdf

Texts for Experiment

Total 60 documents are used for the experiment in this project. There are 30 selected Brown documents, the rest are articles about the debate of profit or non-profit publishing and articles about car survey.

Tools

• MC - a toolkit for creating vector models from text cocuments
  Developed by Univ of Texas at Austin
  Available at http://www.cs.utexas.edu/users/jfan/dm/index.html
 
• "spkmeans" - a documents clustering tool based on spherical K-means clustering algorithm
  Developed by Univ of Texas at Austin
  Available at http://www.cs.utexas.edu/users/yguan/datamining/spkmeans.html
  
• Cluster Browser - a Java program to illustrate clustering results
  Developed by Univ of Texas at Austin
  Available at  http://www.cs.utexas.edu/users/yguan/datamining/clusterBrowser.html

• GetRank - a C++ file that will take the query string as input, generating the ranks for each documents and each clusters.
• Modified Cluster Browser - the modified version is to meet the need to integrate the rank information in the final web pages, and it also generate a new web interface to show the reduced clusters and documents which is used directly for summarization.
• c - a UNIX shell script to integrate all the related programs in a batch.

Experiment Result

   1. Web page for clustering result

          snapshot

  2. Web page for reduced clusters and documents used for summarizing

          snapshot

   Detailed explanation will be found at the final paper.

Reference

1. Gees Stein, Amit Bagga and G. Wise. 2000. Multi-Document Summarization: Methodologies and Evaluations. Conference TALN 2000.

2. www.summarization.com

3. www.cs.columbia.edu/nlp/projects.html

4. www.cs.columbia.edu/~hjing/summarization.htm

5. Regina Barzilay, Kathleen R. McKeown, Michael Elhadad. 1999. Information Fusion in the Context of Multi-Document Summarization.

6. Carbonell, Jame and Jade Goldstein. 1998. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. SIGIR ’98.

7. Inderjit Dhillon and James Fan, 2001. Efficient Clustering of Very Large Document Collection.

8. www.cs.utexas.edu/users/dml/

9. www.site.uottawa.ca/tanka/ts.html

10. transend.labs.bt.com

11. CCS format, www.netlib.org/linalg/html_templates/node92.html

12. K-means clustering, www.engr.sjsu.edu/~knapp/HCIRDFSC/C/k_means.htm