We have relevant datasets, repositories, frameworks and tools of relevance for research and technology transfer initiatives related to knowledge extraction. This section provides an overview on a selection of them and links to download or contact details.

The MdM Strategic Research Program has its own community in Zenodo for material available in this repository  as well as at the UPF e-repository  . Below a non-exhaustive list of datasets representative of the research in the Department.

As part of the promotion of the availability of resources, the creation of specific communities in Zenodo has also been promoted, at level of research communities (for instance, MIR and Educational Data Analytics) or MSc programs (for instance, the Master in Sound and Music Computing)



Back [TEXT] The Online Conversation Threads Repository (Slashdot, Barrapunto, Wikipedia talk)

The Online Conversations Threads Repository. V. Gómez, A. Kaltenbrunner and D. Laniado



This repository contains datasets with online conversation threads collected and analyzed by different researchers. Currently, you can find datsets from different news aggregators (Slashdot, Barrapunto) and the English Wikipedia talk pages. Slashdot conversations (Aug 2005 - Aug 2006) Online conversations generated at Slashdot during a year. Posts and comments published between August 26th, 2005 and August 31th, 2006. For each discussion thread: sub-domains, title, topics and hierarchical relations between comments. For each comment: user, date, score and textual content. This dataset is different from the Slashdot Zoo social network (it is not a signed network of users) contained in the SNAP repository and represents the full version of the dataset used in the CAW 2.0 - Content Analysis for the WEB 2.0 workshop for the WWW 2009 conference that can be found in several repositories such as Konect Barrapunto conversations (Jan 2005 - Dec 2008) Online conversations generated at Barrapunto (Spanish clone of Slashdot) during three years. For each discussion thread: sub-domains, title, topics and hierarchical relations between comments. For each comment: user, date, score and textual content Wikipedia (2001 - Mar 2010) Data from articles discussions (talk) pages of the English Wikipedia as of March 2010. It contains comments on about 870,000 articles (i.e. all articles which had a corresponding talk page with at least one comment), in total about 9.4 million comments. The oldest comments date back to as early as 2001.


When using these data, please use the following references.

The datasets were first analyzed in:

[1] V. Gómez, A. Kaltenbrunner, V. López (2008). Statistical analysis of the social network and discussion threads in Slashdot. Proceedings of the 17th International World Wide Web Conference

[2] V. Gómez, H. J. Kappen, N. Litvak, A. Kaltenbrunner (2013). A likelihood-based framework for the analysis of discussion threads. World Wide Web 16(5-6):645-675 (arXiv version here)

[3] D. Laniado, R. Tasso, Y. Volkovich, A. Kaltenbrunner (2011). When the Wikipedians Talk: Network and Tree Structure of Wikipedia Discussion Pages. Fifth International AAAI Conference on Weblogs and Social Media