×

Modeling user reputation in wikis. (English) Zbl 07260237

Summary: Collaborative systems available on the Web allow millions of users to share information through a growing collection of tools and platforms such as wikis, blogs, and shared forums. By their very nature, these systems contain resources and information with different quality levels. The open nature of these systems, however, makes it difficult for users to determine the quality of the available information and the reputation of its providers. Here, we first parse and mine the entire English Wikipedia history pages in order to extract detailed user edit patterns and statistics. We then use these patterns and statistics to derive three computational models of a user’s reputation. Finally, we validate these models using ground-truth Wikipedia data associated with vandals and administrators. When used as a classifier, the best model produces an area under the receiver operating characteristic (ROC) curve (AUC) of 0.98. Furthermore, we assess the reputation predictions generated by the models on other users, and show that all three models can be used efficiently for predicting user behavior in Wikipedia.

MSC:

62-XX Statistics
68-XX Computer science
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] T. Oreilly. What is Web 2.0: Design patterns and business models for the next generation of software, Communications Strategies, 2007.
[2] B. Alexander. Web 2.0: A new wave of innovation for teaching and learning? Educause Review 41(2) (2006), 32-44.
[3] M. Boulos, I. Maramba, and S. Wheeler. Wikis, blogs and podcasts: a new generation of web-based tools for virtual collaborative clinical practice and education, BMC Medical Education 6 (2006), 41.
[4] B. Leuf and W. Cunningham. The Wiki Way: Quick Collaboration on the Web, Addison-Wesley, 2001.
[5] O. Arazy and E. Stroulia. A utility forestimating the relative contributions of wiki authors, In International AAAI Conference on Weblogs and Social Media, 2009. [Online]. Available:http://www.aaai.org/ocs/index.php/ICWSM/09/paper/ view/157 [Last Accessed December 2009].
[6] R. Raitman and N. Augar. Employing wikis for online collaboration in the e-learning environment: Case study, Proceedings of the Third International Conference on Information Technology and Applications (ICITA’05). Washington, DC, IEEE Computer Society, 2005, 142-146.
[7] E. Elrufaie and D. A. Turner. A wiki paradigm for use in it courses, ITCC ’05: Proceedings of the International ConferenceonInformationTechnology:Codingand Computing (ITCC’05),Vol. 2, Washington, DC, IEEE Computer Society, 2005, 770-771.
[8] Q. Li. Knowledge building community: Keys for using online forums prediction, TechTrends 48 (2007), 24-29.
[9] J. Giles. Internet encyclopaedias go head to head, Nature 438 (2005), 900-901.
[10] J. Seigenthaler. A False Wikipedia ‘Biography’. 2005. [Online].Available:http://www.usatoday.com/news/ opinion/editorials/2005-11-29-wikipedia-edit x.htm[Last Accessed December 2009].
[11] B. Shneiderman. Designing trust into online experiences, Communications of ACM 43(12) (2000), 57-59.
[12] P. Resnick and R. Zeckhauser. Trust among strangers in Internet transactions: Empirical analysis of eBay’s reputation system, In The Economics of the Internet and E-Commerce, Vol. 11, Series in Advances in Applied Microeconomics, M. R. Baye, ed. Amsterdam, 127-157 Elsevier Science, 2002.
[13] A. Josan, R. Ismail, and C. Boyd. A survey of trust and reputation systems for online service provision, Decis. Support Syst. 43(2) (2007), 618-644.
[14] B. Hoisl, W. Aigner, and S. Miksch. Social rewarding in wiki systems - motivating the community, Proceedings of HCI International - 12th International Conference on HumanComputer Interaction (HCII 2007), vol. 4564, Lecture Notes in Computer Science. Berlin/Heidelberg Springer, 2007, 362-371.
[15] Reputation. [Online]. Available: http://en.wikipedia.org/ wiki/Reputation [Last Accessed December 2009].
[16] C. L. Corritore, B. Kracher, and S. Wiedenbeck. On-line trust: concepts, evolving themes, a model, International Journal Human-Computer Studies 58(6) (2003), 737-758.
[17] Knowledge smackdown: Wikipedia vs. citizendium. [Online]. Available: http://www.storysouth.com/comment /2006/ 09/knowledge smackdown wikipedia.html [Last Accessed December 2009].
[18] Alexa’s top 10 websites. [Online]. Available: http://www. alexa.com/ [Last Accessed December 2009].
[19] O. Nov. What motivates wikipedians? Commun. ACM 50(11) (2007), 60-64.
[20] A. H. Maslow. Motivation and Personality. New York, HarperCollins Publishers, 1987.
[21] D. M. Wilkinson and B. A. Huberman. Cooperation and quality in wikipedia, WikiSym ’07: Proceedings of the 2007 International Symposium on Wikis. New York, ACM, 2007, 157-164.
[22] Y. Ganjisaffar, S. Javanmardi, and C. Lopes. Reviewbased ranking of wikipedia articles, In Proceedings of the International Conference on Computational Aspects of Social Networks, Crystal City, Washington DC, 2009.
[23] D. Anthony, S. W. Smith, and T. Williamson. Explaining Quality in Internet Collective Goods: Zealots and Good Samaritans in the Case of Wikipedia, Hanover, Dartmouth College, Technical Report, 2005. [Online]. Available: web.mit.edu/iandeseminar/Papers/Fall2005/anthony.pdf [Last Accessed December 2009].
[24] J.Voss.Measuringwikipedia,Proceedingsof10th International Conference of the International Society for Scientometrics and Informetrics. Stockholm, Sweden, 2005.
[25] F. B. Viegas, M. Wattenberg, and K. Dave. Studying cooperation and conflict between authors with history flow visualizations, CHI ’04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, ACM, 2004, 575-582.
[26] A. Josang, C. Keser, and T. Dimitrakos. Can we manage trust? In Proceedings of the Third International Conference on Trust Management (iTrust’05), Rocquencourt, France, 2005.
[27] S. Ketchpel and H. Garcia-molina. Making trust explicit in distributed commerce transactions, In Proceedings of the International Conference on Distributed Computing Systems, 1996, 270-281.
[28] Y. Atif. Building trust in e-commerce, IEEE Internet Computing 6(1) (2002), 18-24.
[29] L. Xiong and L. Liu. A reputation-based trust model for peerto-peer ecommerce communities [extended abstract], EC ’03: Proceedings of the 4th ACM Conference on Electronic Commerce. ACM, 2003, 228-229.
[30] A. Gutscher. A trust model for an open, decentralized reputation system, Trust Management 238 (2007), 285-300.
[31] eBay Editorial. Evaluating a Member’s Reputation. 2005. [Online]. Available: http://pages.ebay.com/help/feedback/ evaluating-feedback.html [Last Accessed December 2009].
[32] A. C. Squicciarini, F. Paci, and E. Bertino. Trust establishment in the formation of virtual organizations, In ICDE Workshops, 2008, 454-461.
[33] R. Aringhieri, E. Damiani, S. D. C. D. Vimercati, S. Paraboschi, and P. Samarati. Fuzzy techniques for trust and reputation management in anonymous peer-to-peer systems: Special topic section on soft approaches to information retrieval and information access on the web, Journal of the American Society for Information Science and Technology 57(4) (2006), 528-537.
[34] C.-N. Ziegler and J. Golbeck. Investigating interactions of trust and interest similarity, Decis. Support Syst. 43(2) (2007), 460-475.
[35] H. Liu, E.-P. Lim, H. W. Lauw, M.-T. Le, A. Sun, J. Srivastava, and Y. A. Kim. Predicting trusts among users of online communities: an epinions case study, EC’08: Proceedings of the 9th ACM Conference on Electronic Commerce. New York, ACM, 2008, 310-319.
[36] M. Sabel, A. garg, and R. Battiti. Wikirep: Digital Reputation in Virtual Communities, University of Trento, Technical Report, 2005, 209-217 [Online]. Available: http:// eprints.biblio.unitn.it/archive/00000810/[LastAccessed December 2009].
[37] H. Zeng, M. Alhossaini, L. Ding, R. Fikes, and D. L. McGuinness. Computing trust from revision history, In Proceedings of the 2006 International Conference on Privacy, Security and Trust, October 2006.
[38] H. Zeng, M. Alhossaini, R. Fikes, and D. McGuinness. Mining revision history to assess trustworthiness of article fragments, In Proceedings of the 2nd International Conference on Collaborative Computing: Networking, Applications, and Worksharing, 2006.
[39] D. L. McGuinness, H. Zeng, P. P. da Silva, L. Ding, D. Narayanan, and M. Bhaowal. Investigations into trust for collaborative information repositories: A wikipedia case study, In Proceedings of the Workshop on Models of Trust for the Web, May 2006.
[40] S. Javanmardi and C. Lopes. Modeling trust in collaborative information systems, CollaborateCom ’07: Proceedings of the 3rd International Conference on Collaborative computing: Networking, Applications and Worksharing. New York, IEEE, 2007.
[41] B. T. Adler and L. de Alfaro. A content-driven reputation system for the wikipedia, WWW ’07: Proceedings of the 16th International Conference on World Wide Web. New York, ACM, 2007, 261-270.
[42] M. Hu, E.-P. Lim, A. Sun, H. W. Lauw, and B.Q. Vuong. Measuring article quality in wikipedia: models and evaluation CIKM ’07: Proceedings of the Sixteenth ACM Conference International Conference on Information and Knowledge Management, Lisbon, Portugal. New York, NY, USA, ACM, 2007, 243-252.
[43] M. Sabel. Structuring wiki revision history, WikiSym ’07: Proceedings of the 2007 International Symposium on Wikis. Montreal, Quebec, Canada, New York, NY, USA, ACM, 2007, 125-130.
[44] N. T. Korfiatis, M. Poulos, and G. Bokos. Evaluating authoritative sources in collaborative editing environments, Online Information Review 30(3) (2006), 252-262.
[45] K. Chatterjee, L. de Alfaro, and I. Pye. Robust Contentdriven Reputation, School of Engineering, University of California, Santa Cruz, Technical Report UCSC-SOE-0809, 2008. [Online]. Available: http://www.soe.ucsc.edu/ luca/papers/08/ucsc-soe-08-09.pdf [Last Accessed December 2009].
[46] F. B. Vi´egas, M. Wattenberg, and K. Dave. Studying cooperation and conflict between authors with history flow visualizations, CHI ’04: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, Vienna, Austria. New York, NY, USA, ACM, 2004, 575-582.
[47] A. Kittur, B. Suh, B. A. Pendleton, and E. H. Chi. He says, she says: conflict and coordination in wikipedia, CHI ’07: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, San Jose, California, USA. New York, NY, USA, ACM, 2007, 453-462.
[48] P. D. Magnus. Early response to false claims in wikipedia, First Monday 13(9) (2008).
[49] Crawler4j. [Online]. Available: http://crawler4j.googlecode. com/ [Last Accessed December 2009].
[50] H. Zeng, M. Alhossaini, L. Ding, R. Fikes, and D. L. McGuinness. Computing trust from revision history, In Proceedings of the 2006 International Conference on Privacy, Security and Trust, October 2006.
[51] M. Ekstrand and J. Riedl. “rv you’re dumb: Identifying discarded work in wiki article history, WikiSym ’09: Proceedings of the 2009 International Symposium on Wikis, Orlando, Florida. New York, NY, USA, ACM, 2009.
[52] P. Heckel. A technique for isolating differences between files, System Sciences 21 (1978), 264-268. · Zbl 0371.68008
[53] Wikipedia event extractor. [Online]. Available: http:// mondego.calit2.uci.edu/WikipediaEventExtractor/[Last Accessed December 2009].
[54] M. Hess, B. Kerrand, and L. Rickards. Wiki User Statistics for Regulating Behaviour, Technical Report, 2006. [Online]. Available: http://icd.si.umich.edu/684/files/ 684
[55] G. Leusch and H. Ney. Bleusp, invwer, cder: Three improved mt evaluation measures, NIST Metrics for Machine Translation Challenge, Waikiki, Honolulu, Hawaii, 2008.
[56] G. Leusch, N. Ueffing, and H. Ney. Cder: Efficient mt evaluation using block movements, In Proceedings of EACL, 2006, 241-248.
[57] W.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.