Quality of sentiment analysis tools: The reasons of inconsistency


Authors / Editors


Research Areas

No matching items found.


Publication Details

Output type: Journal article

UM6P affiliated Publication?: Yes

Author list: Kouadri W.M., Ouziri M., Benbernou S., Echihabi K., Palpanas T., Amor I.B.

Publication year: 2020

Journal: Proceedings of the VLDB Endowment (2150-8097)

Volume number: 14

Issue number: 4

Start page: 668

End page: 681

Number of pages: 14

ISSN: 2150-8097

URL: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85099117458&doi=10.14778%2f3436905.3436924&partnerID=40&md5=e51788478008ecdd682d842ec9d05039

Languages: English (EN-GB)


View in Web of Science | View on publisher site | View citing articles in Web of Science


Abstract

In this paper, we present a comprehensive study that evaluates six state-of-the-art sentiment analysis tools on five public datasets, based on the quality of predictive results in the presence of semantically equivalent documents, i.e., how consistent existing tools are in predicting the polarity of documents based on paraphrased text. We observe that sentiment analysis tools exhibit intra-tool inconsistency, which is the prediction of different polarity for semantically equivalent documents by the same tool, and inter-tool inconsistency, which is the prediction of different polarity for semantically equivalent documents across different tools. We introduce a heuristic to assess the data quality of an augmented dataset and a new set of metrics to evaluate tool inconsistencies. Our results indicate that tool inconsistencies is still an open problem, and they point towards promising research directions and accuracy improvements that can be obtained if such inconsistencies are resolved. © VLDB Endowment. All rights reserved.


Keywords

No matching items found.


Documents

No matching items found.


Last updated on 2021-26-11 at 23:16