research-article

Free access

What would users change in my app? summarizing app reviews for recommending software changes

Authors:

Andrea Di Sorbo,

Sebastiano Panichella,

Carol V. Alexandru,

Junji Shimagaki,

Corrado A. Visaggio,

Gerardo Canfora,

Harald C. GallAuthors Info & Claims

FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering

Pages 499 - 510

https://doi.org/10.1145/2950290.2950299

Published: 01 November 2016 Publication History

Abstract

Mobile app developers constantly monitor feedback in user reviews with the goal of improving their mobile apps and better meeting user expectations. Thus, automated approaches have been proposed in literature with the aim of reducing the effort required for analyzing feedback contained in user reviews via automatic classification/prioritization according to specific topics. In this paper, we introduce SURF (Summarizer of User Reviews Feedback), a novel approach to condense the enormous amount of information that developers of popular apps have to manage due to user feedback received on a daily basis. SURF relies on a conceptual model for capturing user needs useful for developers performing maintenance and evolution tasks. Then it uses sophisticated summarisation techniques for summarizing thousands of reviews and generating an interactive, structured and condensed agenda of recommended software changes. We performed an end-to-end evaluation of SURF on user reviews of 17 mobile apps (5 of them developed by Sony Mobile), involving 23 developers and researchers in total. Results demonstrate high accuracy of SURF in summarizing reviews and the usefulness of the recommended changes. In evaluating our approach we found that SURF helps developers in better understanding user needs, substantially reducing the time required by developers compared to manually analyzing user (change) requests and planning future software changes.

Formats available

You can view the full content in the following formats:

References

[1]

U. Abelein, H. Sharp, and B. Paech. Does involving users in software development really influence system success? IEEE Software, 30(6):17–23, 2013.

Digital Library

[2]

R. A. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.

Digital Library

[3]

G. Bavota, M. Linares-Vasquez, C. Bernal-Cardenas, M. Di Penta, R. Oliveto, and D. Poshyvanyk. The impact of api change- and fault-proneness on the user ratings of android apps. Software Engineering, IEEE Transactions on, 41(4):384–407, April 2015.

Digital Library

[4]

R. Chandy and H. Gu. Identifying spam in the ios app store. In Proceedings of the 2Nd Joint WICOW/AIRWeb Workshop on Web Quality, WebQuality ’12, pages 56–59, New York, NY, USA, 2012. ACM.

Digital Library

[5]

N. Chen, J. Lin, S. C. H. Hoi, X. Xiao, and B. Zhang. Ar-miner: Mining informative reviews for developers from mobile app marketplace. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 767–778, New York, NY, USA, 2014. ACM.

Digital Library

[6]

A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella, and S. Panichella. Labeling source code with information retrieval methods: an empirical study. Empirical Software Engineering, 19(5):1383–1420, 2014.

Digital Library

[7]

A. Di Sorbo, S. Panichella, C. A. Visaggio, M. Di Penta, G. Canfora, and H. C. Gall. Development emails content analyzer: Intention mining in developer discussions (T). In 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pages 12–23, 2015.

Digital Library

[8]

A. Di Sorbo, S. Panichella, C. A. Visaggio, M. Di Penta, G. Canfora, and H. C. Gall. DECA: development emails content analyzer. In Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016 - Companion Volume, pages 641–644, 2016.

Digital Library

[9]

B. Fu, J. Lin, L. Li, C. Faloutsos, J. Hong, and N. Sadeh. Why people hate your app: Making sense of user feedback in a mobile app store. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, pages 1276–1284, New York, NY, USA, 2013. ACM.

Digital Library

[10]

L. V. Galvis Carre˜ no and K. Winbladh. Analysis of user comments: An approach for software requirements evolution. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 582–591, Piscataway, NJ, USA, 2013. IEEE Press.

Digital Library

[11]

X. Gu and S. Kim. What parts of your apps are loved by users? (T). In 30th IEEE/ACM International Conference on Automated Software Engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pages 760–770, 2015.

Digital Library

[12]

E. Guzman and W. Maalej. How do users like this feature? a fine grained sentiment analysis of app reviews. In Requirements Engineering Conference (RE), 2014 IEEE 22nd International, pages 153–162, Aug 2014.

[13]

E. Ha and D. Wagner. Do android users write about electric sheep? examining consumer reviews in google play. In Consumer Communications and Networking Conference (CCNC), 2013 IEEE, pages 149–157, Jan 2013.

[14]

S. Haiduc, J. Aponte, L. Moreno, and A. Marcus. On the use of automated text summarization techniques for summarizing source code. In Proceedings of the International Working Conference on Reverse Engineering (WCRE), pages 35–44. IEEE, 2010.

Digital Library

[15]

M. Harman, Y. Jia, and Y. Zhang. App store mining and analysis: Msr for app stores. In Mining Software Repositories (MSR), 2012 9th IEEE Working Conference on, pages 108–111, June 2012.

Digital Library

[16]

K. Herzig, S. Just, and A. Zeller. It’s not a bug, it’s a feature: How misclassification impacts bug prediction. In Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13, pages 392–401, Piscataway, NJ, USA, 2013. IEEE Press.

Digital Library

[17]

C. Iacob and R. Harrison. Retrieving and analyzing mobile apps feature requests from online reviews. In Proceedings of the 10th Working Conference on Mining Software Repositories, MSR ’13, pages 41–44, Piscataway, NJ, USA, 2013. IEEE Press.

Digital Library

[18]

C. Iacob, R. Harrison, and S. Faily. Online reviews as first class artifacts in mobile app development. In G. Memmi and U. Blanke, editors, Mobile Computing, Applications, and Services, volume 130 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pages 47–53. Springer International Publishing, 2014.

[19]

V. N. Inukollu, D. D. Keshamoni, T. Kang, and M. Inukollu. Factors Influencing Quality of Mobile Apps:Role of Mobile App Development Life Cycle. ArXiv e-prints, Oct. 2014.

[20]

H. Khalid, E. Shihab, M. Nagappan, and A. E. Hassan. What do mobile app users complain about? IEEE Software, 32(3):70–77, 2015.

Digital Library

[21]

S. Krusche and B. Bruegge. User feedback in mobile development. In Proceedings of the 2Nd International Workshop on Mobile Development Lifecycle, MobileDeLi ’14, pages 25–26, New York, NY, USA, 2014. ACM.

Digital Library

[22]

S. A. Licorish, A. Tehir, M. F. Bosu, and S. G. MacDonell. On satisfying the android os community: User feedback still central to developers’ portfolios. In 2015 24th Australasian Software Engineering Conference (ASWEC, pages 78–87, 2015.

Digital Library

[23]

W. Maalej and H. Nabil. Bug report, feature request, or simply praise? on automatically classifying app reviews. In Requirements Engineering Conference (RE), 2015 IEEE 23rd International, pages 116–125, Aug 2015.

[24]

P. W. McBurney and C. McMillan. Automatic documentation generation via source code summarization of method context. In Proceedings of the International Conference on Program Comprehension (ICPC), pages 279–290. ACM, 2014.

Digital Library

[25]

G. A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41, Nov. 1995.

Digital Library

[26]

L. Moreno, J. Aponte, G. Sridhara, A. Marcus, L. Pollock, and K. Vijay-Shanker. Automatic generation of natural language summaries for java classes. In Proceedings of the International Conference on Program Comprehension (ICPC), pages 23–32. IEEE, May 2013.

[27]

G. C. Murphy. Lightweight Structural Summarization As an Aid to Software Evolution. PhD thesis, 1996.

Digital Library

[28]

AAI9704521.

[29]

J. Oh, D. Kim, U. Lee, J.-G. Lee, and J. Song. Facilitating developer-user interactions with mobile app review digests. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’13, pages 1809–1814, New York, NY, USA, 2013. ACM.

Digital Library

[30]

D. Pagano and W. Maalej. User feedback in the appstore: An empirical study. In In Proceedings of the 21st IEEE International Requirements Engineering Conference (RE 2013), pages 125–134. IEEE Computer Society, 2013.

[31]

F. Palomba, M. Linares-Vasquez, G. Bavota, R. Oliveto, M. Di Penta, D. Poshyvanyk, and A. De Lucia. User reviews matter! tracking crowdsourced reviews to support evolution of successful apps. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on, pages 291–300, Sept 2015.

Digital Library

[32]

B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, EMNLP ’02, pages 79–86, Stroudsburg, PA, USA, 2002. Association for Computational Linguistics.

Digital Library

[33]

S. Panichella, J. Aponte, M. D. Penta, A. Marcus, and G. Canfora. Mining source code descriptions from developer communications. In Program Comprehension (ICPC), 2012 IEEE 20th International Conference on, pages 63–72, 2012.

[34]

S. Panichella, A. Di Sorbo, E. Guzman, C. Visaggio, G. Canfora, and H. Gall. How can I improve my app? classifying user reviews for software maintenance and evolution. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on, pages 281–290, Sept 2015.

Digital Library

[35]

S. Panichella, A. Panichella, M. Beller, A. Zaidman, and H. Gall. The impact of test case summaries on bug fixing performance: An empirical investigation. In Proceedings of the International Conference on Software Engineering (ICSE). IEEE, 2016.

Digital Library

[36]

M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980.

[37]

O. Rambow, L. Shrestha, J. Chen, and C. Lauridsen. Summarizing email threads. In In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) Short Paper Section, 2004.

Digital Library

[38]

S. Rastkar, G. C. Murphy, and G. Murray. Summarizing software artifacts: A case study of bug reports. In Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE ’10, pages 505–514, New York, NY, USA, 2010. ACM.

Digital Library

[39]

S. Rastkar, G. C. Murphy, and G. Murray. Automatic summarization of bug reports. IEEE Trans. Softw. Eng., 40(4):366–380, Apr. 2014.

Digital Library

[40]

F. Sarro, A. Al-Subaihin, M. Harman, Y. Jia, W. Martin, and Y. Zhang. Feature lifecycles as they spread, migrate, remain, and die in app stores. In Requirements Engineering Conference (RE), 2015 IEEE 23rd International, pages 76–85, 2015.

[41]

N. Seyff, G. Ollmann, and M. Bortenschlager. Appecho: A user-driven, in situ feedback approach for mobile platforms and applications. In Proceedings of the 1st International Conference on Mobile Software Engineering and Systems, MOBILESoft 2014, pages 99–108. ACM, 2014.

Digital Library

[42]

A. Sharma, Y. Tian, and D. Lo. Nirmal: Automatic identification of software relevant tweets leveraging language model. In Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on, pages 449–458, March 2015.

[43]

G. Sridhara. Automatic Generation of Descriptive Summary Comments for Methods in Object-oriented Programs. PhD thesis, Newark, DE, USA, 2012.

Digital Library

[44]

AAI3499878.

[45]

G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker. Towards automatically generating summary comments for java methods. In Proceedings of the International Conference on Automated Software Engineering (ASE), pages 43–52. ACM, 2010.

Digital Library

[46]

C. Vassallo, S. Panichella, M. Di Penta, and G. Canfora. Codes: Mining source code descriptions from developers discussions. In Proceedings of the 22Nd International Conference on Program Comprehension, ICPC 2014, pages 106–109, New York, NY, USA, 2014. ACM.

Digital Library

[47]

T. Vithani. Modeling the mobile application development lifecycle. In Proceedings of the International MultiConference of Engineers and Computer Scientists 2014, Vol. I, IMECS 2014, pages 596–600, 2014.

[48]

P. M. Vu, T. T. Nguyen, H. V. Pham, and T. T. Nguyen. Mining user opinions in mobile app reviews: A keyword-based approach. CoRR, abs/1505.04657, 2015.

[49]

Z. Wu and M. Palmer. Verbs semantics and lexical selection. In Proceedings of the 32Nd Annual Meeting on Association for Computational Linguistics, ACL ’94, pages 133–138, Stroudsburg, PA, USA, 1994. Association for Computational Linguistics.

Digital Library

Cited By

Shiri VLiu CYao KGuo JCheng J(2026)Beyond Privacy Labels: How Users Perceive Different Information Sources for Understanding App's Privacy PracticesProceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems10.1145/3772363.3798787(1-5)Online publication date: 13-Apr-2026
https://dl.acm.org/doi/10.1145/3772363.3798787
Mynampaty HJosephine NIsaacs KM McNutt A(2026)Linting Style and Substance in READMEsProceedings of the 2026 CHI Conference on Human Factors in Computing Systems10.1145/3772318.3791597(1-23)Online publication date: 13-Apr-2026
https://dl.acm.org/doi/10.1145/3772318.3791597
Panichella SKhatiri SBirchler C(2026)Deployment for Cyber-Physical Systems: The Relations Between the Testing and Monitoring DevOps PhasesRoadmap for DevOps in Cyber-Physical Systems10.1007/978-981-95-1786-2_6(103-141)Online publication date: 7-Feb-2026
https://doi.org/10.1007/978-981-95-1786-2_6
Show More Cited By

Index Terms

Recommendations

Comments

View Table of Contents