Mapping the Public Voice for Development—Natural Language Processing of Social Media Text Data: A Special Supplement of Key Indicators for Asia and the Pacific 2022

By Asian Development Bank

The publication introduces the foundations of natural language analyses and showcases studies that have applied NLP techniques to make progress on the Sustainable Development Goals. It also reviews specific NLP techniques and concepts, supported by two case studies. The first case study analyzes public sentiments on the coronavirus disease (COVID-19) in the Philippines while the second case study explores the public debate on climate change in Australia.

• Language: English
• Publisher: Asian Development Bank
• Release date: Aug 1, 2022
• ISBN: 9789292697020

Book preview

MAPPING THE PUBLIC VOICE FOR DEVELOPMENT

NATURAL LANGUAGE PROCESSING OF SOCIAL MEDIA TEXT DATA

A Special Supplement of Key Indicators for Asia and the Pacific 2022

AUGUST 2022

Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO)

© 2022 Asian Development Bank

6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines

Tel [phone]; Fax [phone]

www.adb.org

Some rights reserved. Published in 2022.

ISBN 978-92-9269-701-3 (print); 978-92-9269-702-0 (electronic); 978-92-9269-703-7 (ebook)

Publication Stock No. FLS220347-3

DOI: http://dx.doi.org/10.22617/FLS220347-3

The views expressed in this publication are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent.

ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. The mention of specific companies or products of manufacturers does not imply that they are endorsed or recommended by ADB in preference to others of a similar nature that are not mentioned.

By making any designation of or reference to a particular territory or geographic area, or by using the term country in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.

This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) https://creativecommons.org/licenses/by/3.0/igo/. By using the content of this publication, you agree to be bound by the terms of this license. For attribution, translations, adaptations, and permissions, please read the provisions and terms of use at https://www.adb.org/terms-use#openaccess.

This CC license does not apply to non-ADB copyright materials in this publication. If the material is attributed to another source, please contact the copyright owner or publisher of that source for permission to reproduce it. ADB cannot be held liable for any claims that arise as a result of your use of the material.

Please contact [e-mail] if you have questions or comments with respect to content, or if you wish to obtain copyright permission for your intended use that does not fall within these terms, or for permission to use the ADB logo.

Corrigenda to ADB publications may be found at http://www.adb.org/publications/corrigenda.

Notes:

In this publication, $ refers to United States dollars.

ADB recognizes China as the People’s Republic of China.

Cover design by Princess Monique Abaya Po.

Contents

Tables, Figures, and Listings

Foreword

Sound economic research and policymaking depend on timely and granular data—the coronavirus disease (COVID-19) pandemic further heightened the need for realtime and comprehensive information for these purposes. High-quality data can inform us of the ways that knowledge is generated, how governance is perceived by the general public, and the process of policy formulation. Data are increasingly being generated at the personal level, with social media emerging as an important platform. As more and more people use social media to receive information and share their views, social media text data become a valuable source of timely and granular information that can be used to map and track the behavior, opinions, concerns, and expectations of citizens. Thus, publicly available social media text data from diverse social media platforms can be used to study sentiment or topics of concern in society. Analysis of such data can be conducted through the application of natural language processing (NLP), which is the core subject of this special supplement to Key Indicators for Asia and the Pacific 2022.

This report aims to introduce techniques and procedures of NLP, the computational preparation and analysis of text data, to map the public voice and aid development. First, the report introduces essential concepts of communication and elaborates on the theoretical foundations of natural language analyses. Second, the report reviews research on NLP of social media text data by showcasing studies that have applied the techniques to the Sustainable Development Goals. Third, the report reviews specific NLP techniques, including data preprocessing, and dicusses libraries and programming procedures. It also reviews concepts such as keyword extraction to identify relevant terms, topic modeling to detect common themes, and text classification to recognize language features. These NLP techniques are showcased in two case studies. The first shows how topic modeling can be applied to derive insights on the public debate over climate change in Australia. The second demonstrates how text classification can be leveraged to analyze public sentiment on COVID-19 in the Philippines. Finally, the report discusses the challenges and limitations, as well as the potentials, of NLP.

The data innovations introduced in this special supplement can support policymakers and researchers in Asia and the Pacific to better comprehend the public perspective on specific issues within the burgeoning volume of social media text data. As public sentiment and perceptions determine policy success, governments can apply these techniques to listen to online communities more effectively and adjust program implementation and communication strategies accordingly.

The report is the end product of rigorous contributions from a variety of experts. The special supplement team was led by Daniel Boller, under the overall guidance of Kadra Saeed and Elaine S. Tan. Charibeth Cheng wrote the main report. Contributing authors of the case studies and the main report include Cedric Basuel, Stanley Lawrence Sie, and Alyssa Villanueva. Rhoda Abadia, Kelly Bird, Azel Gorne, Zhigang Li, Rosemarie Marquez, Ker Metanoia Oliva, Zelinna Pablo, Yasuyuki Sawada, Lei Lei Song, and James Villafuerte helped refine the results of the case studies and the main report. We also appreciate the support, direction, and feedback of the Asian Development Bank’s Pacific Department, the Philippines Country Office, and the Information Technology Department throughout the information-gathering process. Ma. Roselia Babalo, Joseph Albert Nino Bulan, Rose Anne Dumayas, and Melissa Pascua provided administrative and operational support and helped organize the workshops, launch events, and other related webinars and briefings necessary to deliver a quality publication. Princess Monique Abaya Po designed the cover of the supplement, and Joe Mark Ganaban provided layout, page design, and typesetting services. Paul Dent edited the report.

We hope this special supplement can equip governments, researchers, and development practitioners in Asia and the Pacific to take advantage of social media text data as a rich new source from which to derive insights on topics of concern in society and, ultimately, to accelerate inclusive and sustainable development.

Albert F. Park

Chief Economist and Director General

Economic Research and Regional Cooperation Department

Asian Development Bank

Abbreviations