Assessing Sentiment in Conflict Zones Through Social Media: Case Study of Yemen Shows That Social Media Data Can be Combined with Polling to Determine Levels of Support for Government and Extremists

This report has been professionally converted for accurate flowing-text e-book format reproduction. While it is widely accepted that polling can assess levels of popular support in a geographic area by surveying a cross-segment of its population, it is less well accepted that analysts can use social media analysis to assess sentiment or popular support within the same space. We examine this question by comparing geographically anchored polling and social media data, utilizing over 1.4 million geo-referenced messages sent through the Twitter network from Yemen over the period from October 2013 to January 2014, to assess both support for extremist groups and support to the Yemeni government. From our research, we conclude that social media data, when combined with polling, has a positive impact on analysis. It can also be a reliable source of stand-alone data for evaluating popular support under certain conditions. Therefore, we recommend future research projects focus on improving the quality of social media data and on operational changes to improve the integration of social media analysis into assessment plans.

This compilation includes a reproduction of the 2019 Worldwide Threat Assessment of the U.S. Intelligence Community.

I. Introduction * II. Literature Review * A. Military Doctrine and Assessment * B. Social Media Analysis * III. Background—Yemen * A. Violent Extremist Organizations * B. Houthis * C. Salafis * D. Al-Qaeda in the Arabian Peninsula and Yemen * E. Government of Yemen * IV. Research Methods * A. Hypothesis * B. Data and Methods * 1. Social Media * 2. Sentiment Dictionary * 3. Kernel Density Estimates * 4. Dependent Variables * 5. Independent Variables * 6. Control Variables * C. Regression Analysis * V. Results * A. Finding One - Improves Predictions * B. Finding Two - Similarity of Spatial Patterns * C. Finding Three - Sentiment Matters * D. Finding Four - The Importance of the Topic * E. Finding Five - Potential To Evaluate By Month * VI. Conclusion * VII. Additional Research * A. Impact of Sentiment * B. Evaluation of the Topic * C. Does the Country/Region Matter? * D. Geo-Location * E. Integration of Social Media Analysis

Since nations have existed, they have faced population-centric forms of warfare— rebellion, insurgency, and civil war—in which insurgents typically battle entrenched governments for the loyalty of the populace, with each side's movements and intentions obscured by the fog of conflict. In the modern era, governments rely on laborious, time-consuming, and expensive opinion polls to navigate through this fog. Opinion polls are like green and red marker buoys in a channel, spaced intermittently along the way to help captains determine their positions. Like buoys, polling is reliable and sheds light on known positions in time, but polls are inherently spaced apart from one another and thus lack timeliness. In a sea covered by a thick blanket of fog, buoys can easily be obscured. Like captains navigating in unclear conditions, military commanders navigating through the fog of conflict require timely location information, or else their efforts may founder. Our research attempts to determine whether social media data analysis can serve as a proxy for or fill in the gap between opinion polls. If so, military commanders could continuously monitor population sentiment and determine whether their efforts were affecting the population.