Media Effects Research Lab - Research Archive
Finding credibility metrics through non-message related Twitter features
Student Researcher(s)
Jomara Bindá (Ph.D Candidate);
Shane E. Halse ( Candidate);
Samantha S. Weirman (Ph.D Candidate);
Faculty Supervisor
This paper was based on a project as part of the COMM 506: Research Methods in Communications course.
INTRODUCTION
The inclusive nature of social media, such as Twitter, enables crisis response personnel and civilians to leverage these platforms for sharing and viewing relevant, real-time, and on the ground information in times of crisis or emergency. However, a lack of information gatekeeping processes leads to information overload, as well as concerns about both content and source credibility. As a means to both assess source credibility and account for contextual relevance in a crisis situation, the concept of brokerage, borrowed from the domain of social network analysis, is considered in this study.
Brokers are entities who have organically emerged as hubs of information within a network. Their network placement, as well as their ability to effectively distribute information (or resources) translates into some level of perceived credibility by others in the network. In situations where source credibility must be estimated, but traditional metrics for measuring such are either inapplicable or inappropriate, it is hypothesized that metrics of brokerage might inform perceptions of source credibility.
RESEARCH QUESTION / HYPOTHESES
RQ1: For recipients of crisis-related tweets, controlling for Twitter familiarity, dispositional trust, and twitter metric awareness, what is the relationship between broker metrics and perceived source credibility?
H1: Sources who, on average, use a low number of hashtags in tweets will be perceived as more credible.
H2: Sources who have high a follower to followee ratio will be perceived as more credible.
H3: Sources who have high a tweet to retweet ratio will be perceived as more credible.
RQ2: For recipients of crisis-related tweets, controlling for Twitter familiarity, dispositional trust, and twitter metric awareness, what is the relationship between broker metrics and behavioral intentions?
METHOD
A 2 (avg # hashtags: high vs. low) x 2 (tweet/retweet ratio: high vs. low) x 2 (follower/following ratio: high vs. low) x 3 (control for message content) between-subjects experiment was conducted using an online survey in Qualtrics to test the research questions and hypotheses. The recruited participants (N = 199) were presented with a uniform scenario outlining a hypothetical in which they were residents of a fictional town called Samajar and just encountered tweets relating to an earthquake in their region. After the scenario, each participant was randomly shown one of thirty-two different stimuli. Then, all participants were asked the same questions about behavioral intention and perceived source credibility associated with the stimuli, as well as general questions related to topics of twitter familiarity, dispositional trust, twitter metric awareness, and basic demographics.
All stimuli (images of fake tweets) shared the same layout, profile icon, username and handle, and number of hashtags (two) used in the tweet. The standard Twitter message metrics (likes, retweets) stripped away and replaced with our three broker metrics (average number of hashtags used by the source, tweet to retweet ratio, and follower to followee ratio). We opted to use visual representations (bar graphs) for the two ratios, instead of numerical values. We had thirty-two different stimuli—every combination of the three metrics, plus four different message contents for each combination, to control for message content. Figure 1 shows an example with content 1 with all broker metrics in the high category. Figure 2 shows an example with content 1 with all broker metrics in the low category.
Figure 1: Stimulus with content 1, high HT, high FF, high TRT
Figure 2: Stimulus with content 1, low HT, low FF, low TRT
RESULTS
The findings indicate that follower/followee ratio in conjunction with hashtag usage approached a significant effect on perceived source credibility. In addition, both twitter awareness metrics and dispositional trust played an important role in determining behavioral intentions and perceived source credibility.
CONCLUSIONS/DISCUSSION
Within this study we attempted to leverage heuristic metrics of Twitter messages in a way that would allow data such as average number of hashtags, follower/followee, and tweet/retweet ratios to be displayed upfront for use of the message consumer. We then proceeded to examine how these metrics affected behavioral intentions and how they would influence perceived credibility of the source.
Twitter metrics were found to be a significant indicator of perceived credibility. This finding suggests that in order for our metrics and perceived credibility to be significant, the participants must understand what the metrics are and how they are used within the twitter platform. Likewise, dispositional trust approached significance indicating that those who are more likely to trust found messages more credible. Our theory is that those with a low dispositional trust would be less likely to trust twitter metrics, regardless of how valid an indication those metrics might be.
For more details regarding the study contact
Dr. S. Shyam Sundar by e-mail at sss12@psu.edu or by telephone at (814) 865-2173