Media Effects Research Lab - Research Archive
Perceived Credibility of Automated Journalism vs. Human Journalism
Student Researcher(s)
Shihan Zhao (B.A. Candidate);
Peiyi Wang (B.A. Candidate);
Qi Yuan (B.A. Candidate);
Xi Yu (B.A. Candidate);
Faculty Supervisor
This paper was based on a project as part of the "COMM 418: Media Effects: Theory and Research" course.
INTRODUCTION
This study set out to examine the relationship between automated journalism and individual’s perceived trustworthiness and expertise vs. human journalism. Also, examining the message credibility of automated journalism vs. human journalism. Theoretical concepts, the MAIN model and the Human AI Interaction-TIME model were applied in this study. Through an experiment online survey(N = 106), this study provides evidence that links individuals’ differences and perceptions of automated journalism vs. human journalism.
RESEARCH QUESTION / HYPOTHESES
H1a: People who have high levels of belief in MH will perceive automated journalists to be more trustworthy and have higher expertise than human journalists.
H1b :People who have high levels of belief in MH will perceive news written by automated journalists to be more credible.
H2a: People who have low levels of belief in MH will perceive human journalists to be more trustworthy and have higher expertise than automated journalists.
H2b :People who have low levels of belief in MH will perceive news written by human journalists to be more credible.
RQ: For users with different levels of machine heuristic , who are exposed to the news content about the covid-19 vaccine, what is the relationship between exposed to automated journalism vs. human journalism and the perceived trustworthiness and the perceived expertise of the information of the news story?
METHOD
This study used an experimental design administered through an online survey (N=106). Each respondent was randomly assigned to one of the following two conditions, with no statistically significant differences between the groups for age, gender, education, media consumption of news, or political preference :
1. Automated texts with assigned computer sources as Machine Journalism (N=53);
2. Human texts with assigned human sources as Human Journalism (N=47);
For participant recruitment, first, convenience sampling and snowball sampling were conducted over the social media platforms Facebook, Instagram and Wechat. We also used an online recruitment machine, Amazon Mechanical Turk, which is a forum where requesters can post their work as artificial intelligence tasks (HIT). Each participant will be randomly distributed under two different stimuli, they are Human journalism Vs. Machine journalism. Most surveys are on a 7-point scale. In our research, we believe that Cognitive Heuristic will affect the influence of our independent variable on the dependent variable. All the production of our survey is done through Qualtrics PSU. After collecting those answers from all participants, we exported the data from Qualtrics to JMP (statistical software). For the data analysis, multiple factorial analysis of variance (ANOVA) is conducted. Our experiment also used a crosstab to detect Chi-Square tests.
RESULTS
Hypothesis H1a, H2a were supported and showed that when people have high levels of machine heuristic, they will perceive automated journalist to be more trustworthy and have higher expertise than human journalist, and in the same way, people who have low levels of machine heuristic will perceive human journalist to be more trustworthy and have higher expertise than automated journalist. Next, hypothesis H1b and H2b were not supported, and the results showed the relationship between message credibility and machine heuristic was not significant.
CONCLUSIONS/DISCUSSION
Our findings are consistent with the HAII-TIME Model. And we found that automatic journalism is more suitable for people who have a high level of belief in heuristic, they may be more willing to accept contents from automatic journalists. We have a small number of participants, and in the research, we do not have control variables. Also, the news is focused on covid vaccine area and is from an authoritative website which means that the news already has credibility. For the further study,we want to distribute our surveys in different ethnicities; using news from different fields, instead of vaccines; and adding some control variables, for example, Age and Education level.
For more details regarding the study contact
Dr. S. Shyam Sundar by e-mail at sss12@psu.edu or by telephone at (814) 865-2173