Subreddit communities often develop extensive rules governing the kinds of participation they allow. But does making those policies visible have any effect on the rule-compliance of newcomers to the community? And might there also be a side effect on the number of people who participate? Over 29 days in September, I worked with the moderators of r/science to test this question with a field experiment, an A/B test where we posted sticky comments to some threads and not to others.

TLDR: Yes! Adding a sticky comment with the rules has a positive 7.3 percentage point effect on the chance that a newcomer’s comment will comply with the rules, on average across r/science, holding all else constant. Furthermore, posting the rules increases the incidence rate of newcomer comments by 38.1% on average.

But there’s a catch! In followup analyses, I found that sticky comments had opposite effects in AMAs (question-answer discussions with prominent scientists) compared to non-AMA posts. Posting the rules to a non-AMA thread caused a 59% increase in the incidence rate of newcomer comments, but in AMA threads, sticky comments caused a 65.5% decrease on average, the opposite outcome. Sticky comments also affected the amount of moderator work per post. Posting a sticky comment increased the incidence rate of all comment removals by 36.1% in non-AMA posts and decreased the incidence rate by 28.6% in AMA posts on average across r/science.