Svarsfrekvensen i undersökningar från webbpaneler. Några resultat Svarsfrekvensen i undersökningar från webbpaneler. Några resultat. johan.martinsson@pol.gu.se www.lore.gu.se

Varför bryr vi oss om svarsfrekvens? Inte självklart att den är lika viktig i panelundersökningar som i tex slumpmässiga populationsurval Frågan är om den ens är viktig där! (Groves & Peytcheva 2008) Men: ekonomiska skäl tex kapacitet / effektivitet vid riktigt låga nivåer risk för ”svarsdominans” av ett fåtal

Vad påverkar svarsfrekvenser i frågeundersökningar? Kulturella / samhälleliga faktorer Individuella egenskaper Undersökningens utformning, val av undersökningens/enkätens upplägg etc

Some questions about when and how to send your surveys How long after recruitment do you send your first survey? Do long quarantine periods hurt response rates? What day of week do you send it? What time of day do you send it? Who should send it? What purpose should you state with your survey? How many reminders are worthwhile?

1. How long after recruitment do you send your first survey 1. How long after recruitment do you send your first survey? what we did: 4,000 people recruited from Aftonbladet vote compass in September approximately 1,500 randomly assigned not to receive survey until November the remaining 2,500 receive a post-election survey three days after election Next, both groups received a survey in late November

Svarsfrekvenser vid olika tidpunkter (%)

2. Do long quarantine periods hurt response rates? what we did: In February 2015, 4,000 panel members were randomly assigned into two different groups 2,000 were to continue to receive surveys regularly, while 2,000 were put in quarantine for at least 6 months all 4,000 received the same survey again in October 2015

3. What day of week do you send your survey? what we did: We used a sample of approx. 13,000 respondents that were to receive a survey These were randomized into 7 groups to cover each day of the week They also received a reminder on the same weekday as the invitation Source: LORE Methodological Note 2014:20

4. What time of day do you send your survey? what we did: In November 2014 we used a fairly big sample of 47,000 and randomized it into 6 different time points btw 7 am and 9 pm We examine the participation rate after 24 hours and after 6 days (before first reminder was sent out) Source: LORE Mini Note 2014:17 and 2014:18

5. Who should send it? what we did: In a survey in September 2014 we randomized 17,000 newly recruited respondents into 4 groups We used a 2x2 design that varied gender and status: Male vs female name and ”professor” vs ”research assistant” We also added an extra treatment group with a non personal sender, i.e. ”The Citizen Panel”

6. What purpose should you state with your survey? what we did: Simultaneously with the non-personal (institutional) sender in the previous experiment, we varied the stated purpose with the surveys. No purpose, international research, contribute to the public debate, or a combination.

7. How many reminders are worthwhile? what we did: We have conducted a number of randomized experiments to examine the effects of reminders. Source: LORE Methodological Notes 2014:3, 2014:5 and 2015:7

Let us return to the seven questions Does it matter? 1 Time after recruitment to first survey Yes 2 Long quarantine periods No 3 Day of week In the (very) short run 4 Time of day 5 Sender Yes (slightly) 6 Stated purpose 7 Number or reminders Yes (ofcourse)

Spelar det någon roll då? Får vi annorlunda svarspersoner vid högre svarsfrekvenser i panelundersökningar?

En snabb titt på representativiteten vid olika svarsfrekvenser antal påminnelser 1 2 3 svarsfrek 46 58 62 64 16-29 år 10 8 9 Kvinna 39 Högutbildade 54 Arbetslösa Inte intresserade av politik 11 Mkt lågt politikerförtroende 12

Spelar det någon roll då? - kanske för effektivitet och ekonomi Får vi annorlunda svarspersoner vid högre svarsfrekvenser i panelundersökningar? - nej, att gå från 46 till 64 i svarsfrekvens tycks inte spela någon som helst roll i denna studie

Read more at www.lore.gu.se/research Laboratory of Opinion Research (LORE)