[Edit: This thread has served its purpose. The survey, a series of anonymous Discourse polls, has been created and is now closed. The results can be seen at this link.]
I’ve seen many online groups that do anonymous demographic surveys to get an idea of the general shape of the community. Would people want to do one if I made it and posted it to this forum? If I did, would it be better to do it through the Discourse poll feature or through something like Google Forms, which supports things like anonymous short answer questions and better data visualizations, with the caveat that the spreadsheet of all form responses could be made available to anyone and would always be available to the form’s creator because of how Google Forms works? Or should I use another external survey site, since there are a lot of them?
[Edit to clarify: Google Forms provides you with both a page of anonymous, aggregated survey results, which I 100% plan to make public, and a spreadsheet of individual answers which is always available to the form creator and can be hidden or made publicly available online.] If I did do a poll on Google Forms or something, would people want me to make the full spreadsheet of individual answers publicly available or no? I’m leaning towards no, since I think it would incentivize more people to answer. I would be able to see it no matter what, but I would never share the answers spreadsheet or individual answers if people don’t want them shared. I’m aware that people only have my word to go on and many people wouldn’t want to give too much information to a stranger online, even if I promise that only I can see it. So for that reason, people may wish to use Discourse polls instead, or have someone more trusted run the Google Form, or do something else I haven’t thought of.
I’d mainly ask the kinds of questions that census bureaus ask, though I want to put a few IF-specific questions in there, and people can suggest questions in the replies if they have ideas. I like doing long surveys and demographic polls, and love looking at demographic polls, though others might disagree.
Personally, I would prefer using anonymized Discourse polls, just so it’s all centralized. You can’t do anonymous short-answer responses, but you can make single choice, multiple choice, number rating, and ranking polls, as well as display them in bar or pie graphs.
If we use Google Forms or similar, I would like it to be anonymized and the responses spreadsheet made public, although I’m not sure what the added benefit of having short-answer forms would be.
I like this idea! I would vote Google Forms (or a different survey site) over Discourse polls because then the survey could be open to people without accounts on this forum—unless you specifically wanted to do a survey of only forum participants, but I think a wider survey of people interested in IF could be beneficial. I also like the idea of having short answer questions; I have vague ideas for some, I’ll share them once I’ve thought them through a little more.
As long as it was anonymous, I don’t see an issue with sharing it; I feel like doing a survey becomes a lot less useful if the data is never made public.
Yeah, I’ve never made a poll with Google Forms but every time I’ve seen the results shared it’s been aggregate data with no way to see how each individual responded, which I think is perfectly fine. And it’s not as if you can know who answered the poll as long as it doesn’t explicitly ask you your name (which I don’t see why it should?).
to get an idea of the general shape of the community
Which community? I’d encourage any surveys to be hosted on this Discourse site. The value of that? It identifies the population you are sampling. Otherwise it’s just a form on the internet that anyone can come along and fill out. What would we conclude from that?
Well, unless you think the poll is going to get shared widely outside the forum/IF community, which I think is very unlikely, it still identifies the population you’re sampling: people who visit the intfiction forums with at least some regularity and therefore presumably are interested in interactive fiction. The real difference between Discourse and Google Forms is whether you want to include lurkers without an account.
I use Google forms a lot, and I’ll provide a bit of information here for anyone interested in trying it for the first time.
When setting up a google form, it is possible to automatically collect email addresses of the people taking the survey. The survey then gives a warning (near the top, in fine print) that your address is being collected. The default setup is to not collect addresses, but this is something to be aware of if you’re taking a survery written by someone you don’t know; read the fine print near the top.
The forms app can autogenerate representations of the data (most of these are aggregate pie charts and bar charts, but it is possible to view individual responses.) The app also allows the creator to easily transfer data to a google sheet, where more powerful forms of statistical analysis can be applied.
The text formatting in google forms is horrible. I dare say impossible, except that there is a time consuming work around. This probably won’t affect most users, but I use the software often to write chemistry quizzes, so I’m constantly frustrated by the difficulty to form superscripts and subscripts.
If you are using google forms to write quizzes (instead of surveys) there are some easy to use features for providing feedback to students for their right or wrong answers, either immediately or after they’ve taken the assessment.
I’m not a Google distruster though I understand there can be security issues with anything. I am more likely to click a poll here than open up a separate browser window just cuz less work involved, but I also understand that an external form is more helpful if you’re polling other communities besides this one.
Doug’s right, and for those unfamiliar with Google Forms it’s worth reading Doug’s post above.
For clarification, I would certainly release the aggregated results of the demographic survey, but my question about whether the answers spreadsheet should be shared comes down to the fact that, in Google Forms, the answers spreadsheet means individual answers are always available to the creator of the survey, and if that spreadsheet was shared, individual answers would be available to the entire public Internet. I’ve edited the first post in this thread to make this clear.
So this could potentially be a huge privacy violation depending on how specific the questions are. I have no intention of asking for people’s names or usernames, but identifying someone without that can still be very easy if they answer all the questions. Making the answers spreadsheet public would deter people from answering as many questions as they would otherwise. This is what I meant when I said that making the answers spreadsheet private would incentivize more people to answer.
I do know of a Google Forms survey creator who used a workaround: both the aggregated results and the individual answers spreadsheet were made public, but a question at the end of the form asked if the respondent wanted their individual response to be private, and if they answered “yes”, their response was deleted from the spreadsheet before it went public, while still being included in the creator’s analysis and the aggregated results. This is a potential workaround.
I don’t know how other survey sites work; the downside is I’ve never used them and don’t know how they work.
svlin is right about the fact that this is optional. Google Forms allows you to make anonymous surveys that anyone online can answer, even without an account. I plan to do this. It would make it easier to put in false responses, but I don’t think that’s a significant risk and the tradeoff is worth it.
For now, I’m leaning towards “anonymous Google Form that anyone can answer”; I don’t mind if it gets crossposted elsewhere, since I can just include a mandatory question about which communities the respondent frequents, which can be used to identify Intfiction users in particular. And I’d rather have more people answering than less; I like the data analysis tools the Google Forms spreadsheet offers, and more people answering helps to anonymize individual responses.
For more detail about polling, please see this post on Discourse Meta.
I was thinking pollers could export data, but apparently that’s only available to Staff of the forum. There’s always potential if you’d like to run a poll here, you could request the CSV from us[1] later to combine with other data.
Can I export poll data for further analysis?
Staff can use the Discourse Data Explorer to export poll data as a csv, which can be imported into the data analysis program of their choice.
Just as a heads up I’m stupid and I believe the Data Explorer is something Dannii might need to help with as I’ve never used it directly, but I know he made me a report once that showed statistics of what themes forum users are using. ↩︎
I think most people want Google Forms, but this is still a useful guide anyway, so thanks. It’s good to know poll responses can be exported from Discourse, in case I do settle on doing it here.
I almost want to create a Discourse poll here on what kind of poll we should use. That might be too meta, though.
Oh, I see! Yes in this case I definitely understand keeping the answers spreadsheet private—or seeking a different survey program than Google Forms; there must be other decent free options out there that wouldn’t have this issue.
I think this is an interesting idea, and I would be curious to see the results! I’d be glad to participate, especially if individual responses are not made public.
I like this idea! And it could allow you to see demographic breakdowns of different communities, if you wanted to do that level of analysis.
The biggest issue with making individual responses available is that if you include any free text responses, and people write anything in there which is specific enough to be individually identifying, you can then match it to the rest of their responses, removing their anonymity.
The survey platform which my employer uses to survey staff satisfaction separates all of the free text responses from the rest of the data before anyone can view it. I don’t know if there’s any openly available platform that can do that?
Yeah, even something like “gender” (if it’s a write-in) can be potentially uniquely identifying. But even if it’s not a write-in, a rare answer (e.g. nonbinary, maybe not rare in this community, but y’know) in combination with one or two other bits of information can be uniquely identifying.
As an update, I’ve looked at different survey programs but haven’t really found one I liked. A lot of survey websites/apps out there seem to be geared towards businesses that want to do marketing analysis, and require you to pay to get features like having more than 100 responses recorded in the survey.
I think I’ll just stick with an anonymous Google Form that anyone can answer, and swear to never share the individual spreadsheet of answers. But I will make the default page of aggregate results, which Google provides by default, publicly available to everyone. I’ll also do result breakdowns per community, if I get enough responses to do that without threatening anonymity.
I’ll try to avoid asking questions that are too personal or potentially identifying. Still not sure if I want to include free response questions or not. Can’t think of any good ones at the moment.
I’ll likely release the survey in a few days. People can comment with ideas for questions they want included. I’m not sure if the survey should be posted in a new Intfiction thread or just added to the end of this thread as a new post. Would posting it as a new thread help visibility?