Document Questions Study
University of Hawai‘i Online Survey Consent Form
Aloha! You are invited to participate in a research study conducted by Bernadette Tix from the Department of Information and Computer Science at the University of Hawai‘i. This research project is a part of a PhD Dissertation studying document creation by Artificial Intelligence programs.What am I being asked to do?
If you participate in this project, you will be asked to fill out a survey, as well as engage in a back-and-forth conversation with an AI chatbot about creating a document for your personal use, such as a letter, memo, or email.Taking part in this study is your choice.
Your participation in this project is completely voluntary. You may stop participating at any time. If you stop being in the study, there will be no penalty or loss to you.Why is this study being done?
The purpose of this project is to determine whether AI can produce better outputs by asking clarifying questions about what a user needs before creating AI-generated documents. I am seeking adult volunteers to generate documents using an AI, and to give feedback on the output which will allow us to determine whether or not the clarifying questions asked by the AI improved the AI-generated documents.What will happen if I decide to take part in this study?
The survey will consist of a brief interaction with an AI chatbot, most likely lasting 10-15 minutes, as well as brief entry and exit surveys. The entry survey will ask for basic demographic information such as your age, gender, and prior experience with AI. When interacting with the AI, you will be asked to think of a document that you would like the AI to write and will answer questions the AI comes up with about your document needs. The exit survey will ask you to rate two different outputs from the AI with questions like, “Would you find this document usable in its current state?” “Is this document close to what you had in mind when you made the request?” and “How would you rate the overall quality of this document?” The survey is anonymous and will be conducted entirely online. The total length of the survey, including interaction with the AI, is expected to last 20-30 minutes.What are the risks and benefits of taking part in this study?
I believe there is little risk to you for participating in this research project. You may become stressed or uncomfortable answering any of the survey questions. If you do become stressed or uncomfortable, you can take a break at any time. You can also stop taking the survey or you can withdraw from the project altogether. There will be no direct benefit to you for participating in this survey. The results of this project may help improve AI’s ability to work in cooperation with humans by giving AI the ability to ask questions and engage in dialog with humans before executing commands.Confidentiality and Privacy:
You will not be asked for any personal information, such as your name or address. Please do not include any personal information in your survey responses or in your responses to the AI. I will keep all study data secure on a private server. Only my University of Hawai'i advisor and I will have access to the information. Other agencies that have legal permission have the right to review research records. The University of Hawai'i Human Studies Program has the right to review research records for this study.Future Research Studies:
The data from this study will not be used or distributed for future research studies.Questions: If you have any questions about this study, please email me at bjavery@hawaii.edu
You may also contact my advisor, Dr. Kimberly Binsted, at binsted@hawaii.edu
You may contact the UH Human Studies Program at 808.956.5007 or uhirb@hawaii.edu to discuss problems, concerns and questions, obtain information, or offer input with an informed individual who is unaffiliated with the specific research protocol. Please visit http://go.hawaii.edu/jRd for more information on your rights as a research participant.
If you agree to participate in this project, please give electronic consent by checking the box below:
Please Wait...
The experiment is currently being set up. This may take up to two minutes.Part 1 of 9: Demographics Screener
This study is only open to participants who are over the age of 18 and fluent in English. Please answer the questions below to continue.
This study is only open to participants who are over the age of 18 and fluent in English. Thank you for your interest! Please close the browser to exit the survey.
Part 2 of 9: Demographics
Welcome!
Thank you for taking the time to participate in this study. In this study, you will be using AI to create a short document of your choosing. Completion of the study may take as little as a few minutes or as much as half an hour, depending on the complexity of the document you are trying to create. Please DO NOT press the back button on your broswer or refresh the page, as this will reset the survey and you will not be able to recover your work in progress. When you are ready to begin, please answer the following demographic questions about yourself and then click the "continue" button below.Part 3 of 9: Describe your Document
On this page, you will be communicating with an AI that is capable of writing short documents such as letters, memos, emails, and short reports. Please think of a document you would like the AI to create for you. This could be a document you actually need, or one that you have just made up for the experiment. Either way, please think in detail about what you would need this document to include. When you are ready, write out what you need in the textbox below. After you submit your prompt, the AI will ask you a series of questions, and you will then be given two versions of the document you requested, and asked for feedback on which version you prefer.Example prompts:
- “Please write a cover letter for a job working in tech support for a college student with one year of prior tech support experience.”
- “Write an email to my boss asking them if they can meet next week to discuss a project which is behind schedule.”
- “I want to reach out to a friend I haven’t spoken to in a while. Give me an outline for a letter to help me figure out what I should say.”
Enter your prompt below:
Please Wait ...
The AI is working on your request. This may take several minutes.Part 4 of 9: Questions and Answers
To give you the best results, the AI would like to ask you a few questions about the document you need. Please provide your answers and then click "continue."Questions:
Would you like to provide any thoughts or feedback about these questions?
Part 5 of 9: Rating the Douments
Thank you for your thoughtful answers! The AI has considered your prompt and your responses to its questions, and has generated two different possible documents for you. Please read each one and answer the questions that follow.
Which document would be more useful to you in its current state?
Do you have any feedback about document 1?
Do you have any feedback about document 2?
Part 6 of 9: Refining the Douments
Documents generated by AI can sometimes be improved by giving the AI further instructions after it has completed its initial attempt. If you would like to refine one or both of these documents, enter further instructions below. Please try to get each document as close as you can to a document you would actually want to use.
Enter additional instructions for document 1 below:
Enter additional instructions for document 2 below:
Part 7 of 9: Rating the Douments
Thank you for taking the time to refine the documents. Please rate the two documents again, based on how they’ve turned out now after the refining process.
Which document would be more useful to you in its current state?
Do you have any feedback about document 1?
Do you have any feedback about document 2?
Part 8 of 9: Results and Continuation
Thank you for completing the study! The documents you have created are provided below for your use. If you would like to keep a copy of the documents you created, you can copy them below. Once you exit the browser or navigate away from this page, you will no longer be able to access these documents.
Document #1
Document #2
Would you like to create another document, or proceed to the exit survey and exit the study?
Part 9 of 9: Exit Survey
Thank you for rating the AI’s responses. For the final step in this study, please rate the following statements on a scale of “Strongly Agree” to “Strongly Disagree”.It was annoying to have to answer questions even though I had already explained what I wanted the AI to do.
Strongly Disagree Strongly Agree
I felt like the AI was more engaged with my problem because it asked follow-up questions.
Strongly Disagree Strongly Agree
I would be willing to answer follow-up questions from an AI if answering questions led to better results.
Strongly Disagree Strongly Agree
I liked that the AI showed me two options to pick between, instead of only picking the option it thought was best.
Strongly Disagree Strongly Agree
Answering the questions asked by the AI made me think about my request in ways I hadn’t previously considered.
Strongly Disagree Strongly Agree
Do you have any additional feedback or comments (optional)?
Thank You!
You have completed the experiment! Thank you for your participation. If you have any further questions, please contact Bernadette Tix at bjavery@hawaii.edu. Please close your browser to exit.
If you would like to keep a copy of the documents you created, you can copy them below. Once you exit the browser or navigate away from this page, you will no longer be able to access these documents.
Document #1
Document #2
For SurveyCircle users (www.surveycircle.com): The Survey Code is: Y9K3-7F11-RXY1-3HFX
Redeem Survey Code with one click: https://www.surveycircle.com/Y9K3-7F11-RXY1-3HFX/