Guidelines 2.0 staging

From WeAreDynamo Wiki
Jump to: navigation, search

We will use this page to discuss changes to the guidelines for academic requesters. We will publish ver 2.0 when these changes are agreed upon and implemented. The guidelines will be frozen again when everyone agrees with the changes.

DONE

  • The AMT Policies page gives a list of examples of types of HITs that violate the Terms of Service. One of these examples was “HITs that require Workers to download software”. At some point after the last Archive.org capture on March 17, 2015, Amazon changed this sentence to say “HITs that require Workers to download software that contains any malware, spyware, viruses, or other harmful code.”. This change now means some parts of the Dynamo Guidelines for Academic Requesters are outdated and need changing. Read more: http://www.wearedynamo.org/forum/actions/topics/call-for-comments-turker-authored-guidelines-for-research-on-amt?page=5#post-527
  • Unfortunately, many HITs, even from academic sources, currently don't come close to even that modest minimum. Fair Payment, which is referenced from the main page's section about pay, explains where the $6/hr minimum came from (it is, and has been for years, the most commonly stated target minimum number among turkers, some of whom have very mixed feelings about trying to ask for a higher minimum), and lists many reasons why researchers should strongly consider paying more than that minimum in many situations. The idea is for requesters to read that page and make a more-informed decision on what they would feel ethical to pay for their particular HITs, without setting firm demands for a higher minimum than the turker community would necessarily agree with at this time. If a researcher decides the ethical thing for themselves to do is to pay turkers at rates equivalent to their local research assistants, turkers would likely be pleased. --Gorgeous monarch butterfly 2 September 2014
  • Since the guidelines were published more than a year ago there has been changes in requesters' attitudes to payment. Some Turkers feel like the stated minimum is no longer used as a "minimum pay" and is instead viewed as a "best practice". Now is a good time to discuss this and decide how it should be reflected in the guidelines. -- Light dragonfly 18 December 2015

TODO

  • DoesNotExist needs to be added to Avoiding Duplicates. Also, we might want to make mention of the fact that a Turker takes time to find the HIT, accept it and then return it. This means that time is unpaid since they can't submit the HIT. Any of the methods other than DNE are subject to this waste of time, but using DNE means they can't accept the HIT and therefore don't waste as much time on it. --Dark bird of paradise 30 August 2014
  • Add a template for reporting the use of MTurk in academic articles. This could instruct researchers to report the number of HITs, the number of Turkers, how long their tasks took, how much they paid, etc.
  • Consider removal of this paragraph: "If you do repost, use the same requester account, and the same or very similar HIT title, HIT description, and preview content when reposting, if at all possible. You can indicate in the description and preview text when it was previously posted (e.g. 'If you took this study in March 2014 or June 2014, please don't try to retake it.'). In addition to situations where several weeks or months have passed between postings, it may occasionally be necessary to take down and repost a HIT to make changes to settings such as its pay or qualifications, if you unintentionally didn't use optimal settings the first time."
  • Turkers would have to use a script or keep detailed records to have access to this information, so it's not a good way to attempt to screen out repeats (especially in regards to the "e.g."). Amazon only gives workers access to 45 days' history in their accounts, and it takes too long to search through completed work to have access to this data. The only reliable way to screen out repeats is through other methods; this might mislead requesters into thinking workers can figure it out on their side based on description, HIT title, requester name, previous posting date, etc. -- Obedient otter, February 1, 2016
  • Create guidelines for all general requesters, and not specific to academic requesters. This would be easiest and most convenient if attached to the academic requesters guidelines, but on a separate page (URL) of its own. The current pay by requesters is decreasing over time when it should be increasing. Having a page directed at all requesters would be a good resource for every requester to gather updated information, and stress that minimal pay constitutes coercion.
  • Create a template for consent forms on MTurk. It could be something like: fill out this form about your study and when you submit it Dynamo will give you a Dynamo approved consent box to copy past to the top of your HIT. We could put in information that both Turkers care about and IRBs cares about. Such as: how long do you expect this task to take, how much are you paying, (we could use that to calculate an hourly rate and put that in automatically), what is the email address and phone number of your university’s IRB (we could also pull that in automatically based on their institution), etc.