Guidelines for Academic Requesters

From WeAreDynamo Wiki
Jump to: navigation, search

About the project

Version 2.0

The guidelines are currently going through a phase of editing to release the second version. When these edits are finalized and agreed upon the guidelines will be frozen again.

"Treat your workers with respect and dignity. Workers are not numbers and statistics. Workers are not lab rats. Workers are people and should be treated with respect." - turker 'T', a Turkopticon moderator

This document consists of this main Guidelines for Academic Requesters page, and several subpages with important additional details, which are referenced with "Read more" links at relevant points throughout this main page:

Use as a shortcut to quickly reach this document.

As of August 21st 2014, the guidelines and all subpages are locked. To learn more about how future changes will happen, see Meta: Maintaining the guidelines

The full guidelines document, consisting of this page and all subpages, can be Downloaded as a PDF. Print/download any individual pages using the options in the sidebar.

Goal: Guidelines that IRB will use to approve responsible AMT research

Plenty of academic research passes through AMT or is about Turkers, but ethics boards (IRBs) who review and approve research protocols often don't know how workers want to be treated. Turkers have collectively authored these guidelines to help educate researchers and let Turkers hold them accountable to a higher standard.

For Turkers: what can you do when these guidelines are violated?

If an academic requester is not being a positive member of the Mechanical Turk marketplace and community, Turkers may want to reach out to fix the situation. If a Turker wants to report concerns about a HIT that may be in violation, they can post details in a private thread (only readable by registered Dynamo members) in the Dynamo forum.

Dynamo members may then:

  • Encourage the worker to email the requester, and to CC if the worker would like copies to be posted in a private Dynamo thread for ongoing feedback. Dynamo can help provide a template email to make sure it's framed well, and provide feedback.
  • If there is no timely response or inadequate response from the requester, the Dynamo admins can help the worker adapt a template email to contact the requester's IRB.
  • If there is no timely response or inadequate response from the IRB, and if it's a serious problem, the Dynamo community helps the worker adapt a template email to colleagues of the researcher, or to other administrators at the university, as appropriate. (Use this option sparingly!)

Resources and template emails for these communications are available here: Resources for communicating with requesters

These are suggestions and resources available for MTurk workers to utilize if and when they choose, not an obligatory plan of action.


Clearly identify yourself to give workers a sense that you are accountable and responsible

Your HIT should include a consent or intro page with the following information:

  • the full name/s of the researcher/s responsible for the HIT's project;
  • the university/organization/s they're affiliated with and its state/country;
  • their department name, lab, project group, etc;
  • a direct line of communication, including an email address to contact the IRB (phone calls may cost Turkers money)

Also, convey as much information as you can in your:

  • requester display name
  • HIT description
  • HIT preview

Why? Workers generally are more willing to take a chance on a requester they're not familiar with (particularly one who hasn't yet been reviewed by any workers on Turkopticon). Academic requesters seem legitimate by virtue of their position. Also, academic requesters are part of a university 'chain of command' with IRB oversight and a means of redressing worker grievances should something go wrong.

What about my privacy? Turkers who want to know (for the above reasons) can often figure out much of this information for an academic requester who doesn't provide it; however, this takes workers' time and effort, and burns their good will.

Example: When a large batch of HITs was posted by a new requester with no Turkopticon reviews and whose only visible identification was their first-name-only requester display name, some Turkers hesitated, trying to decide if it was too risky to do more than a few. When a Turker was able to identify the requester's full name and affiliation with a major university, the Turkers felt more confident to do a larger quantity of those HITs.

Example: Researchers working on spam algorithms did not identify themselves in HITs. Turkers grew concerned that the HITs were coming from spammers trying to bypass filters. Turkers avoided doing the HITs and posted negative reviews and discussion comments.

Provide reasonable time estimates

State up front how long the task is likely to take for a careful person unfamiliar with the task. Know that task experts always underestimate how long it takes for novices to complete a task [Hinds 1999]. Err on the side of overestimation to avoid disappointment and frustration.

Why? Turkers calculate estimated earnings based on time estimates, and their target earnings inform their choice of HITs. If a HIT takes longer than estimated, Turkers may speed through it to keep it to the requester-provided estimate, hurting quality and damaging requester reputation. Read more

Approve work as soon as possible

Set your auto-approval time as short as reasonably possible. 7 days should generally be sufficient. Many requesters approve work in less than 3 days, and some in less than 24 hours. Many workers rely on MTurk to pay bills and manage their cash flow, so timely pay makes a big difference in their lives. Read more

Maintain worker privacy

Don't require workers to provide personally identifying information to complete your HITs. This includes:

  • email address
  • birth dates
  • real names
  • Facebook logins

Don't require workers to register on sites that require this kind of personal information to complete your HITs, or similarly require a Facebook login.

If you don't follow the Terms of Service, particularly in the aforementioned ways that pose potential threats directly to workers, some workers will give your requester account negative Turkopticon reviews with flags for ToS violations, and report your HITs to Amazon. Read more

Abide by AMT Terms of Service

When you established a requester account with Amazon Mechanical Turk, you promised to adhere to Amazon's MTurk Terms of Service (ToS). To conform with these guidelines, AMT academic researchers shall provide their IRB with a copy of the ToS, as a requisite part of submitting their application for IRB approval.

The MTurk Terms of Service include some protections for Turker privacy and systems. See a list of prohibited uses of Amazon Mechanical Turk in the MTurk 'General Policies' FAQ page or in the 'General Policies' section of the MTurk Requesters FAQ page. Note that requiring users to download software that contains any malware, spyware, viruses, or other harmful code is against AMT's Terms of Service. Some workers are willing to download software, but others will refuse as it can be a security risk to their systems. Read more

Ensure conditions for rejecting work are clear and fair

Rejections leave workers with a mark counting against them on their 'permanent record' at MTurk that may take them below a qualification threshold necessary for certain other HITs. Before deciding a rejection is justified, be sure you've considered several factors:

  • State any reasons for which you plan to automatically reject submissions.
  • Test your instructions and attention checks with compensated workers to ensure they are not ambiguous or unclear.
  • Make sure your survey will actually provide the promised completion code to workers who complete it, and that the code is correctly saved in your database. Learn how to do completion codes well
  • Keep lines of communication with workers open through email and forums. Workers run into 'edge cases', particularly in large batch HITs.
  • Don't reject workers solely based on majority rules, even if you use majority internally for your analyses.
  • Reject work only as a last resort. Know how to undo a rejection before you do. After thirty days, a rejection can never be reversed. Don't be in a hurry to pull the trigger.

Example: There have been several situations where requesters wrongly rejected large amounts of workers for 'incorrect completion codes'. The requester was randomly generating the codes and they were not being correctly stored in their database for matching.

Read more about ensuring rejections are fair

Do not block workers to avoid duplicate subjects

Blocks should only be used for bad-faith workers, as they can result in workers being suspended by Amazon. Suspensions of this type are equivalent to a permanent ban in most cases; this simple mistake can cost livelihoods.

Say up-front if you do not want duplicates. However, recognize that workers cannot easily remember whether they participated in your survey several months ago. There are several tools requesters can use when setting up their HITs to make this easier, rather than expecting workers to keep your records. Learn how to avoid duplicate subjects (retakes) fairly

Maintain a responsive line of communication with Turkers

Check the email account associated with your MTurk requester account frequently. Respond to messages from workers as quickly as possible, preferably in less than 24 hours. Visit worker forums to seek advice and find knowledgeable Turkers to vet your HIT.

Read more about how to communicate well with workers

Pay Turkers fairly. They are a workforce, not a volunteer study population

Crowdsourcing workers are a labor force. Many depend on income from crowdsourcing as critical income. Crowdsourcing workers are legally considered contractors and therefore are not protected by any minimum wage laws. When requesters pay a fair wage and treat workers like people, both sides receive positive results.

Pay (at least) community norms of minimum Turking wage

Underpayment of crowd workers is anything less than the current federal minimum wage in the United States.

Since Turkers work independently, they are responsible for their own computers, electricity, taxes, health care, etc. Different workers consider fair pay anywhere from $6 an hour to $22 an hour. Learn why

If your task takes longer than you predicted, you can send workers bonuses to bring the wages up to ethical levels after the fact. In July 2014, a requester did this unexpectedly for workers who took one of their surveys, basing their target pay rate on Washington state's $9.32/hr minimum wage.

Clearly communicate possible bonuses

Explain what the potential amount will be and how to earn it, and how soon workers should expect it to be paid. Pay promptly. Read more

Compensate for qualifier/screener surveys

If you are using qualifier surveys, compensate all those who correctly complete the survey. Read more

Do not experiment with forum relationships for research

Forums only work because of delicate relationships of trust and mutual aid among participants. Sociological experiments such as breaching experiments can sow discord and destroy relationships. Positivist research that attempts to control and measure a forums effects can confuse workers, create anxieties in the community, and drain community energy as members try to make sense of the unusual intervention. To learn how a forum works, talk to administrators about your project, goals, and a plan for creating mutually beneficial research with workers.

Example: One academic experiment simulated requesters with varying ratings in Turkopticon to measure the effects of ratings on worker behavior and outcomes. Turkers found some of the requesters and smelled something fishy but did not know if it was a scam, academic research, vandalism, or something else; through what amounted to at least 50 hours of sleuthing over two days, Turkers across reddit and turkopticon-discuss hypothesized that this was a research project. The researcher wanted to make positivist knowledge claims about ratings, workers, and the economics of Turking, but neither he nor the IRB understood that:

  • simulating fabricated requesters and reviews broke the fragile trust that makes Turkopticon ratings meaningful to workers
  • that worker harm includes not only unpaid wages in AMT, but also the time they spent anxiously trying to track down these mysterious apparitions

Links to other resources on AMT and online research ethics


"Turking is work, even if it is for science, and academic researchers shouldn't assume that people are happy to do it for fun. They should pay and respect people's time." - Dr. Lilly Irani (of Turkopticon), Department of Communication, University of California at San Diego

"What we need to do is teach requesters about the human side of Mturk. Mturk encourages anybody that uses Mturk to think of us as little computing units, not as people." - Project2501 (a Turker)

"Dehumanization is the result of an unjust order that engenders violence in the oppressors, which in turn dehumanize the oppressed. Because it is a distortion of being more fully human, sooner or later being less human leads the oppressed to struggle against those who made them so. In order for this struggle to have meaning, the oppressed must not, in seeking to regain their humanity (which is a way to create it) become in turn oppressors of the oppressors, but rather restorers of the humanity of both. This, then, is the great humanistic and historical task of the oppressed: liberate themselves and their oppressors as well." - Freire's Pedagogy of the Oppressed

"Turkers are people, the work they do might feel like magic at times but at the end of the day we can't forget that they're human beings just like you and me." - William Kyle Hamilton

Signatories: Ratify these guidelines

Workers, to lend the strength of your support to these guidelines, please click on the "Sign" button in the Dynamo signing campaign, to sign with a pseudonym. (You will need to submit a HIT on MTurk to receive a code to register for a Dynamo account, ensuring there is one Dynamo account per worker.)

Researchers, to show your agreement to follow these guidelines in your future research, please send us an email at so we can add your signature. Please email us from your academic email account to help verify your identity.

Signatures from MTurk Workers

The following pseudonyms each represent one individual Turker who has completed at least 100 approved HITs.

  1. Gorgeous monarch butterfly (14 August 2014)
  2. Courageous cockroach (14 August 2014)
  3. Fancy cod (15 August 2014): I support this
  4. Faithful fly (15 August 2014)
  5. Dark bird of paradise (15 August 2014)
  6. Elated sea urchin (15 August 2014)
  7. Lonely wombat (15 August 2014)
  8. Amused hedgehog (15 August 2014)
  9. Jolly otter (16 August 2014)
  10. Terrible cat (16 August 2014)
  11. Funny giant panda (16 August 2014)
  12. Obedient otter (16 August 2014)
  13. Determined firefly (16 August 2014)
  14. Glamorous mollusks (16 August 2014)
  15. Bad cat (21 August 2014)
  16. Elated mongoose (23 August 2014)
  17. Innocent widow spider (24 August 2014)
  18. Outstanding swan (24 August 2014)
  19. Brainy coyote (24 August 2014)
  20. Jolly blue jay (24 August 2014): Yes!
  21. Cheerful panda (24 August 2014)
  22. Elegant peacock (25 August 2014)
  23. Grumpy asian elephant (25 August 2014)
  24. Lovely spectacled bear (25 August 2014)
  25. Bad siberian husky (25 August 2014)
  26. Talented pigeon (25 August 2014)
  27. Wandering bandicoot (25 August 2014)
  28. Black dragonfly (25 August 2014)
  29. Gleaming boa (25 August 2014)
  30. Vivacious widow spider (25 August 2014)
  31. Frightened owl (27 August 2014)
  32. Evil coyote (27 August 2014)
  33. Amused gazelle (27 August 2014)
  34. Stupid basilisk (27 August 2014): I really hope academic researchers and microtask firms become aware of turkers as human beings and respect the work we do. the wages some of them offer are completely unfair.
  35. Vast anaconda (27 August 2014)
  36. Splendid raccoon (27 August 2014)
  37. Gifted green fly (27 August 2014)
  38. Wandering cat (27 August 2014)
  39. Frantic prawn (29 August 2014)
  40. Disturbed cat (29 August 2014)
  41. Inquisitive falcon (29 August 2014): I 100% support this.
  42. Lovely cat (30 August 2014)
  43. Jittery orangutan (30 August 2014)
  44. Angry monkey (30 August 2014)
  45. Elegant owl (30 August 2014)
  46. Jealous tiger (31 August 2014)
  47. Nutty buck (31 August 2014)
  48. Handsome donkey (31 August 2014)
  49. Oldfashioned cray fish (31 August 2014)
  50. Splendid bird of paradise (1 September 2014): I've been using mechanical turk for years now and seen plenty of workers and researchers alike have bad experiences that turn them off from the platform. i hope these guidelines serve as a resource to help improve the mturk experience for all.
  51. Poor camel (1 September 2014)
  52. Innocent scorpion (1 September 2014)
  53. Innocent turtle (1 September 2014)
  54. Poised seal (1 September 2014)
  55. Crazy wallaby (1 September 2014)
  56. Awful blackbird (1 September 2014)
  57. Eager yak (1 September 2014)
  58. Better hippopotamus (1 September 2014)
  59. Lazy llama (1 September 2014)
  60. Hungry chupacabra (1 September 2014)
  61. Jolly bull frog (1 September 2014)
  62. Healthy leatherback sea turtle (1 September 2014)
  63. Vivacious warbler (2 September 2014)
  64. Calm meerkat (2 September 2014)
  65. Nasty dog (2 September 2014)
  66. Comfortable lemur (2 September 2014)
  67. Creepy wolf (2 September 2014)
  68. Glamorous ferret (3 September 2014)
  69. Homely lemming (4 September 2014)
  70. Famous bee (4 September 2014)
  71. Innocent boa (10 September 2014)
  72. Blue vervet (22 September 2014)
  73. Doubtful baboon (2 October 2014)
  74. Beautiful giraffe (6 October 2014)
  75. Mysterious irrawaddy dolphin (13 October 2014)
  76. Worrisome osprey (16 October 2014)
  77. Gifted horse (16 October 2014)
  78. Doubtful chimpanzee (17 October 2014)
  79. Beautiful llama (23 October 2014)
  80. Fierce lama (2 November 2014)
  81. Dangerous leopard (4 November 2014)
  82. Careful owl (18 November 2014)
  83. Repulsive beaver (18 November 2014)
  84. Embarrassed caribou (19 November 2014)
  85. Nutty owl (19 November 2014)
  86. Terrible meerkat (19 November 2014)
  87. Tender grasshopper (19 November 2014)
  88. Graceful spectacled bear (19 November 2014)
  89. Bloody seal (19 November 2014)
  90. Victorious chihuahua (19 November 2014)
  91. Uptight monarch butterfly (19 November 2014)
  92. Confused skunks (19 November 2014)
  93. Cheerful yorkshire terrier (19 November 2014)
  94. Anxious scorpion (19 November 2014)
  95. Perfect diamond back rattler (19 November 2014)
  96. Ugly swordfish (19 November 2014)
  97. Sleepy snow hare (19 November 2014)
  98. Poised jackrabbit (19 November 2014)
  99. Disgusted flamingo (19 November 2014)
  100. Gleaming leopard (19 November 2014)
  101. Lucky lemur (19 November 2014)
  102. Defiant whale shark (19 November 2014)
  103. Clear tazmanian tiger (19 November 2014)
  104. Ugliest chihuahua (19 November 2014)
  105. Precious leatherback sea turtle (19 November 2014)
  106. Mysterious great white shark (19 November 2014)
  107. Frantic buffalo (19 November 2014)
  108. Outstanding drake (19 November 2014)
  109. Curious turtle (19 November 2014)
  110. Ill peacock (20 November 2014)
  111. Lazy vulture (20 November 2014): ٩( ᐛ )و
  112. Black rabbit (20 November 2014)
  113. Eager lobster (20 November 2014)
  114. Grieving opossum (20 November 2014)
  115. Tender green fly (20 November 2014)
  116. Odd king cobra (20 November 2014)
  117. Puzzled velociraptor (20 November 2014)
  118. Cautious seahorse (20 November 2014)
  119. Homely gorilla (20 November 2014)
  120. Powerful african tree pangolin (20 November 2014)
  121. Envious otter (20 November 2014)
  122. Comfortable shark (20 November 2014)
  123. Puzzled peacock (20 November 2014)
  124. Wicked monkey (20 November 2014)
  125. Energetic whale shark (20 November 2014)
  126. Happy giraffe (20 November 2014)
  127. Fantastic penguin (20 November 2014)
  128. Bored swordfish (20 November 2014)
  129. Crazy snow leopard (27 January 2015)
  130. Defeated dogue de bordeaux (27 January 2015)
  131. Enthusiastic wildebeest (27 January 2015)
  132. Expensive slugs (27 January 2015)
  133. Faithful seahorse (27 January 2015)
  134. Famous narwhal (27 January 2015)
  135. Fancy wasp (27 January 2015)
  136. Fierce muskrat (27 January 2015)
  137. Fragile insect (27 January 2015)
  138. Frantic lynx (27 January 2015)
  139. Gifted green poison dart frog (27 January 2015)
  140. Glorious black rhino (27 January 2015)
  141. Graceful rhino (27 January 2015)
  142. Hilarious komodo dragon (27 January 2015)
  143. Homely yeti (27 January 2015)
  144. Jittery vervet (27 January 2015)
  145. Naughty penguin (27 January 2015)
  146. Poised lion (27 January 2015)
  147. Stormy hawk (27 January 2015)
  148. Testy cat (27 January 2015)
  149. Thoughtless dolphin (27 January 2015)
  150. Ugly chameleon (27 January 2015)
  151. Unusual ant (27 January 2015)
  152. Unusual gorilla (27 January 2015)
  153. Upset kangaroo (27 January 2015)
  154. Wandering jaguar (27 January 2015)
  155. Weary urchin (27 January 2015)
  156. Wideeyed porcupine (27 January 2015)
  157. Wild potto (27 January 2015)
  158. Zany dinosaur (27 January 2015)
  159. Zealous starfish (27 January 2015)
  160. Eager black bear (27 July 2015)
  161. Arrogant sea lion (27 July 2015)
  162. Bright crow (27 July 2015)
  163. Hilarious komodo dragon (27 July 2015)
  164. Stormy hawk (27 July 2015)
  165. Wild potto (27 July 2015)
  166. Enthusiastic wildebeest (27 July 2015)
  167. Faithful seahorse (27 July 2015)
  168. Wandering jaguar (27 July 2015)
  169. Fragile insect (27 July 2015)
  170. Fancy wasp (27 July 2015)
  171. Jittery vervet (27 July 2015)
  172. Bored chimpanzee (27 July 2015)
  173. Thoughtless dolphin (27 July 2015)
  174. Disturbed chimpanzee (27 July 2015)
  175. Innocent dormouse (27 July 2015)
  176. Thoughtless monkey (27 July 2015)
  177. Thankful platypus (27 July 2015)
  178. Helpless nautilus (27 July 2015)
  179. Blue grey whale (27 July 2015)
  180. Tender chipmunk (27 July 2015)
  181. Panicky fish (27 July 2015)
  182. Bored green fly (27 July 2015)
  183. Adorable tasmanian devil (27 July 2015)
  184. Defeated dogue de bordeaux (27 July 2015)

Signatures from Academic Researchers/Requesters

  1. Niloufar Salehi, Stanford University, Department of Computer Science (Light dragonfly) (13 August 2014)
  2. Lilly Irani, University of California at San Diego; Turkopticon co-founder (Tense ringworm) (13 August 2014)
  3. Michael Bernstein, Stanford University (Excited iguana) (14 August 2014)
  4. William Kyle Hamilton, University of California at Merced (Lazy urchin) (20 August 2014)
  5. Ali Alkhatib, Stanford University (Tired cricket) (25 August 2014)
  6. Katharina Reinecke, University of Michigan School of Information (1 September 2014)
  7. Nicole Ellison, University of Michigan (1 September 2014)
  8. Jessica Hullman, University of California Berkeley (1 September 2014)
  9. Niki Kittur, Carnegie Mellon (1 September 2014)
  10. Miriam Cherry, Saint Louis University School of Law (1 September 2014)
  11. Emilee Rader, Michigan State University, Department of Media and Information (1 September 2014)
  12. Rick Wash, Michigan State University, School of Journalism and the Department of Media and Information (1 September 2014)
  13. Jeffrey P. Bigham, Carnegie Mellon University. (1 September 2014)
  14. D. Yvette Wohn, New Jersey Institute of Technology, Department of Information Systems (1 September 2014)
  15. Christopher Cox, University of Wisconsin--Madison, Psychology Department. (1 September 2014)
  16. Dr David Brake, Professor of Journalism, Humber College, Dep’t of Media Studies & Information Technology (1 September 2014)
  17. M. Six Silberman, Turkopticon and University of California, Irvine (1 September 2014)
  18. Chris Callison-Burch, University of Pennsylvania (1 September 2014)
  19. Julie Kientz, University of Washington (1 September 2014)
  20. Besmira Nushi, on behalf of Systems Group, Department of Computer Science, ETH Zurich (2 September 2014)
  21. Don Norman, Director, Design at UC San Diego: Think Observe Make, Prof. Emeritus Cognitive Science & Psychology, UC San Diego (2 September 2014)
  22. Maxine Eskenazi, Carnegie Mellon University (2 September 2014)
  23. Sara C. Kingsley, University of Massachusetts Amherst (16 September 2014)
  24. Ayhan Aytes, American University of Paris (18 September 2014)
  25. Pao Siangliulue, Harvard University (24 September 2014)
  26. Kenneth Arnold, Harvard University (30 September 2014)
  27. Tim Daly, Assistant Professor of Marketing, United Arab Emirates University (13 October 2014)
  28. Sarah T. Roberts, Faculty of Information and Media Studies, Western University (13 October 2014)
  29. Rob Semmens, on behalf of the AAA Lab at Stanford University (11 November 2014)
  30. Shamika Goddard, Union Theological Seminary in the City of New York (18 November 2014)
  31. Jamie Hughes, Department of Psychology, University of Texas of the Permian Basin (18 November 2014)
  32. Shona Tritt, New York University, Department of Psychology (19 November 2014)
  33. Ekant Veer, University of Canterbury, New Zealand (20 November 2014)
  34. Tom Cunningham, Assistant Professor, IIES Stockholm University (20 November 2014)
  35. Sebastian Sadowski, Business Administration, University of Groningen (20 November 2014)
  36. Joe Miele, CEO, Academic Survey Publishing (21 November 2014)
  37. Jeffrey L. Foster, Ph.D., University of Western Sydney, School of Social Sciences and Psychology (23 November 2014)
  38. Laina Y. Bay-Cheng, PhD, Associate Professor, School of Social Work, University at Buffalo (23 November 2014)
  39. Hyung Jun Ahn, Associate Professor, School of Business, Hongik University, Korea (23 November 2014)
  40. Catherine Wong, Stanford University (25 November 2014)
  41. Marlon Mooijman, Department of Social and Organizational Psychology, Leiden University (25 November 2014)
  42. Robert Gregg, Queen's University, Belfast (29 November 2014)
  43. Hazel Pearson, Centre for General Linguistics (ZAS), Berlin (2 December 2014)
  44. Sandro Ambuehl, Department of Economics, Stanford University (9 December 2014)
  45. Xiao (Sean) Ma, Assistant Professor, University of Arkansas, Walton College of Business (27 January 2015)
  46. Ayse Zeynep Enkavi, Psychology Department, Stanford University (27 January 2015)
  47. Alexander J. Quinn, Purdue University (27 January 2015)
  48. Cecilia Aragon, University of Washington (27 January 2015)
  49. Long Ouyang, Psychology Department, Stanford University (2 February 2015)
  50. Nicholas Moores, Psychology Department, Stanford University (2 February 2015)
  51. Michael C. Frank, Language and Cognition Lab, Psychology Department, Stanford University (2 February 2015)
  52. M. H. Tessler, Department of Psychology, Stanford University (2 February 2015)
  53. Joshua Introne, Assistant Professor, Media & Information, Michigan State University (4 February 2015)
  54. Raynee Gutting, Department of Political Science, Stony Brook University (15 February 2015)
  55. John Weidner, (20 February 2015)
  56. Mark Cartwright, Northwestern University (27 April 2015)
  57. Utpal M. Dholakia, Professor of Marketing, Rice University (27 July 2015)
  58. Michael A. Cohn, Osher Center for Integrative Medicine, University of California, San Francisco (18 August 2015)
  59. Emily Grubert, Emmett Interdisciplinary Program in Environment and Resources, Stanford University (18 August 2015)
  60. Benjamin Keep, Graduate School of Education, Stanford University (7 October 2015)
  61. David Miller, Stanford University Department of Communication (12 October 2015)
  62. Sunny J. Kim, Geisel School of Medicine at Dartmouth, Dartmouth College (21 October 2015)
  63. Hajin Kim, Stanford Emmett Interdisciplinary Program in Environment and Resources (7 December 2015)
  64. Graham M. Watson, PhD Candidate, Psychology, Walden University (11 February 2016)
  65. Michael Walker, Doctoral student, UTS Business School (22 June 2016)
  66. Steven Dow, Cognitive Science Department and Design Lab, UC San Diego (4 September 2016)
  67. Tracy L. Caldwell, Professor and Chair, Psychology, Dominican University (6 September 2016)
  68. Daniel L. K. Yamins, PhD, Assistant Professor, Stanford University (15 October 2016)