Problem
1. Describe how crowdsourced workers might have been employed to help gather data for The Quant Shop challenges:
• Miss Universe.
• Movie gross.
• Baby weight.
• Art auction price.
• Snow on Christmas.
• Super Bowl/college champion.
• Ghoul pool.
• Future gold/oil price?:
2. Suppose you are paying Turkers to read texts and annotate them based on the underlying sentiment (positive or negative) that each passage conveys. This is an opinion task, but how can we algorithmically judge whether the Turker was answering in a random or arbitrary manner instead of doing their job seriously?