Bored or Intrigued - Setting Up a Classification Annotation Project

As detailed in our LinkedIn post, we collected more than 6000 votes from people around the world finding that DALL-E 3 better portrays the explicit sentiment and emotion in this experiment. Below we go through the steps we took to set up and run the experiment, allowing you to replicate ours or configure your own.

Step 1: Generate (or download) the images

We generated images using DALL-E 3 (through ChatGPT) and Midjourney. You can download our images, or if you prefer to generate your own, use variations of the following prompt, selecting between bored or intrigued:

/imagine artwork featuring a young woman at a bar. dark hair, large eyes, full lips, wears an elegant dress. first date,
she is sitting at a small table looking <bored/intrigued>. She is pictured front-facing, medium close-up. classy cocktail
on table. Oil painting style, larger brush strokes in the background. detail on the woman, intricate details and textures
around the eyes and lips. warm but slightly muted colors --stylize 421 --ar 4:5

Step 2: Configure with Rapidata

Setting up and starting an experiment can be done in less than two minutes through our self-serve platform:

  1. Navigate to https://app.rapidata.ai/.

  2. Click on Images as the data type.

  3. Use the carousel, click on Classification and confirm

  4. In the first input, enter the question for the annotators. In this case we use First date, is she bored or intrigued?

  5. In the second input, enter the options to choose from. Click the add button to confirm. In this case She is intrigued and She is bored. After adding the desired options, click continue.

  6. Select where to get responses from. We recommend Anywhere since it is by far the fastest solution. Click continue

  7. If you are not logged in already, you will be prompted to log in (or sign up if you do not have an account) before data can be uploaded. This is easily done through sign-in with Google, but can also be done manually.

  8. Upload all the images, either by dragging or clicking on the upload area. On this screen the name of the experiment can be changed as well as the number of votes (guesses) collected for each image. The 100 credits supplied on signup is sufficient for a total of 1000 votes. So if you are using the 12 images we supplied, you can select up to 80 votes per image. We recommend selecting between 25 and 50.

  9. After updating the number of votes, press Update to save them.

  10. Now you can click on the start button to start running the experiment. After the experiment passes a short approval process, the answers will start coming in - you can follow the progress on this page. You will be notified via email when all the votes have been collected.