Skip to main content

Reading the output

Downloading covfee annotations​

Covfee annotations can be downloaded from the admin panel, either for the complete project ("Download results") or for a specific HIT using the buttons on the HIT's row.

About continuous annotations​

Annotation resolution​

Covfee records continuous annotations in a fixed-size array with a configurable resolution via an $fps$ variable (frames or annotations per second). Every index of this array is associated to a particular media time (eg. video time) via:

$$ i = round(mediaTime * fps) $$

The process to record one data point in covfee is the following:

  1. Covfee reads the current $mediaTime$ (eg. video time) of the media being annotated.
  2. Covfee reads the annotated value. This may be the position of the cursor or whether a key is being pressed.
  3. Covfee calculates the index $i$ of the array corresponding to this $mediaTime$ (see above) and stores the data point in this position.

The $fps$ variable can be set separately for each continuous task.

  • For video tasks, we recommend to set $fps$ equal to the frame rate of the video, as covfee will attempt to record one annotation for every frame of the video (see next section). This also means that no rounding will be done by covfee as every $mediaTime$ will correspond exactly to a frame number.
tip

If $fps$ is not set for a video task, covfee will default to 60fps. This is because covfee cannot read the frame rate of the video on the fly. Make sure to set the $fps$ property of the task for optimal results.

  • For other continuous tasks (eg. audio), fps can be set to any desired value. In practice, however, most browsers will not trigger events faster than 60fps. If $fps$ is not set, covfee defaults to 60fps.

  • In adition to the annotated value, covfee stores the value of $mediaTime$ for every data point. This may be used to test or eliminate the effect of rounding post-hoc.

Skipped data points​

  • For continuous annotation of videos in the browser, covfee tries to record a new data point on every frame of the video (via requestVideoFrameCallback()). Therefore, in principle, continuous video annotations in covfee may have the same resolution as the input video. In practice, however, browsers may skip frames when playing a video, depending on the user machine's speed and load.
info

In our experience it is rare to see more than one or two frames skipped in a row with 60fps video in most modern machines.

  • For other types of media covfee tries to record a data point on every screen refresh (via requestAnimationFrame()), which is every 60Hz on most machines. In practice however, the browser may also skip calls depending on the computer's load.

Therefore, it is possible that covfee skips continuous data points regardless of the value of $fps$. It is possible however to know when a data point has been skipped as this will correspond to a row of zeroes in covfee's output:

It is your choice whether to ignore or interpolate missing data points.

info

The way in which videos are processed in the browser may be hard to understand if you are used processing video in Python, C++ or other traditional environments. In the browser, there is no guarantee that a video can be played frame by frame, although it is possible to change a video's playbackRate to speed it up or slow it down. The browser takes care of playing the video as close as possible to the desired playbackRate but depending on the speed and load of the user's machine, the browser may skip frames of the video to meet the playbackRate.