Building a BSc Thesis 1: Representative Experimental Design

I'm running a student-initiated project for my undergraduate thesis next academic year, where I'm hoping to strap on some eye-tracking glasses on volleyball players while they perform variations of a spiking task. But how did I even come up with this idea in the first place? Over the next few weeks, I'll be taking you through some of the key papers that inspired the project.

To start, I knew I wanted to do something in volleyball. After all, I had spent the past 2 years working as a mental skills coach and skill acquisition consultant for various volleyball clubs, and had not only built a solid network of connections, but also grown quite fond of the sport. Next, I knew that I wanted to use eye-tracking glasses in my dissertation project. Not only was the technology available, but I also had access to a supervisor who was an expert in conducting such research. At this point, it was a no-brainer to capitalise on these currently available resources and expertise.

So, we now have the sport (i.e., volleyball) and the main dependent variable-measuring apparatus (i.e., an eye-tracker measuring gaze behaviour). But what kind of tasks would I want participants to complete? To answer this question, we turn to a paper by Dicks et al. (2010), who examined gaze behaviour differences in football goalkeepers under different task constraints.

Internal and external validity

Dicks et al. (2010) start the paper by recognising some limitations of research designs in psychology. Generally, researchers tend to oversimplify their research designs in such a manner that prioritises internal validity over external validity. Now, that's a bit jargon-heavy, so let's break down what these terms mean. Internal validity refers to the extent to which you can confidently establish a causal relationship between variables in your study. In other words, it is how strongly you can assert that a dependent (i.e., measured) outcome was preceded and caused by an independent (i.e., manipulated) variable.

On the other hand, external validity refers to whether or not the results from the current study can be generalised to other contexts and populations. Hence, another way to phrase the original statement above is: in general, researchers employ research designs that strongly establish causal relationships within the study, but that fail to produce results that adequately generalise to the population of interest. 

Video simulations vs. in situ conditions

One way to achieve internal validity is by maintaining high levels of experimental control. In eye-tracking paradigms investigating perceptual expertise in athletes, this is typically done by providing participants with 2-dimensional video simulations in research laboratories instead of exposure to real-life and natural (i.e., in situ) conditions in their usual performance contexts. Think about how this might help with internal validity. Compared to the more unpredictable in situ conditions, pre-recorded videos are generally less messy and allow us to control for potential confounding variables -- this is an important step to eliminate other alternative accounts of the causal relationship observed.

Why might such a reliance on contrived video simulations pose a problem for the external validity of results? The answer lies in another type of validity introduced by Egon Brunswik (1956), that is, ecological validity. Ecological validity refers to the extent to which an experimental task represents the real-world setting it is trying to study. In many ways, the ecological validity of a research design is linked to the external validity of experiment results -- ecological validity is a necessary condition to achieving external validity.

Let's apply this to eye-tracking research. In real-world contexts, athletes are exposed to and surrounded by information from a 3-dimensional world. This is a nontrivial contrast with the frontal, 2-dimensional information presented by video simulations. In Brunswik's words, video simulations are not representative of real-world situations that athletes typically find themselves in. Consequently, it is not difficult to see how results from video simulation paradigms might not be generalisable to their relevant performance contexts. 

Response types

Dicks et al. (2010) point out another way in which eye-tracking paradigms might lack ecological validity. Often, for the sake of feasibility or because the research has a sole focus on perceptual abilities, participants are frequently asked to give verbal responses or indicate their movements with a joystick.

Now, if you believe that athletic performance is predicated on a sequential process starting from perceptual anticipation and that ends in action production, then there might not be any issue in adopting such a paradigm. On the other hand, if, like the authors, you subscribe to Gibson's (1979) ecological psychology, then you might recognise the importance of studying organism-environment relations. Specifically, Gibson argued that perception is primarily of affordances, that is, opportunities for action offered by the environment that are relative to the action capabilities of the actor.

Put another way, you can't study perceptual expertise divorced from the action and movements that perception is supposed to guide! When you get participants to respond in non-action-relevant ways, there is no way to tell whether the gaze behaviour patterns you observe are actually what happens in the performance context, or if they are just an artefactual result from your experimental design. 

Potential explanations for response type differences

In fact, there is neuroscientific support for different types of brain processes involved in various response types. Here, scientists have argued for a dual-stream hypothesis of visual processing in the brain (Milner & Goodale, 1995). Briefly, the visual cortex is thought to have two separate streams (see Fig.1): the dorsal stream is responsible for the visual control of goal-directed movements (i.e., vision-for-action), while the ventral stream is responsible for perceiving object information (i.e., vision-for-perception).

Fig. 1 The two visual systems (source: link)

The argument here is that both streams are sensitive to different types of information, depending on the task demands. If only simple verbal responses are required, there is a heavier reliance on the ventral vision-for-perception stream, and the participant detects and uses a certain informational source (e.g., allocentric information) to support this. On the other hand, if complex and representative movements are required, then the dorsal vision-for-action stream picks up the slack and makes one more sensitive to different sources of information (e.g., egocentric information).

I will caveat the above by mentioning that it is likely a very simplified view of the actual mechanisms that take place. For those who have been reading my other posts, you'll also know my views about functional localisation in the brain (spoiler: I believe you can't ascribe independent functions to spatially separate brain regions!). That being said, I think the above still serves its purpose in providing support for why we should reconsider the methods used in conducting psychological research, and more specifically, in perceptual-motor research employing eye-tracking paradigms.

The paper (actually)

With the background of Dicks et al. (2010) out of the way, let's take a look at the experiment they ran in response to the methodological issues outlined. Here, they looked at how the gaze and movement behaviour of goalkeepers varied as a function of the conditions in which they faced a penalty taker. Conditions varied across two factors, namely stimulus (e.g., video simulation vs. in situ) and response (e.g., verbal vs. joystick vs. simplified movements vs. interceptive movements). This led to a total of 5 conditions: video simulation-verbal, video simulation-joystick movement, in situ-verbal, in situ-simplified movement, and in situ-interceptive movement. 

While there is quite a lot of data to unpack, we're going to focus on one key result (see Fig. 2). Here, the authors found that under limited movement conditions (i.e., every condition aside from in situ-interceptive), goalkeepers spent more time fixating on the kicker rather than on the ball. On the other hand, goalkeepers spend roughly equal amounts of time focusing on both the kicker and the ball in the full interceptive condition.

Fig. 2 Cumulative mean percentage viewing time on body location vs. ball (Dicks et al., 2010)

More specifically, those in the interceptive condition shifted their fixations from the head of the kicker to the ball quite early on during the penalty run-up. Meanwhile, goalkeepers tended to fixate more on the kicker's kicking and non-kicking legs as the penalty run-up evolved, only fixating on the ball during the moments right before ball-foot contact.

Implications for my undergraduate thesis

The results from Dicks et al. (2010) clearly show how various experimental task constraints can elicit very different gaze behaviour patterns from participants. In particular, if the experimental task is not representative of the real-world task to which we want to generalise the results of the study (i.e., the experimental task lacks ecological validity), then the data obtained serves little purpose in terms of theory advancement and practical application.

So, how has this study influenced my thinking towards my final year project? Here, I have decided to make the difficult but more exciting decision to embrace the messiness of real-world settings and to conduct my study in a fully representative way. That is, I will not be using any video simulations, whether through large screens or virtual reality headsets, and will be conducting my data collection with participants engaging in an in situ task. Additionally, I will be getting them to perform full, complex, and natural (interceptive ball) movements instead of more conventional verbal or joystick responses typically required of participants in eye-tracking research.

So, I will be conducting an in situ eye-tracking experiment with volleyball players. This was something I vividly remember expressing as a non-negotiable to my supervisor way back in February. I was initially hesitant to fully embrace an in situ design, given the logistical difficulties in running one efficiently. However, conversations with different renowned researchers in the field made me realise that my undergraduate thesis might actually be the perfect opportunity to tackle such challenges. If anything, it provides a safe space for me to fail without the fear of any high-stakes repercussions.

Concluding remarks

At this stage, I've only outlined the type of study I will be running. What kinds of questions will I attempt to answer? What are the conditions that I will be creating? These will be answered in due time as we look at the other key studies that informed my final research question!

References

Brunswik, E. (1956). Perception and the representative design of psychological experiments. In University of California Press eBooks. https://doi.org/10.1525/9780520350519

Dicks, M., Button, C., & Davids, K. (2010). Examination of gaze behaviors under in situ and video simulation task constraints reveals differences in information pickup for perception and action. Attention Perception & Psychophysics, 72(3), 706–720. https://doi.org/10.3758/app.72.3.706

Gibson, J. J. (1979). The ecological approach to visual perception. Houghton Mifflin Harcourt (HMH).

Milner, A. D., & Goodale, M. A. (1995). The visual brain in action. http://journalpsyche.org/files/0xaa48.pdf