Week 6 Quantitative Research

In today’s SLAT7806 Research Methods lecture, we delve into discussing quantitative research.

So, like last week, we’re beginning the lecture by providing a definition of quantitative research and going through its core features. Then we’re going to focus on the concept of a research variable, which is central for quantitative studies.

In particular, we will particularly focus on being able to distinguish between independent and dependent variables and we will also discuss different levels or scales of variables. After that, we are going to discuss the quantitative research design with a specific focus on experimentation.

Then we will go through the differences between descriptive and inferential statistics, and then finally, we will go through some core strengths and weaknesses of quantitative research. Alright, let’s dive in.

Let’s begin today’s lecture by getting a general feel for and understanding of quantitative research. And we start this by having a look at defining features of quantitative research. So, as we already know quantitative research along with qualitative research, or research families, quantitative research is the type of research that focuses on quantifying relationships between entities or phenomena. So here, think of it as a cause-and-effect type of relationship. So, we’re trying to assert that certain phenomena influence something else, or affect something else.

It is also the type of research that collects measurable numerical data. So, these types of data will be presented as numbers, organized in tables, and visualized as figures and graphs rather than presented in words. And finally, quantitative research analyses the data by statistical means. So today, we’re going to be distinguishing between descriptive and inferential statistics with a particular focus on descriptive statistics. But as the semester progresses, we will find out about other research methods associated with quantitative statistical analysis.

Just like qualitative research, quantitative research is employed in many different academic disciplines. It originated in natural sciences, but then branched out into many other fields, including social sciences, medicine, journalism, and of course the humanities. If we look at linguistics and applied linguistics, quantitative methods are often used to investigate language use (corpus linguistics) and cognitive processes associated with language – (psycholinguistics, language processing and representation in the mind and brain).

Assumptions

So the assumptions that quantitative research makes about reality are quite different from those of qualitative research. So instead of perceiving reality as being socially constructed and subjective, reality is empirical and observable - it can be broken down into parts.

And these parts can be called observations or phenomena or variables (phenomena under investigation). The important part is that variables can be identified, separated from one another, quantified, and measured – either directly or indirectly. Thus, quantitative research asserts that reality is single and objective, which means that we all share one reality and that it can be broken down into parts, those parts are referred to as variables or phenomena under investigation.

The variables therefore can be easily identified, separated from one another, quantified, and measured. For example, to understand errors made by language learners, we need to collect data on all variables that may cause errors and then use statistical methods to see if and how strongly each independent variable (predictors) affects the dependent variable (outcome | result, i.e. errors produced by language learners).

A critical aspect of quantitative research is how to operationalize variables (operationalization means how you measure, categorize, or define variables) So, I would encourage you to reflect on this for a little bit. In this scenario, it is quite easy to quantify and measure years of instruction. For instance, we can investigate a particular group of learners, for example, Chinese language learners and look at how their error rates change with the number of years that they have been studying English. So, in the tutorial that will come after this lecture, we will talk about how to quantify different variables or phenomena. And I invite you to think about it beforehand. A critical aspect of quantitative research is how to operationalize variables (operationalization means how you measure, categorize, or define variables)

Falsification

So, in terms of its nature, quantitative research presents a stark contrast with qualitative research. So instead of being subjective and focused on individual instances, quantitative research is falsification oriented – this means that quantitative research aim is to assess existing models and theories by testing if they are wrong.

Let’s say that we hypothesize that errors are produced at different rates depending on language background – so if someone has German, French, Cinese, or Korean as their first language, and their proficiency level is e.g. operationalized as years of training or as scores on a language proficiency test. We test now want to test this hypothesis by counting errors in essays written by learners from different language backgrounds with different levels of proficiency.

It is also very specific. So it means that quantitative research has a specific particular narrow question at its core. And what is even more interesting is that this research question remains consistent throughout the project. Of course, the researcher may take notes of additional observations, but the idea is that those additional insights do not contribute or alter the research design, they merely serve as inspiration for future research to be undertaken.

So, this is said to be about the linear and focused nature of quantitative research. The logic of quantitative research is deductive rather than inductive, which means that we start with a broader theory. Based on this theory, we define some testable hypotheses. And then we test this by gathering some observations. These observations either support or lead us to reject the hypothesis and therefore support or reject the theory.

The data that is collected within the scope of quantitative research is said to be reliable and replicable, which means that another researcher wishes to conduct a similar study, they should be able to do so irrespective of the context that the study is taking place.

And finally, one of the key aims of quantitative research is to be predictable and generalizable. So it means that if we collect enough evidence, then we’re able to extrapolate the results beyond investigating to make inferences about a broader population.

Features

Alright, so, let me summarize the features of quantitative research. So quantitative research focuses on determining causes for social phenomena, once again, what influences a particular phenomenon what affects a particular phenomenon. Usually, those observations and causes are examined either in a controlled and manipulated environment (using experimentation in a lab, or in natural settings based on corpus data). So for example, in the lab, and as a control and manipulation is exercised by the researcher, who in turn remains objective and detached from the data. So that means that the researcher is a designer and an administrator rather than a participant in their study. Corpus data could consist of transcriptions of dinner table conversations or telephone calls or essays written by university students.

The purpose of the quantitative approach is to predict, generalize and provide a causal explanation for the phenomenon under investigation. obtained data is analyzed systematically and statistically. And data is presented in a technical write-up. Finally, to conclude this section, let me go through some situations in which quantitative analysis is inevitable. Situations So if your study has its main argument based on the counting of things – for example words or occurrences of sth - by definition, you will need to employ a quantitative approach to research. So if you’re interested in investigating linguistic diversity in Australia, essentially your research question is how many different languages are spoken in Australia? So, therefore, to answer that question, you would need to be able to present your data numerically.

Studies that aim at proving that two or more groups of people are different would also employ quantitative research methods. So for instance, we could be interested in looking at whether women use tag questions more frequently than men.

And lastly, any study aiming at showing the two variables are related in some way, or in statistical terms, we would use the term correlated. So here we may investigate things like whether age and success in second language acquisition are correlated. And so an assumption that we could make here is that the younger somebody starts acquiring their second language, the better the outcome is going to be.

Variables

So I have already used the term variables throughout the previous lectures. And I’m hoping that through the context, you’re able to understand what variables mean or have an intuition of what they mean. Today, however, we’re aiming to provide a more formalized definition for variables. So, quantitative research is all about measurement.

And so, therefore, variables are units of observation that vary in our research, and the ones that get measured and quantified. So variables to be investigated, in other words, to be paid attention to in a particular study design are determined by the research question and the study aim.

So if we look at the age effects on heritage language maintenance in immigrant communities in Australia, we can determine that age and language maintenance are the two variables that are mentioned in the question. There is no indication that for example, gender or literacy are also of interest.

So, therefore, participants in our study will be grouped and compared based on their age, and not based on their gender. And we will develop a tool that measures and quantifies the language maintenance and not their literacy skills. Your relevant variables are said to be beyond the scope of a particular study. However, they are not useless as they can serve as the inspiration for future research. Indep and dep. variables One of the key distinctions that quantitative researchers make between variables is independent and dependent variables. The independent variables are also referred to as predictive variables, and it is useful to think of them as the cause in a particular relationship. dependent variables are also referred to as outcome variables, and they are the effect of the relationship. So as independent variables represent a phenomenon that is under investigation.

The Independent or predictive variables represent the external factors that influence the investigation outcome. These are the variables that are selected and systematically manipulated by the researcher to determine if or to what extent they will influence the dependent variable.

The dependent variable is the central variable that is being investigated. And that is affected by the independent variables. That’s the variable that we are measuring. So if we’re asking how age affects heritage language maintenance, then age acts as an independent or predictive variable, and heritage language maintenance is something that we’re going to be measuring.

So that’s our dependent or outcome variable. If we’re looking at how task difficulty affects test scores, the task difficulty is an independent variable, and the test scores are the dependent variable. It is entirely possible to have multiple independent and dependent variables in a single study. However, what it means is that our research design is going to be more complex, and it will require more sophisticated statistical tools to be able to analyze our data.

The expected relationship between the independent and dependent variables is formulated through a hypothesis. And if you recall, this relationship can be formulated as directional or non-directional. And the distinction between variables has to do with different levels of measurement. And here we focus on distinguishing between categorical and numeric or continuous variables.

Variable scales

Let’s turn to different types of variables or variable scales. So we’re going to start by discussing the categorical variables. So basically, these have to do with entities that are divided into distinct categories based on the label that they’re assigned. Here we can look at nominal, categorical and ordinal variables.

So nominal variables are distinguished based on the label, but they have no particular order. So for instance, if we look at the nominal variables, we can distinguish between correct versus incorrect responses to the particular question.

If we look at categorical variables, what it means is that a single variable has more than two categories. So we can recruit participants with their native language background being English, Japanese, Portuguese, etc. So once again, we distinguish between different groups based on the label that we’re assigning to those groups, but not based on the order. So if I reordered native language, English, and native language Portuguese backgrounds, it would not make any difference Ordinal categorical levels of variables have an assumption that we have a label and a certain order to the variables. So for instance, we can group students based on this score. So they can get a passing grade, a credit grade, and a distinction grade.

So here, even though we can determine an order of variables, the interval between those variables does not have to be the same. So for instance, if you’re using a scale of satisfaction for a particular cause, the gap between highly satisfied and satisfied may be perceived as being closer than between satisfied and neutral.

Okay, so now we’re moving on to discussing continuous variables. So continuous variables refer to entities that have a distinct score or numerical value. And here will distinguish between interval and ratio variables. So interval variables are represented by things such as temperature.

So here, we also have values that are ordered. But unlike the ordinal variables, we have the same intervals between each point on the continuum. So the distance between one degree Celsius and two degrees Celsius is the same as between two and three. But there are two limitations when we discuss the interval data or interval variables.

The first one is that zero is used arbitrarily. So when we say that it’s zero degrees outside, it doesn’t mean that there is no temperature, it doesn’t mean that the quality of temperature is absent, it’s just that we have assigned this arbitrary number of zero to represent that it’s really cold outside. And then the second limitation is because there is no zero, we cannot make mathematical conclusions about the relationships between different points on those scales.

So for instance, we cannot say that 40 degrees above zero are twice as hot as 20 degrees above zero, it’s definitely hotter, but we cannot say that the relationship is exact, right?

And then finally, as a ratio variables have order the same interval between different points and no limitations. So here we talk about things such as height, weight, or linguistic studies reaction times. So here, zero means the absence of equality.

So if somebody has shown zero milliseconds in their reaction time to a particular stimulus, it means that there was no reaction. And we can also make inferences about mathematical relationships between different points on the ratio scale. So for example, 20 milliseconds over reaction time, are twice as long as 10 milliseconds of reaction time.

So what I’m going to do is I’m going to put in some more summarizing videos that talk about different levels of variables. And hopefully, the more you get familiar with them, the clearer it becomes.

Research designs

So we already know that a variety of research procedures can be employed for quantitative research design, such as instance, interviews, and questionnaires. However, today, we’re going to focus in particular on experimentation, as experimentation is truly at the core of quantitative research. So experiments are research procedures that are controlled and manipulated in a way that we create environments for multiple groups to be compared, or for the performance of a single group to be recorded at multiple quantities. So for example, pre and post-training, one of the groups provide a baseline for comparison, and we call the group a control group.

So the baseline is sort of the expected performance as other groups will be compared against that baseline. And those groups are referred to as treatment experimental or test groups. The experimental setting should reflect the natural environment. So ideally, the procedures that are used in experimental research are representative of what the participants are doing in their real life. So about language studies, those procedures would need to have to do with the production or perception of language. However, at the same time, the materials and procedures are selected and designed in a way that simplifies those real-life tasks. So this allows for precise investigation of the variables in question. So allows us to zero in on a subset of variables, but at the same time, it creates an only naturalistic environment at its best.

So one final point I would like to make in this section is on the distinction between true experimentation or simply experimentation, and quasi experimentation. So this distinction has to do with participant sampling or participant recruitment. into experiments, participants are randomly drawn from a wider population, and also randomly assigned to control and test groups.

So for instance, to test the efficiency of a new socio-cultural awareness training program, we may want to recruit a sample of 50 participants who work in multicultural environments, and randomly assign them to two groups, a group with training and a group with no training. At the end of our experiment, we measure the group’s level of awareness and draw conclusions based on observed differences.

In quasi-experiments, we have pre-existing participants samples or samples, and they may be tested without random selection or assignment to groups. So for instance, if we have the same aim of testing the efficiency of a new socio-cultural awareness training program, we can measure the level of awareness of the teaching staff at the School of Languages and Cultures, the University of Queensland, then administer the training, measure the level of awareness again, and draw our conclusions based on observed differences.

So quasi experimentation is often more practical and reasonable venture experimentation because sometimes we’re interested in the performance of a particular sample.

Corpora

Another very common method in quantitative studies is the use of corpus data. Corpora are digital speech samples (transcripts of conversations, essays, news articles, blog posts, etc.). Corpora aim to provide natural language samples that should reflect either Distinct genres or text types (Specialized corpus, e.g. essays by English learners with Chinese as L1), entire languages (monitor corpus, many different text types in one language), or historical developments of a language (Historical corpus, English texts written from 1100 to 1900).

What makes corpora ideal for quantitative analyses is that corpora provide frequency information about language use and often also provide information about the speakers/writers. Basic questions that can easily be answered using corpora are, for example, How is a phrase used?, How often is a word used in American vs British English? Or How often is a word used together with other words (e.g. Merry + Christmas – co-occurring words are called collocations).

Descriptive and inferential statistics

Now, since quantitative findings are presented numerically, it is important to understand what kind of numerical values we’re interested in, and what other ways of presenting our data, and to do that we need to understand the distinction between descriptive and inferential statistics. So descriptive and inferential statistics differ and what they are doing.

Descriptive Statistics have the purpose of organizing, displaying, and summarizing the collected data. So here the numerical representations include percentages, mean and median values, which are the measures of the center of our distribution, where the values are clustered, and also the values of the range such as the measure of spread, or dispersion.

In terms of visualization, we would use histograms, boxplots, line charts to present descriptive statistics. Inferential statistics, on the other hand, have the purpose of testing theories and hypotheses under investigation by making generalizations about the populations from which the participant’s sample is drawn by applying appropriate statistical tests. So the statistical tests that you might already be familiar with or have encountered in your readings would be t-tests, chi-square tests, and regressions, we will come back to them a bit further down the track this semester. The results of such tests are reported in tables or in the form of statistical parameters with values for effect sizes or significance. The results of inferential statistics are often visualized by effect size plots.

Today, we’re going to focus specifically on descriptive statistics.

So, within the scope of descriptive statistics, we can describe a mean or an average of a particular group. And this measure is useful for continuous variables. So for example, we could use the mean to describe the English proficiency level of our international students, or for example, an average exam score.

The median value is quite similar to the mean value however, it doesn’t represent the average, but the middle value. It is also useful for continuous variables, but when there are outliers or extreme values that are unlike the values in the rest of the participant sample. So for example, here, we could look at house prices in Brisbane salaries, or exam scores if a couple of students received either outstandingly high grades or extremely poor grades, while everyone else received grades that are closer to average. So by looking at the median, we’re trying to mitigate the influence of the outlier values.

We can also look at the range which would be by providing the lowest and the highest value within our sample. Once again, it is useful for continuous variables, but instead of indicating the center of the clustering of the values indicates the spread.

And finally, we can also look at the mode or the value that occurs most frequently within our sample. And that is useful for categorical variables. So for instance, we can look at different students’ native language backgrounds.

In this week’s tutorial, we’ll be practicing producing descriptive statistics for a specific data set. Now, just like the qualitative approach to research, quantitative studies also have their strengths and weaknesses, which means that in some situations that are more applicable than others. So let’s begin with discussing the positives of quantitative research.

Strengths and weaknesses

So first of all, the quantitative approach is going to be especially useful for addressing specific questions about the relatively well-defined phenomenon. So if you think that your research interest lies within those parameters, a quantitative approach is going to apply to the study that you decide to conduct.

Furthermore, quantitative research uses deductive logic, which means that it is more easily perceived and viewed as so-called real science, which essentially means that it’s closer to natural sciences. And it provides stronger empirical evidence, which is grounded in numbers and statistical inferences, than qualitative approaches.

Finally, the fact that quantitative research is objective and generalizable means that provided that the experiment was well designed and that sufficient amount of data was collected, we can reliably draw inferences about the wider population, based on our particular test at sample beyond the context and the setting of the experiment itself. We can also make accurate predictions about the general phenomenon and also offer informed recommendations to promote change and improvement on the weaknesses of quantitative research. So quantitative analysis requires high-quality data in which variables are measured well. So, essentially it means that they’re well controlled and that the values of the variables are accurately representative of differences in their characteristics of interest.

This, of course, can be quite challenging, especially when you’re researching complicated or understudied areas that do not lend themselves too well to being measured with specific variables. Such areas include things like culture, identity, attitude, and motivation because all of them are intrinsically subjective.

Another criticism of quantitative research is that it has a somewhat artificial nature and also a narrow focus of quantitative studies. Finally, we can say that employing statistical methods for analysis is usually perceived as more inaccessible to untrained researchers in comparison to qualitative interpretive analysis, which means that not a lot of researchers are inclined to give quantitative approaches ago.

Key points

All right, so let’s consolidate what we have discussed today. Quantitative research aims to test a specific hypothesis (to see if certain explanations align with reality or not) and they aim to quantify relationships between phenomena – but they can also be used to explore data.

This testing of hypotheses and models is performed by examining the relationship between pre-determined variables under investigation, and this relationship is usually identified as the cause and effect. We’re trying to do that objectively, using numerical values and statistical inferences, which minimizes or eliminates any researcher bias.

And quantitative research aims to establish a general understanding of behaviors and phenomena across various settings and contexts. So quantitative findings are replicable and generalizable, which is one of the key strengths of the qualitative-quantitative approach. However, the quantitative approach may not apply to investigating abstract and difficult quantifying phenomena. That is why it is always a good idea to have both the quantitative and the qualitative components in your study.