Скачать 377.33 Kb.
1. Introduction 5
2. Theoretical Framework 6
3. Developing a Literature Review. Our biggest dilemma 7
3.1 What kind of research plan: qualitative Vs quantitative? 7
3.2 Overcoming the divide 8
3.3 Survey research: let’s talk about tools, styles, techniques, plan, method or methodology? 9
4. Survey Research 9
4.1 Introduction 9
4.2 Definition 10
4.3 General characterization 10
4.4 Survey design 11
4.5 Survey dimensions 11
4.6 Types of survey studies 12
4.7 Survey execution 12
4.7.1 Self-Administered Surveys 12
4.7.2 Interviews 13
184.108.40.206 Face-to-Face Interviews 14
220.127.116.11 Telephone Interviews 14
4.7.3 Questionnaires 15
4.7.4 Focus Group or Group surveys 17
4.7.5 Electronic Surveys 19
4.7.6 Sample Surveys 21
5. Data & data analysis 21
5.1 Data Gathering and Collecting Methods 21
5.2 Choosing the Appropriate Tools and Techniques 21
5.3 Data analysis 22
6. Strengths and limitations of survey research 23
6.1 Strengths 23
6.2 Limitations 23
6.3 “How Easy It Is to Ask the Wrong Question” 24
7. Three survey synthetic examples 25
7.1 Changchit et al (2008) 25
7.2 Aduwa-Ogiegbaen & Uwameiye (2008) 26
7.3 DiMarco, John (2009). 28
8. Conclusions 30
9. FAQ’s About Survey Methodology: The Questions and their Answers 32
10. References 41
11. Other Fonts and References for Further Study 48
Emerging In Educational Research Arena: The Methodological Dilemma in Survey Research.
Supplies To Survey Methodology.
Rui Manuel Guimarães Lima
CIDTFF – Universidade de Aveiro – Portugal
The illustrated study is the result of our work developed in the Curricular Unit of Investigation Methodologies in Multimedia in Education context, from the Multimedia and Education Doctoral Program from Universidade de Aveiro – Portugal.
As a junior researcher, we look/face this opportunity as a triple challenge: first, and not most important, the chance to develop our English skills; in second place, perhaps the possibility to enlarge our public target; and third, we’ll try to bring some critical contributions into the nearby Survey Research as a significant method for all the educational researchers.
Therefore, in last instance, with this paper we’ll provide our critical point of view in a very specific “arena” (Yates, 2004) of educational research: the Survey Methodology.
Key-words: Education, Research, Survey Methodology, Methods, Dilemma, Methodological,
Critical, Educational Technology, Surveys, Interview, Questionnaire, Focus Group, Quantitative Research, Qualitative Research.
The research act it is a process which acquires scientific knowledge; however there isn’t any method totally safe in order to eradicate error fonts in the implementation of any scientific research or investigation project. (Coutinho, 2005: 175)
At this moment, we found a great variety of studies catalogued under the polysemic definition of survey. Probably the most famous examples of surveys are the population census and the public opinion studies. The surveys always start to raise a question or a problem such as “How many?” or, “With which frequency?" occurs a phenomenon where researchers try to find the answer, inquiring a sample representative of the general population. (Stern & Kalof, 1996)
Usually, the surveys are used to analyse the incidence, distribution and relationships between variables, how the study is as they really is, in natural context, without any manipulation (Miller, 1991; Bravo, 1992; Wiersma, 1995, ; Stern & Kalof, 1996; Meltzoff, 1998), and they are always being classificated in function of three basic goals: describe, explain and explore.
We would like to refer that, simultaneously, in our workgroup, we implement a social community in Ning (accessible from here [http://incognita-surveys.ning.com/], denominated Survey in a minute, to give support to the course, to put into practice with our colleagues, and as well as our working Tertullian space interaction.
Finally, we are using a wiki platform (available in http://wikis.ua.pt/mime09/index.php) to compose this study.
2. Theoretical Framework
There are a great variety of surveys, as discussed in the literature, however if we want to systematize the research developed in the field of survey research, we can conclude that there are five essential survey typologies: (i) descriptive surveys; (ii) explanatory surveys; (iii) exploratory surveys; (iv) cross-sectional surveys; and (v) longitudinal surveys.
The descriptive surveys, as their name suggest, if the objective of the researcher is to discover the incidence and distribution of determined traces or attributes of one given population, without explain them, the survey is purely descriptive of the status quo of one determined situation and is applied at an only moment (Bravo, 1992; Babbie, 1997). He is classic survey or sample survey, where the researcher studies the distribution of the trace (changeable) in a representative sample of the population (sample), it stops of it inferring for the description in the population of which the sample was extracted.
Although the main purpose of the explanatory surveys it is always to describe, some surveys can have the additional intention to try to determine relations between the variables, coming close themselves, in this case, of the post-fact and co-relational plans (Moore, 1983; Babbie, 1997; Black, 1999).
It agrees to relate that the inclusion of clarifying goals in surveys, that, already of itself they produce great volume of data for the descriptive analysis, implies the resource of technique analysis of multivariate statistics (for the simultaneous examination of two or most variables), complex task that requires formation of the researcher and additional cares in the interpretation of the gotten results (Calder & Sapsford, 1996).
In the exploratory surveys the main purpose consists of supplying tracks for future studies and therefore also they are assigned by exploratory studies (Babbie, 1997).
Imagine that we want to make a study on a complex and widespread thematic, for example: “pertaining to school failure: causes and consequences”. Although we have on the subject our ideas, we would be well more sensible not to initiate the main study only on the basis of our proper conceptions, but first to carry through one exploratory survey: on the basis of a little structured questionnaire, we interviewed, for example, 50 pupils with different levels of success, without great concerns to constitute a representative sample, nor to get given very structuralized data. On the basis of this initial information, that does not answer to the basic question, would be prepared with more severity the definitive study (Babbie, 1997).
These are the three great goals that can and must continue a type plan survey, but there are also cases where we accumulate two or same three aims in one same study. Defined(s) the purpose(s), the following phase consists in choosing the format that survey goes to assume: transversal or longitudinal (Babbie, 1997 - this author still relates the combinations of both originating the mixing formats, case of the studies of “parallels samples”, of “context” or the “socio-metric studies”).
In the first case (of cross-sectional surveys), the data are collected at one alone moment in the time in a representative sample of a population, either to describe, either to detect possible relations between outlines/ variables.
Some surveys (longitudinal surveys), descriptive or explanatory, allow the analysis of data throughout a period of time that they make possible to the researcher “to detect and to explain occurred changes in the time” (Babbie, 1997:57). This is the case of trend studies, cohort studies and panel studies.
In the case of trend studies we have a very vast population (for example, university pupils), and of this population are extracted samples for studies at different moments in the time. As a result, although analyze different people at each moment (in general these studies imply long periods of mediation between the measurements what it takes that the university pupil of the first study already had conclude your Course in the second), all the samples represent the population, being then legitimate to say itself of trends. (Babbie, 1997)
Relatively to the cohort studies, the population in analysis is always much more specific than in the previous case. For example, we form a sample of 20 finalists pupils of Mathematics/Education of the Course of 2000 and we evaluate its professional expectations, and passed 5 years, we make a new study with one another sample of 20 populace of this same classroom to inquire the verified evolution: although the two samples are different, both are representative of the 2000 Course.
For its side, the panel studies, always collects the data in the time to the same sample of people, therefore called “panel”. It is the more sophisticated form of survey for making possible the explanation of phenomenon (it is difficult to inquire the panel about the reasons of verified evolution), but it is the most difficult to materialize for depending too much on the loss of respondents throughout the time (Babbie, 1997).
3. Developing a Literature Review. Our biggest dilemma
Until the last moment let us stow in the doubt if we would have to assign, before, this chapter for State of the Art.
In good truth, a rigorous boarding to literature provoked us it emergency of plus this quandary. However, the extremely time expense in the demanding readings, at the same time that in them it tore new horizons, collated us with another type of questions, starting necessarily for this. However, before everything, this complex process, allowed us to conclude that exist, as already previously we point out, an enormous variety of studies catalogued to under the polysemic definition of survey.
For consequence, and such had been the readings that in had sent for many others, understands to adopt the present assignment, structuralizing this chapter in three distinct parts, even though complementary, that we decide to assign them of the present form: (1) What kind of research plan: qualitative Vs quantitative?; (2) Overcoming the divide; and (3) Survey Research: method, methodology, tool, technique, style, instrument or plan?
3.1 What kind of research plan: qualitative Vs quantitative?
The positivist paradigm, also called quantitative, empirical-analytical, rationalist, empiricist, looks for to adapt the model of Natural Sciences to investigation in Social Sciences, using basically a methodology of quantitative identikit (Shaw, 1999; Mertens, 1998; Anderson & Arsenault, 2005,; Latorre et al, 1996; Bisquerra, 1989; Usher, 1996).
The Education research traditionally followed this paradigm and of its successor, the postpositivist (Mertens, 1998), stimulated for the “positivists” ideas, so en vogue in the XIX th Century, been born of the thought of Augustus Comte, considered the first representative of the “positivist school”, through which it would extend to the study of the society the proper methods of Physical Sciences (Oldroyd, 1986:256, quotations marks in the original), that it defended, in the footpath of Bacon empiricism, the priority of the positive stadium of the knowledge based on the comment.
This way to see the world, inspired by a realistic ontology where it intended “… to discover as the things is, and as they work exactly” (Guba, 1990:19, italic in the original), and whose end would be of “… foreseeing and controlling the phenomenon’s (Guba, 1990: Idem), found in the experimental methodology the instrument most efficient for its concretion: if a real world exists - objectivist epistemological base - to discover, that it operates according to natural laws, considers then Guba that the investigator must raise hypotheses and submit them at the empirical confrontation (fake) under rigorous experimental control (Ibidem) .
On the other hand, the qualitative or interpretative paradigm, also assigned for hermeneutic, naturalist or still, more recently, constructivist (Guba, 1990; Creswell, 1994; Crotty, 1998), still have its origins in the last Century (Bogdan & Bilken, 1994). In the opinion of Flick (1998), the period that assigns for “traditional” would have coincided with the birth of the Anthropology and the Sociology of Chicago.
In the decade of 1990 the qualitative paradigm reaches the current phase, assigned for Flick (1998) for 4th moment, that seats in the construction of theories that adapting at very specific situations/problems, a tradition that today congregates great number of adepts and that in literature she was known for grounded theory or “reasoned theory” (Punch, 1998; Flick, 1998; Strauss, 1987; Savenye & Robinsson, 1996).
In accordance with this paradigm, to explain the social and educative world, argues Usher (1996), has that to go to search the deep meanings of the behaviours that are constructed in the human interaction. Investigated and investigator interacts and each one for itself moulds and interprets the behaviours in agreement with its socio-cultural projects, in a process of double search of felt that usually calls “hermeneutical pair”. In contrast with Natural Sciences, in socio-educative contexts the being(researcher) as the object (being) of the research has the common characteristic to be, at the same time, “felt interpreters” and “constructors” (Usher, 1996:19).
The research is, then, as a “horizons fusing”, because, conscientious of its preconceived ideas, - its “horizon” -, the investigating search incessantly the knowledge opening “its” to other perspectives (or horizons) that they are established with yours, complete it and expand them.
3.2 Overcoming the divide
From your perspective as an EdD student, much depends on the type of research you are undertaking, the overall research design and your own preferences, bearing in mind that your time for research will be restricted and your priority will be to link your research to practice in the field of education. Your research is likely to involve a mixture of research methods such as interviews and questionnaires, or observation and keeping a reflective diary, which means that in terms of methodology you are likely to cross what some perceive as the qualitative and quantitative divide. It does not seem that long ago that some researchers challenged positivist assumptions which in turn led to the justapositioning of qualitative over the quantitative methods, or vice versa. However, most researchers nowadays, particularly those in education, tend to be more pragmatic by mixing and matching according to needs and preferences. (Burgess et al, 2006: 56)
Most recently, Roberts (2002), when a trainee researcher, states what while his PhD study was stimulating, challenging and an ideal conduit for exploring knowledge via discussion, argument and defence, he found the paradigm conflicts a bit confusing:
To have the opportunity to explore method, methodology, epistemology, ontology and what may constitute ‘acceptable’ researcher practice is an enriching experience. I had previously heard of interpretivists decrying positivists et cetera, but I was unprepared to find quantitative researchers and lecturers openly dismissing qualitative approaches, and finding dismissals by qualitative researchers of the use of a quantitative approach; such are disappointing and confusing to the trainee researcher. (Roberts, 2002: 1)
Not surprisingly, an increasing number of researchers are against such demarcations. They consider “paradigm wars” a waste of time. Instead, we should be concentrating on the development of better research skills and the understanding of all, not just a few, research methods. Combining research methods is, in any case, an everyday occurrence.
Gorard (2002), for example, argues that the researcher should not ignore or avoid evidence because it might be the ‘wrong’ sort. There may be letters attached to uncompleted questionnaires from correspondents in a survey design, or pencilled comments in the margins of completed ones, which the researcher would not want to ignore, or when conducting interviews in a school, the researcher would not want to ignore school brochures received at the same time.
This is how Jean Barnett introduced her methodology chapter in her final thesis:
Research reports such as this dissertation inevitably make both explicit and implicit claims about knowledge. As a researcher, I have made a number of choices during the progress of the research study about the methodological approach, methods, research instruments, and the site of study. This chapter presents a justification of these choices. The first subsection presents a discussion of research methodologies and the argument that is made for an interpretive stance. This is followed by an overview of the qualitative/quantitative debate.
Subsequent subsections argue for the use of survey and case study approaches, using questionnaire, interview and diary methods. Finally, ethical considerations, including those relating to the site of the study, are discussed. (Barnett, 2004: 48)
Jean, in her opening paragraph, tells the reader what she intends to discuss in this chapter and how this discussion relates to her findings. You can almost see how her discussion on methodology is developed and understand that she has had to tackle quite complex ideas which shaped the research she has undertaken.
3.3 Survey research: let’s talk about tools, styles, techniques, plan, method or methodology?
As we foresaw, the biggest problem with we had confronted throughout this study, and major factor responsible for the exponential and unmeasured increase of time spent in our research, was necessarily the analysis of the underlying polysemy of the survey concept.
Effectively, had been several the theses with that we had to collate. Thus, it has some authors who face surveys as research tools in Education (Verma & Mallick, 2005, : 112; 116). Others classify them as styles of educational research, arriving exactly to support that “it is important to distinguish between design, methodology and instrumentation. Too often methods are confused with methodology and methodology is confused with design. Part Two provided an introduction to design issues and this part examines different styles, kinds of, and approaches to, research, separating them from methods – instruments for data collection. We identify eight main styles of educational research, on the developing field of Internet-based research and computer usage. Although we recognize that these are by no means exhaustive, we suggest that they cover the major styles of research methodology. These take in quantitative as well as qualitative research, together with small-scale and large-scale approaches. As with the previous parts, the key here is the application of the notion of fitness for purpose. We do not advocate slavish adherence to a single methodology in research; indeed combining methodologies may be appropriate for the research in hand.” (Cohen et al, 2007,: 165)
By the way of choosing a data-gathering technique, Gray et al (2007: 127) affirm that researchers may collect data from subjects through face-to-face interviews, telephone contacts, or self-administered questionnaires. All three approaches allow the same options for the kinds of information that can be gathered. What varies is the degree of personal contact used to obtain the data. Also Kalof et al (2008: 121) consider that a survey instrument is a collection of techniques.
With another prism, Landsheere (1993) sustains the survey as the investigation plan more utilized in educational research. Other authors (Marsh, 1982; Mertens, 1998) consider yet the survey not only a descriptive plan mode but also, between another investigation plans, an autonomous method, culminating all this “visions spectre” with the defence and rigorous explanation of what is really a survey methodology (Grooves et al., 2004).
4. Survey Research
Survey research as defined by Kerlinger (1964) deals with the incidence, distribution and interrelation of sociological and psychological variables.
Surveys are administered by mail, over the telephone, in person and on the Internet. Financial costs, time allocated for data collection, sampling frame, the kind of survey questions and researchers’ preferences all influence the mode by which surveys are conducted.
Survey research is one of the most commonly used methodologies in the social sciences. Survey research refers to the set of methods used to gather data in a systematic way from a range of individuals, organizations, or other units of interest. Specific methods may include questionnaires (on paper or online), interviews (conducted by any method; e.g., individual interviews done face to face or via telephone), focus groups, or observation (e.g., structured observations of people using internet access stations at a public library). (Julien, 2008: 846)
Survey research has had a somewhat chequered history in Education and in the Social Sciences. As Pole & Lampard (2002: 89) point out, this has varied from 'uncritical acceptance to irrational distaste' to the extent that more than 20 years ago, Marsh (1982) felt it important to defend the survey against its critics in the social sciences. Today, this is unnecessary, but as Pole & Lampard suggest, Marsh's (1982: 7) definition of a social survey still provides us with a very good starting point, especially her identification of three key features. These also allow us to build on the definitions shown above.
In her view, a survey takes place when: (i) systematic measurements are made of the same set of properties or variables, for each of a number of cases; (ii) the resulting data can be laid out in a form of a rectangle, or matrix, in which the rows correspond to the cases and the columns respond to the properties or variables; and (iii) the intention is to look at patterns in the variables by aggregating information from the cases. (Ibidem )
In Education much research is carried out in the name of survey, or a survey approach, and the published literature reflects eclectic definitions, some of which are particularly inclusive. (Scott & Morrison, 2006: 232)
Consider Denscombe's (1998: 7) definition of survey as 'an approach in which there is empirical research pertaining to a given point of time which aims to incorporate as wide and inclusive data as possible': perhaps the broadest definition in circulation. The key point being made, however, is that survey is an approach or strategy rather than a single method or technique and draws upon a range of methods that include questionnaire, interview and document survey, for example. A less inclusive definition comes from Cohen et al (2000: 169):
Typically, surveys gather data at a particular point in time with the intention of describing the nature of existing conditions, or identifying standards against which existing conditions can be compared, or determining the relationship between specific events.
Perhaps its most familiar usage can be identified in terms of a rather narrower definition in which its most commonly identifiable characteristics are to collect information: (i) from a group of people in order to describe characteristics such as attitudes, opinions, beliefs, aptitudes, abilities, or knowledge; (ii) by asking questions in which the responses or answers to those questions constitute the data of the survey; or (iii) from a sample rather than every member of the population. (Adapted from Fraenkel & Wallen, 2003: 396)
4.3 General characterization
Whichever the format that assume, is characteristic of one survey: (i) the technique of data collection, that can materialize, for example, through the application of a questionnaire where the information is gotten inquiring the individuals, what, in the case of certain personal variables as they are perceptions, attitudes, behaviours or values that can transform the questionnaire into a species of “self report” (Moore, 1983:175), whose validity will go to depend necessarily on the honesty of the inquired ones (Mertens, 1998:105). The questionnaires can coat diversified formats of which they detach the questionnaires, the interviews or the psychological tests (Mitchell & Jolley, 1996); (ii) the constitution of the target group that applies at the study and that it can be the totality of the population or, what it is more vulgar, the constitution of a representative sample of the individuals of the population (Miller, 1991; Stern & Kalof, 1996).
The scheme represented in Table 1 (adapted of Wiersma, 1995,: 178), synthesize the steps that compose a plan survey: