The Multi-Site Study: An Innovative Research Methodology
by
Josée Audet and Gérald d'Amboise⁺

Abstract

The Multi-Site Study is a qualitative research approach that we designed to gain an in-depth knowledge of an organizational phenomenon that had barely been researched: strategic scanning. It combines several approaches to case study research, borrowing from the positivist tradition, the interpretative approach and the qualitative research corpus. It involves the observation and analysis of several sites using namely cross-case comparisons and explanation building techniques to analyze data. The following report primarily explains the thought process that led to the research decision, a description of the process itself is then presented, followed by an illustration and discussion of the results obtained and finally, a note of reflection on the entire experience.

Qualitative research approaches have traditionally been favored when the main research objective is to improve our understanding of a phenomenon, especially when this phenomenon is complex and deeply embedded in its context. Its many methodologies and techniques have helped researchers get a better grasp of a variety of management situations. Qualitative research has now grown into a wide domain, having evolved much beyond its original scope of qualitative data collection. However, a consensus has yet to be reached to determine the exact qualitative research boundaries and the main components of a qualitative research design (Lee, 1999). There exists few roadmaps with detailed instructions to guide the researcher through this methodological maze. For some researchers, such ambiguity can constitute a source of anxiety. However, some others will view it as an opportunity for innovation, that is, an opportunity to "break the mold" and conceive a research strategy that will meet the researcher's specific needs and objectives.

Understanding a phenomenon that has barely been researched requires a qualitative approach that is both adaptive and innovative. Scanning behavior of small firms, the phenomenon that we had set out to study, was exactly the type of research object that required such an approach. More precisely, scanning activities performed in small and medium-sized enterprises ("SMEs") had not been systematically studied and this area of research was very much underexplored. Scanning is defined as the collection, dissemination and interpretation of information related to a firm's environment. Taking the road less traveled, the decision was thus made to create a unique research approach. This paper describes the Multi-Site Study, a qualitative research approach specifically designed to gain an in-depth knowledge of strategic scanning activities performed by SMEs. The report primarily explains the thought process that led to the research decision, a description of the process itself is then presented, followed by an illustration and discussion of the results obtained and finally, a note of reflection on the entire experience.

An Illustration Of The Multi-Site Study

1.0 Genesis Of The Project

This research started the same way as most research projects: with only a vague idea of the topic to be investigated, the depths of the research still to be discovered. In this case, the topic of interest was the strategic scanning activities of SMEs. In his seminal work on scanning, Aguilar defines this concept as the gathering of information "about events and relationships in a company's outside environment, the knowledge of which would assist top management in its task of charting the company's future course of action" (1967, p. 1). Our definition of the concept was broader, as it also included the activities pertaining to the interpretation and dissemination of information. For example, going to a trade show to keep abreast of current developments in the industry is a scanning activity. In our perspective, sharing the information gathered at the show with other members of the organization and giving it meaning are also strategic scanning activities.

As literature on scanning activities progressed, the idea was further refined and a general research question was formulated to help focus the readings: "Which characteristics of strategic scanning activities are associated with the success of SMEs?" After reading extensively on scanning, a decision was made concerning the research strategy. The literature review revealed that few studies had been done among SMEs on the topic of scanning activities. Furthermore, most of these studies were of a descriptive nature. No grand theory had yet been uncovered to explain the relationship between scanning activities of SMEs and their performance. At best, associations had been found between various scanning dimensions and performance. Testing these developing theories would essentially contribute to knowledge on the strategic scanning activities of SMEs. However, it also became apparent that such knowledge could be further enhanced if a more open stance was adopted, that is, if we were to let findings emerge from the field. This realization triggered the decision to break new grounds.

Before designing a "road map", priorities for the research methodology were established. Primarily, the research design should allow concurrently for some theory testing and theory building, which implies that both a deductive and an inductive logic are to be followed at different phases of the research. The reason guiding this choice was such that the research would be based on the few previous findings while remaining open to the new information and understanding of the phenomenon that were likely to emerge from the field. Although qualitative research lends itself to both theory testing and generation (Lee, 1999), a design combining both is not a common occurrence. Little guidance could thus be found in the literature. Secondly, the desired design was one that would combine flexibilty and rigour, two qualities often irreconcilable. The realization was that it would be safer going into the field having translated thoughts into a theoretical framework and drafted a well structured research protocol. These fears of having a lack of direction and focus may well have been a vestige of our positivist upbringing. On the other hand, it was not necessary to unduly restrict the endeavour; flexibility was required to stay as close as possible to the phenomenon of interest.

After having carefully read the works of previous authors, especially those of Yin (1993, 1994) and Eisenhardt (1989), it was decided to combine several approaches of case study research and merge them into a research strategy that would be referred to as the Multi-Site Study. As will be further described and explained, this strategy borrows from the positivist tradition, the interpretative approach and the qualitative research corpus. It involves the observation and analysis of several sites using cross-case comparisons to analyze data.

2.0 Theoretical Framework

Following a thorough literature review, a theoretical framework was developed using the only three scanning dimensions that have been linked to the performance of the firm by previous authors (see Figure 1). These dimensions are: 1) the intensity of scanning, 2) the level of integration of the information collected through scanning activities to the strategic decision process, and 3) the level of fit between the intensity of scanning activities and the level of strategic uncertainty in a firm's environment. Specific research questions were formulated, together with research propositions that were in fact similar to hypotheses.

It can also be said that this study follows a deductive logic as it refers specifically to the existing theoretical corpus and the initial theoretical framework was to be confronted to an emerging framework as a form of theoretical validation. On this level, the strategy very much ressembles Yin's approach. In this author's view, "good use of theory will help delimit a case study inquiry to its most effective design" (1993, p. 4). He also says that in case study research "theory development as part of the design phase is essential, whether the ensuing case study's purpose is to develop or to test theory" (1994, p. 27). This contrasts sharply with Eisenhardt's position that "research is begun as close as possible to the ideal of no theory under consideration and no hypotheses to test" (1989, p. 536).

Feeling constrained by the obvious limitations of the theoretical framework, several other dimensions of strategic scanning activities that were suspected to impact on the performance of the firm were identified. They are: 1) the impetus for scanning, 2) the time horizon of scanning, 3) the level of structure of scanning activities, 4) the value of scanning to the organizational culture and 5) the information network of the small business owner. These "exploratory" variables were added to the study design to broaden the research perspective. Their choice was partly based on previous studies, intuition and common sense. The variables from both the theoretical framework and the exploratory variables were then defined and operationalized.

3.0 Research Design

A research design was devised in a more traditional fashion, specifying namely:

• sampling criteria (theoretical dimensions guiding selection of cases);
• size of sample;
• measures for all variables;
• data collection methods;
• data analysis techniques.

3.1 Sample

A nonprobabilistic sampling method was favored as generalization in a statistical sense was not one of the objectives. For this reason, "probabilistic sampling is not necessary or even justifiable in qualitative research" (Merriam, 1998, p. 61). Recommended however, is purposeful sampling, that is, selecting a sample from which the maximum can be learned. According to Yin (1994), sample selection should be dictated by a replication logic instead of a statistical one. More precisely, each site (or case) should be considered as an experiment in itself, subsequent sites being used either to confirm or refute previous findings. Sites should therefore be selected if they are expected to yield similar results (literal replication) or on the contrary, completely opposite results (theoretical replication) according to theory. Eisenhardt writes that "cases may be chosen to replicate previous cases or extend emergent theory, or they may be chosen to fill theoretical categories and provide examples of polar types." (1989, p. 537).

In light of the above, it was decided to select cases that represented polar types along two theoretical dimensions: the performance of the firm and the level of uncertainty in the firm's environment. The choice of performance is self-explanatory since it is the dependent variable in the theoretical framework. The choice of the second dimension is based on previous empirical findings linking the level of uncertainty in the environment to certain aspects of strategic scanning. By selecting extreme cases, the aim is to amplify differences that may exist between types of cases, thereby making these differences easier to observe. When comparing findings across homogeneous cases (e.g., pairs of high performing firms or pairs of firms evolving in a highly uncertain environment) similar results are expected (literal replication). On the other hand, when comparing findings across different types of cases (e.g., high performing firms versus low performing firms), it can be expected to find opposite results (theoretical replication).

For obvious reasons, the sample size in a multiple-site study cannot be large. Any sample exceeding ten cases would indeed make it virtually impossible for the researcher to analyze adequately the staggering amount of data to be collected. This is even more so in this particular project where the context dictated the use of only one investigator (study done in partial completion of a Ph.D. degree). Eisenhardt (1989) recommends a sample size of four to ten organizations (or sites). It was decided to initially limit the sample size to eight SMEs.

Selecting extreme cases along the performance dimension proved to be a daunting task. The investigator tried to obtain financial information from external sources (consultants and business magazines) before contacting potential respondents, but such information was tentative at best. Indeed, small business owners tend to be very secretive about the financial performance of their firm, especially if they are not doing as well as they wish they would. Furthermore, as SMEs are largely privately-held firms, no public financial information is available. It was thus very difficult to know with certainty whether a firm's level of performance fitted with the sampling criteria before interviewing the owner of the firm.

Selecting sites along the other dimension was much easier. It was assumed that high technology firms engaging in international activities evolved in a highly uncertain environment, as opposed to local firms operating in traditional industries.

The resulting sample of SMEs was as follows:

Prior to entering the field, scales were developed for the selected indicators to operationalize the variables. There are two reasons behind the use of such quantitative measures. First of all, they are likely to facilitate cross-case comparisons that are to follow. As will be explained in section 3.4, comparison between scores is easily done and outliers immediately attract the investigator's attention. Secondly, multiple indicators with scales provide the investigator with more confidence in the validity of the measure. This is especially important since only one investigator performed data collection and analysis, which heightens the risk of bias in the interpretation of data. It is generally preferable to use multiple investigators as convergence of observations enhance confidence in the findings (Eisenhardt, 1989). However, as previously mentioned academic requirements precluded the use of multiple investigators.

3.3 Data Collection

Data was collected through semi-structured interviews with the owner of the firm and one or two employees that are involved in scanning activities. Interviews lasted from one to four hours. An interview guide was used to avoid losing focus and to ensure that all relevant questions were asked. Questions were both closed and open-ended. Indeed, while some indicators required a brief and precise answer, it is also desirable to let information emerge from the field. Respondents were thus given the opportunity to express their thoughts on the topic of interest as freely as possible. Finally, a point was made to verify with the respondents the relevance of the questions in relation to their scanning activities. This was done in order to refine the operationalization of the variables observed in the field, should further studies be done on the same topic.

The following questions from the interview guide will illustrate the point:

Variable: intensity of scanning

Indicator: frequency of attendance to business/social meetings

• "Are you involved in any political, social or business association? If so, how frequently do you participate to such events in a typical month?"
• "Do you consider these events as good opportunities to gather information? If so, what type of information do you usually gather? Can you comment?"

The first set of questions stems from a deductive logic: the purpose is to measure one dimension of a variable from the theoretical framework (intensity of scanning activities). The other set of questions follows an inductive logic with the objective of allowing any relevant information on the topic to surface. More precisely, it was wanted to explore in more depth the nature of scanning activities that took place during business/social meetings and the type of information that was so collected. In addition, verification was also needed as to whether this indicator was a proper measure of the intensity of scanning in a small firm.

Initially, the interview guide was sent ahead of time to the respondent. It was felt that by reading the questions before the interview, the respondent would have time to think about his answers and to generally reflect on the scanning activities performed in his firm. However, this strategy was quickly adjusted as the length of the interview guide had nearly scared away the first potential respondent!

In most cases, respondents allowed the tape recording of the interview. When such recording was not possible, the investigator managed to take notes while listening to the respondent. Notes were reviewed the same day or the day after the interview and the within-case analysis (see next section for description) was performed as soon as possible, while the information concerning the case was still fresh in the mind of the researcher. Even when the interviews were taped, the investigator tried to do the individual case analysis shortly after meeting with the respondents. The taped interviews were not retranscribed since it would have been much too time consuming and expensive. Instead, tapes were carefully listened to over and over again, notes being taken along, together with citations from the respondents. As a framework in which to place and categorize data already existed, the task was made much easier (as will be explained in the following section).

3.4 Data Analysis And Results

Eisenhardt (1989) recommends starting data analysis with an in-depth study of each individual site, this first step being called "within-case analysis". This entails sifting through all the data, discarding whatever was irrelevant and bringing together what seemed most important. The idea was to allow the most significant observations to emerge from all data gathered in the field, while reducing the volume of data. To facilitate the cross-case analyses that were to follow, all eight individual cases were written following the same format: a brief introduction describing the organization and its business environment; a detailed description of the strategic scanning activities of the firm followed, each variable and each indicator at a time, starting with the three variables from the theoretical framework; then the exploratory variables were presented and finally, the variables that had emerged from the field. At the end of the report was a summary presenting the scores obtained on all variables. This proved to be the easiest way to put order in the vast amount of data gathered during the interviews. The individual cases varied in length from 20 to 45 pages. As Eisenhardt (1989) suggests, such a preliminary analysis is helpful to develop an in-depth understanding of each case before moving on to the next level of analysis.

The second step of the analysis consists of a cross-site search for patterns. Using Eisenhardt (1989) as a base, a unique methodology was developed to structure this type of analysis. The cross-case search for patterns was first executed along the performance dimension. To begin with, firms of similar performance levels were grouped together. Essentially, the result was a sample divided in one group of four (4) high performing firms and another group of four (4) low performing firms. The organizations were then paired within their group and iteratively compared to each other in order to identify similarities and differences among them. Similarities were retained to form what was termed a "configuration" of the pair having been compared. Configurations were then paired and compared using the same logic, leading to a final set of two configurations (see Figure 3 for an illustration of the cross-case analysis logic).

This search for patterns was facilitated by the fact that most variables had multiple indicators measured on quasi-interval scales: the differences and similarities in scores were really apparent. Such comparison between scores was likely to reveal the existence of underlying phenomena, which were then investigated in more depth with the qualitative data in hand. In other words, quantitative measures were useful to attract the researcher's attention to underlying phenomena, while qualitative data coloured and enriched the interpretation of such phenomena.

The idea behind all these comparisons is to identify strategic scanning characteristics or practices shared by all high performing firms and some others shared by all low performing firms. These bundles of scanning dimensions were to form a configuration of variables and relationships exposing the scanning activities of the said group of firms. A final comparison is to be made between these two configurations that would thus result in those characteristics of strategic scanning activities that are truly unique to high performing firms to emerge. In order to be unique to high performing firms, a characteristic has to be shared by all high performing firms and be found in none of the low performing firms. These scanning practices that are unique to the high performing firms are presumed to be positively associated with the success of the organization. As a form of validation of the initial theoretical framework, these scanning dimensions or practices unique to the high performing firms are to be compared to the initial framework. If need be, the framework is to be adjusted to take into account findings that emerged from the analyses.

In light of these results, the decision was made to pursue a cross-case analysis along the performance dimension but with a split sample. More precisely, two separate sets of analyses were performed, one for high technology firms and a second for SMEs from the traditional sectors of the economy. The new sets of cross-case analyses led to very interesting results, as several scanning dimensions revealed to be unique to the high performing firms of each group. To illustrate the findings, the emerging configuration of the scanning characteristics of the better performing high technology firms is reproduced in Figure 4 (see the emerging configuration of scanning practices of low tech high performing SMEs in the Appendix). It is worthy to mention that the variables included in the emerging framework depicted in Figure 4 stemmed mostly from inductive logic, that is, they were not part of the initial framework. Some were exploratory variables while others were only brought to the researcher's attention during the field investigation.

To obtain even more meaning from the data, a third level of analysis was put in place: an "explanation building" analysis was performed. This mode of analysis consists of explaining a phenomenon by stipulating a set of possible causal links about it (Yin, 1994). Yin suggests to begin such an analysis by taking the data collected from a first case to build a logical sequence of events explaining the case outcomes. The hypothesized set of events is then verified in a second case. If it is confirmed, the researcher proceeds with a third case, and so on and so forth. If at any point in the process the hypothesized explanation does not hold, an alternative explanation has to be developed and verified again until one holds with all the cases.

The procedure was slightly different since those variables that were unique to the high performing firms of each subgroup (high tech and low tech SMEs) had already been identified. In other words, there was no need for replication at this point since the emerging frameworks were the result of prior replications. A tentative explanation of how scanning practices may have contributed to SMEs' success was developed based on: 1) the understanding of the phenomenon as observed in the field and expressed in the emerging framework and, 2) theory and empirical results pertaining to scanning. The idea behind such an analysis is to elaborate an explanation that will be both congruent with reality and theoretically sound. Being able to build such explanation, that holds true across all cases is a form of validation of the plausibility of such explanation.

An explanation building analysis was performed for both groups. To illustrate the nature of this type of analysis, the results pertaining to high technology SMEs are reproduced in Figure 4, following the emerging framework.

After having performed the cross-case analyses for both groups, it became evident that many scanning characteristics that were shared by high performing firms had not been selected to form the emerging framework. Indeed, the replication logic required that all scanning dimensions that were not unique to the high performing firms to be discarded. The fear was that the rigour of this logic might have somehow truncated reality. Indeed, it is quite possible that some of the characteristics that were not included in the emerging framework contributed to the firm's success, even though they were not a guarantee of high performance on their own. The belief is that the amalgamation of all scanning characteristics shared by high performing firms, unique or not, leads to their success. The scanning dimensions that are found in the emerging frameworks may have been the most significant, but to be truly effective they had to be supported by all these other scanning practices shared by high performing firms but that were also found in some lower performing firms. Consequently, two additional frameworks of strategic scanning activities (high tech and low tech SMEs) were developed, taking into consideration all scanning dimensions that were shared by high performing firms (whether they were unique or not). This resulted in a rich and detailed picture of scanning activities performed by high performing SMEs (see Figure 5 for an illustration of the extended framework of high tech SMEs). This last framework proved to be an extremely valuable tool for the development of recommendations on best strategic scanning practices for SMEs.

As pointed out by Patton, "it need not be antithetical to the creative aspects of qualitative analysis to address issues of validity and reliability" (1999, p.1190). To enhance the quality of our research design, we used several of the tactics recommended by Yin (1997) for case studies (see Figure 6 below).

We were able to collect data from more than one member of each respondent firms, giving us greater confidence in the measures of the constructs. We also obtained information about the firms from consultants, business magazines and promotional material. We thus achieved triangulation of sources and methods triangulation (Patton, 1999).

3.5.2 Internal Validity

We used the pattern matching mode of analysis after having performed the cross-case search for patterns (see Figure 3). Indeed, as a form of theoretical validation the emerging framework (empirically based pattern) was confronted to the initial theoretical framework (predicted pattern). As previously mentioned, we also made extensive use of the explanation building mode of analysis.

3.5.3 External Validity

We iteratively compared and contrasted pairs of firms that were either predicted to be similar (literal replication) or different (theoretical replication), depending on their performance level. Following such a replication logic both strengthens and broadens analytical generalizations.

3.5.4 Reliability

Before entering the field, we had developed a thorough case study protocol. Included in this protocol were all the questions to be asked and the constructs these questions were meant to measure or document. Furthermore, when writing individual cases for each firm studied (what we refer to as "within-case analysis"), we in fact created a case study database. Indeed, we took great care to organize and present the data in a logical and meaningful way.

A Look Back

At the onslaught of this project, two objectives were identified: 1) to be able to test a theory on strategic scanning activities by building on prior research endeavours and, 2) to generate new theory on the same phenomenon based on findings emerging from the field. The first objective has been achieved as all of the initial hypotheses were rejected. Indeed, results clearly showed that the nature of the scanning activities that could be associated to success depended on the level of uncertainty in the firm's environment. In other words, there is no universal set of scanning practices that will consistently lead to higher performance, regardless of the level of uncertainty that prevails in a firm's environment. This supports previous findings that highlighted the influence of uncertainty on scanning activities, but contradicts others where no such relationship was found. In addition, new theories or explanations on the relationship existing between scanning and the performance of the firm were generated. For example, further reflection on the data collected from high technology SMEs led to the proposition that scanning practices, organizational learning and performance may be interrelated: scanning practices would enhance learning, which in turn would contribute to the success of the firm. Granted, the results are tentative as the samples from which they were based on are too small for any sort of generalization but nevertheless, they provide valid directions for future inquiry.

Furthermore, the desire was to base this research on a solid structure, but one that would not hinder the exploration in the field. This was achieved with success. Indeed, the guidance provided by the detailed research design proved to be of great value to the investigation. Furthermore, there was ample flexibility in the design to allow for surreptitious findings to emerge from the field. The fact that all variables forming the emerging frameworks were not part of the initial theoretical framework is a good proof of that.

The research experience allowed for identification of certain limitations in the methodology used, given the nature of the phenomenon studied. The methodology appears to be better suited for the study of a phenomenon where few moderating or intervening variables are expected to have an impact on the relationship between the dependent and independent variables, or simply on the independent variable. This was not the case here as there are indeed countless organizational or even contextual factors that can influence the performance of the firm. As a result, it is possible for a firm to have "superior" scanning practices but still show a low performance because of these other factors. Furthermore, the relationship between strategic scanning and performance is not a direct one. Scanning is expected to provide information that may or may not be used; if it is used, this information may lead to a good or bad decision, which may or may not be fully implemented before ultimately having or not an impact on the performance of the firm. In fact, there may be a considerable time lag between the adoption of "superior" scanning practices and their impact on the performance of the firm. In a study done using the same research strategy (Thibodeau, 1999), the relationship investigated was that of the exporting behaviors of SMEs and their export performance. As the relationship between the two variables was more direct, there was much less "noise" in the results.

It is also acknowledged that with this type of research sample selection is the most crucial decision the investigator will have to make. One objective should be to minimize the influence of external variables. Choosing to incorporate such an external variable to the design (such as the uncertainty of the environment) led to unexpected results. In retrospect, it may have been a better idea to select more homogeneous SMEs with only noticeable differences in their level of performance. Otherwise, the research can end up leading into unanticipated directions and it may become difficult to find a common thread between the sites.

In light of all of the above, it is strongly recommended to fellow researchers to experiment with the strategy depicted here. This research strategy has allowed for the observation of the phenomenon of interest and the analyzation of the data collected from a variety of perspectives. This has truly enhanced the understanding of strategic scanning activities of SMEs. Furthermore, there was enough structure in the methodology to provide guidance and ensure that there was focus at all times. Finally, this endeavour has resulted in new insights and possible directions for future research.

References

Author Note

⁺Dr. Josée Audet is assistant professor at the Université du Québec à Trois-Rivières, teaching management, strategy and entrepreneurship to business students. Her main research interests are related to small business management and entrepreneurship. Correspondence regarding this article can be addressed to Département des sciences de la gestion et de l'économie, Université du Québec à Trois-Rivières CP 500, Trois-Rivières, Québec, Canada G9A 5H7; josee_audet@uqtr.uquebec.ca.

⁺After earning a Ph.D. in business at UCLA, Gérald d'Amboise devoted his entire academic career to the study of small and medium-sized enterprise managerial practices. His teaching of research methodologies in business led him often to question some of the traditional approaches to the study of business practices. Now retired from teaching, he works regularly with doctoral and master students at Laval University in Quebec City and participates in a number of ongoing research and publication projects. Correspondence regarding this article can sent to Faculté des Sciences de l'Administration, Université Laval, Québec, Canada, G1K 7P4; gerald.damboise@mng.ulaval.ca.

Article Citation

      Audet, J., & d'Amboise, G. (2001, June). The multi-site study: An innovative research methodology [50 paragraphs]. The Qualitative Report [On-line serial], 6(2). Available: http://www.nova.edu/ssss/QR/QR6-2/audet.html

Josée Audet and Gérald d'Amboise
2001 copyright

Return to the top of the paper.

Return to the Table of Contents.