Reading peer-reviewed literature (Theory Testing Articles)

One of the responsibilities of Blog.Net authorship is a review of a contemporary, academic article published in a peer reviewed journal.  For many of you, this will be your first time reading such material.  As such, it might prove a difficult read.  As with any professional publication, academic research is replete with jargon.  Additionally, a number of the topics reviewed (specifically research design) are beyond the experiences of a Bachelors or Masters business degree.  Making the reading even more challenging, most academic authors are rather wordy and somewhat obtuse in their writing style (a charge for which I am certainly guilty).

However, fortunately, while difficult, academic articles are interpretable.  In this post, I will discuss the typical layout of a theory testing (hypothesis testing) article. 

Theory testing research is, perhaps, the most commonly published peer-reviewed article.  With the exceptions of journals that focus entirely on theory development (e.g. Academy of Management Journal), peer-reviewed journals either focus entirely on theory testing, or devote the majority of their publication pages to theory testing articles.  For the business student, these articles will have some familiarity.  The articles will advance a set of hypothesis, gather data and then conduct statistical analysis testing the data's "fit" to the predictions of the hypothesis.  While the format of such articles will vary from journal to journal, there are several common sections you will find:

Abstract - this is generally a very brief overview of the article.  Typically 150 words or less, the abstract establishes the domain of the article and general findings.  When I am searching for articles, I often use a keyword search of abstracts.  If the topic is important enough to make a 150-word limited paragraph - it is probably (but not always) the major element of the article.

Introduction - the opening of the paper serves the purpose of establishing the topic area of study, demonstrating the importance of the topic area, and establishing the specific purpose of the study discussed in this particular paper.

Literature Review - The next section of the paper is typically a literature review of some form. In this section, the author identifies past research findings in the domain area.  Additionally, the author will typically identify a specific theory which would explain the domain of study.  For example, a study of post-acquisition success for mergers and acquisitions is likely to draw from theories of cultural compatibility as a potential explanation for success/failure.  From this reviewed theory, the author will advance several hypothesis which will be explored in this paper.

For the business student, the hypothesis themselves might be a bit confusing.  Unlike the typical course in business statistics, academic articles rarely present the hypothesis in an H0 - null hypothesis and Ha - alternative hypothesis format.  Typically only the Ha variant is expressed and the H0 is understood (but not written).  The hypothesis will be consistent with the theory reviewed and past research findings.

Details of the Study - At this point, the paper typically shifts to the study itself.  You will generally find information on where the data was sourced and how variables were measured.  Business students should recognize that the peer-review process places a knowledgeable editor and two or more topic experts into a blind reviewing format (e.g. reviewers do not know who the author is and the author does not know who the reviewers are).  For the study to be published, the data collection has to have met acceptable standards of practice.  However, the student reader should pay attention to several elements here as these impact the conclusions you should draw from the study.  Some points of consideration are:

1.  What was studied (managers, businesses, industries, etc.) and were there any commonalities of the subjects of the study that might limit the findings?  While the conclusions of a study may lend themselves to broad consideration (e.g. this applies to nearly everyone), often demographics of the subjects narrow the range of people for whom conclusions might be drawn.  For instance, a study of small business owners in France might offer insights into the behaviors of small business owners in other advanced economies, but might not apply to small business owners in developing economies or large businesses anywhere for that matter.

2.  What variables were used and how were they measured?  Nearly every strategy study examines some aspect of business performance, but performance itself is a varied and nebulous topic.  Additionally, in some cases something that improves one aspect of performance might actually hinder performance in another area.  As an example, in my article 'Kicking up dust: growth as an irrational market response,' I point out the conflicted relationship between growth in sales and firm profitability.  Certainly growing the firms sales might be considered as a performance measure and definitely profitability is a performance measure - but increasing one does not always increase the other.  In the article you read, variables were defined and measured a certain way.  Be certain that your conclusions are consistent with the variables - and avoid treading into areas not represented by those variables.

Analysis Section - Generally, the next section explores the analysis of the data.  Here again, the business student may find some things they recognize, but many things they find confusing.  Remember, for the article to be published in a peer-reviewed source, the editor and reviewers (who are knowledgeable about the techniques used) approved the analytic approach used by the authors.  This alone does not make the authors inherently right, but it should allay your concerns about the veracity of the analysis.  I will put together a blog post soon which provides some insight into reading statistical analysis.

Discussion/Conclusion - The final section of the paper is typically the easiest to read for the non-academic.  Jargon is typically thrown out the window and the obtuseness of academic writing in the prior sections is usually downplayed for fairly straightforward explanations.  In this section you will find several elements of importance.  First, the authors will usually discuss what they found - and did not find - in their study.  This is simply a plain-English restatement of which hypothesis were affirmed and which were not supported.  Second, the authors will generally discuss implications.  What do these findings imply for practicing business owners and operators?  Usually the authors themselves will make these suggestions - start with their suggestions to help you frame your own suggestions.  Third, where should research go next?  Some things that were expected to be found weren't, some things emerged unexpectedly, and sometimes the setting itself limits how certain we can be of the conclusions (I touch on this in pt. 1 of Details of Study).  The authors will generally frame these limitations as directions for future researchers - given what we know in this study, what remains unknown or should be clarified?