In addition to refereed "target articles," PSYCOLOQUY publishes refereed peer commentary on those articles, as well as authors' responses to those commentaries. This form of interactive publication ("scholarly skywriting") represents the revolutionary dimension of the Net in scholarly communication (Harnad 1992), but it too must be implemented under the constraint of peer review.
The objective of those of us who have glimpsed this medium's true potential is to establish on the Net an electronic counterpart of the "prestige" hierarchy among learned paper journals in each discipline. Only then will serious scholars and scientists be ready to entrust their work to them, academic institutions ready to accord that work due credit, and readers able to find their way to it amidst the anarchic background noise.
How is peer review normally implemented, in conventional paper journals? The journal has an Editor and an Editorial Board. With some journals it is the Editor in Chief, with others it is the Editor in consultation with the Board, or with Action Editors, who selects the referees, usually one or two per manuscript, a third or more consulted if a deadlock needs to be broken. The referees advise the Editor(s) by submitting reports (sometimes anonymous, sometimes not) evaluating the manuscript and making recommendations about acceptance/rejection and revision. The reports are advisory rather than binding on the Editor, who makes the actual decision, but a good Editor chooses his referees well and then for the most part trusts them; besides, it is only the very narrow specialty journal whose Editor has the expertise to judge all submissions on his own. The idea of peer review is also to free publication from the domination of any particular individual's preferences, making it answerable to the peer community as a whole -- within the discipline or specialty. (Interdisciplinary journals always have added problems in achieving peer consensus, and indeed, even with specialty journals, referee disagreement rates suggest that consensus is more than one can expect from peer review; nor is it clear that it would be desirable; Harnad 1985).
In the social sciences and humanities, journals pride themselves (and rank their quality) on the magnitude of their rejection rates. Eighty to ninety percent of submissions rejected is not unusual for the most prestigious journals in these fields. Prestige in the physical sciences and mathematics is not associated with such high rejection rates; indeed, they tend to be the reciprocal of social science rates, and biological, medical and engineering periodicals' rates fall somewhere in between (Hargens 1990). In all these fields, however, irrespective of the prevailing rejection rates, there is a prestige hierarchy among journals, with some known to accept only the best work in the field, and some not much more selective than the unrefereed vanity press that exists at the bottom of each field's hierarchy. It is thought that the lower rejection rates in physics may occur because in this field authors exercise more self-selection in choosing which journal to submit their work to, saving only the best for the elite journals. It is also true that in all fields virtually everything that is written gets published somewhere; in the social sciences a manuscript may be submitted to a succession of lower and lower standard journals until it finds its niche; in physics authors may head more directly for something within their reach the first time round.
Another pertinent feature of this hierarchical system of quality control is that most published work is rarely if ever cited. Only a small percentage of what is published is ever heard of again in the literature. This may be because too much is being published, but it may also reflect the inevitable wheat-to-chaff ratio in all human endeavor (Harnad 1986). As a consequence, a scholar is protected on both sides: There is not much risk that a truly valuable piece of work will fail to be published, though it may not make it to its rightful level in the hierarchy, at least not right away. (Peer review is far from infallible.) On the other hand, it is also safe for a scholar, in this monumental information glut, to let the quality control mechanism calibrate his reading, saving it for only the best journals. Again, there is some risk of missing a gem that has inadvertently been triaged too low, but, given the prevailing odds, that risk is itself low.
I have not described a perfect or ideal system here; only the reality of peer review and the reasonably reliable rank-ordering it imposes on scholarly output. It should be apparent that there is nothing about this system that could not be implemented electronically, indeed, there are several ways in which electronic peer review can be made more efficient, fairer, and perhaps even more valid in the electronic medium. The "point faible" of the peer review system is not so much the referee and his human judgment (though that certainly is one of its weaknesses); it is the SELECTION of the referee, a function performed by the Editor. Hence it is really the Editor who is the weak link if he is selecting referees unwisely (or, worse, not heeding their counsel when it is wise). Editors usually have "stables" of referees (an apt if unflattering term describing the workhorse duties this population performs gratis for the sake of the system as a whole) for each specialty; in active areas, however, these populations may be saturated -- a given workhorse may be in the service of numerous stables. So one must turn to less expert or less experienced referees. In practice, the problem is less the saturation of the true population of potentially qualified referees but the saturation of that portion of it that an Editor KNOWS of and is in the repeated habit of consulting.
One of the results of this overuse of the workhorses is that the entire refereeing process is a very sluggish one. One does one's duty, but one does it reluctantly, other duties take priority, manuscripts sit unread for unconscionably long times, referees are delinquent in meeting the deadlines they have agreed to, and sometimes, out of guilt, hasty last-minute reports are composed that do not reflect a careful, conscientious evaluation of the manuscript. There is much muttering about publication delay, a real enough problem, especially in paper publication, but peer review itself is often responsible for as much of the delay as the paper publication and distribution process itself.
Now, as I said, there are no ESSENTIAL differences between paper and electronic media with respect to peer review. And the Net is populated by frail human beings, just as the paper world is. But the Net does offer the possibility of distributing the burdens of peer review more equitably, selecting referees on a broader and more systematic basis (electronic surveys of the literature, citation analysis, even posting Calls for Reviewers to pertinent professional experts' bulletin boards and allowing those who happen to have the time to volunteer themselves). The speed with which a manuscript can be circulated electronically is also an advantage, as is the convenience that many are discovering in reading and commenting on manuscripts exclusively on-screen. All in all, implementing the traditional peer review system purely electronically is not only eminently possible, but is likely to turn out to be optimal, with even paper journal editors preferring to conduct refereeing in the electronic medium (I am certainly doing this more and more with the paper journal I edit).
Once peer review is in place on the Net, once the quality hierarchy has been established, serious scholars will no longer have reason to hesitate to confer their best work to the electronic-only medium. Yet my prediction is that this state of affairs will NOT prove to be the critical factor in drawing the scholarly community onto the Net with their serious work. Much has been said about what the critical "value added" feature of the Net will be that succeeds in winning everyone over. We have spoken of decreased costs, but I think that even my estimate that the true expenses of electronic publication will be only 20-30% of paper publication will not be what does the trick. Decreased publication lags and more equitable refereeing on the Net will also be welcome but still not, I think, the decisive factors. Not even the global access to eyeballs unrestrained by the barriers of subscription cost, xeroxing, mailing or postage, nor the possibility of a (virtually) free world electronic periodical library sitting on every scholar's desk thanks to network links, nor the powerful electronic search and retrieval tools (built on anonymous ftp, archie, wais, gopher, veronica, WorldWideWeb, and their progeny) that will be within everyone's reach -- none of these, remarkable as they are, will be the critical value-added feature that tilts the papyrocentric status quo irreversibly toward the electronic airways.
I think I may be peculiarly well placed to make this prognostication. For over fifteen years I have edited a paper journal specializing in "Open Peer Commentary": BEHAVIORAL AND BRAIN SCIENCES (BBS, published by Cambridge University Press) accepts only articles that report especially significant and controversial work. Once refereed and accepted, these "target" articles are circulated (formerly only as paper preprints, but these days in electronic form as well) to as many as 100 potential commentators across specialties and around the world, who are invited to submit critical commentary, to which the author will respond Harnad 1979, 1984b). Among the criteria referees are asked to use in reviewing manuscripts submitted to BBS is whether open peer discussion and response on that paper would be useful to the scholars in the fields involved (the material must impinge on at least three specialties). Each target article is then copublished with the 20 - 30 (accepted) peer commentaries it elicits, plus the author's Response to the commentaries. These BBS "treatments" have apparently been found useful by the biobehavioral and cognitive science community, because already in its 6th year BBS had the 3rd highest "impact" factor (citation ratio; adjusted: see Drake 1986; Harnad 1984a) among the 1200 journals indexed in the Social Science Citation Index. BBS's pages are in such demand by readers and authors alike that it has (based on an informal survey of authors) one of the highest reprint request rates among scholarly periodicals and, of course, the characteristically high rejection rate for submissions -- attesting as much to the fact that there is more demand for Open Peer Commentary than BBS can fill as to the fact that BBS's quality control standards are high.
Yet BBS has some inescapable limitations, because its tempo is far too slow. Peer review (using 5-8 referees, from 3 or more specialties), is, as usual, a retardant, but even if one starts the clock at the moment a target article is accepted, and even if one allows for the fact that preprints are in the hands of one hundred peers within two weeks of that moment, their commentaries received six weeks after that, the author's response four weeks after that, and then the entire treatment appears in print 4-6 months later, these turnaround times, though perhaps respectable compared to conventional forms of paper publication, are in fact hopelessly slow when compared to the potential SPEED OF THOUGHT.
I have discussed the chronobiology of human communication in more detail elsewhere (Harnad et al. 1976; Harnad 1991). Suffice it to say here that the tempo of a spoken conversation is in the same neighborhood as the speed of thought, whereas weeks, months, or years of lag between messages are not. Whatever ideas could have been generated by minds interacting at biological tempos are forever lost at paper-production tempos. Scholarly Skywriting promises life for more of those potential brainchildren, those ideas born out of scholarly intercourse at skyborne speeds, progeny that would be doomed to still-birth at the earthbound speeds of the paper communication.
I hasten to add -- so as to dispel misunderstandings that have already been voiced in the literature (e.g., Garfield 1991) -- that I am not advocating oral speeds for all scientific "publication." First of all, the time to pass through the filter of peer review already puts some brakes on the speed of interaction. Second, even unmoderated electronic mail correspondence is not as fast as a conversation (nor would it be comfortable if it were -- as anyone who has engaged in real-time e-writing "conversations" can attest). Nor is the goal the undisciplined babbling that we all recognize from "live" symposium transcripts. The goal is something in between: Much faster than paper-mediated interaction, but not as fast or unconstrained as oral dialogue. Moreover, the virtue of "Scholarly Skywriting" is as an available OPTION. Just as not every article is suitable for BBS, not every idea or finding is a candidate for interactive publication. But at last the option is there.
And once you have tasted it (as I have -- e.g., see Hayes et al. 1992), I think you too will be convinced that it adds a revolutionary new dimension to scholarly publication and, even more important, will, I predict, increase individual scholars' productivity by an order of magnitude (all those stillborn ideas that now have a lease on life!).
Peer commentary, after all, whether refereed or not, is itself a form of peer review, and hence of quality control (Mahoney 1985). Let us be imaginative in exploring the remarkable possibilities of this brave new medium. My argument here has been on behalf of conventional peer review as the principal means of controlling quality, whether on paper or on the Net, and whether for target articles or commentaries. But once such rigorous, conventional constraints are in place, there is still plenty of room on the net for exploring freer possibilities, and the collective, interactive ones, are especially exciting.