I previously blogged the draft of a paper by Andrew J. Oswald (University of Warwick), An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers.  The paper has now been published at 74 Economica 21 (2007).  Here is the abstract:

Scientific-funding bodies are increasingly under pressure to use journal rankings to measure research quality. Hiring and promotion committees routinely hear an equivalent argument: ‘this is important work because it is to be published in prestigious journal X’. But how persuasive is such an argument? This paper examines data on citations to articles published 25 years ago. It finds that it is better to write the best article published in an issue of a medium quality journal such as the OBES than all four of the worst four articles published in an issue of an elite journal like the AER. Decision-makers need to understand this.

From the Introduction:

The paper collects data on the accumulated lifetime citations to papers published 25 years ago. It uses these to construct a simple test. The data come from issues of six economics journals of varying levels of reputation. These data show the expected ranking. However, and more interestingly, they also reveal that the best article in an issue of a good to medium-quality journal routinely goes on to have much more citations impact than a "poor" article published in an issue of a more prestigious journal.

Update:  My colleague Michael Solimine pointed me to Ian Ayres & Frederick E. Vars, Determinations of Citations to Articles in Elite Law Reviews, 29 J. Legal Stud. 427 (2000), which analyzes the most, and least, cited articles in the Harvard Law Review and Yale Law Journal in 1980-95 and urges "extreme modesty" in citation analysis.