Evolutions in Scholarship

Taking the sting out of the “sting”

Monday, October 7, 2013 4:34 pm

Did you hear the story last Friday on NPR’s Morning Edition about open access (OA) journals and peer review? About the OA “sting” from Science, “Who’s Afraid of Peer Review?” Yeah. The “sting” has angered many OA advocates, me included, and has generated many responses; several of note are linked below.

In the admittedly few responses I’ve had a chance to read now that I’m back in the office (oh, why must such stories break when I’m on vacation?!), most focus on methodological flaws, including no control group, which are important to call out. DOAJ’s response calls attention to its recent work to clarify and tighten its criteria for inclusion. But what I’ve yet to see are a breakdown of numbers that show just how small a percentage of journals listed in DOAJ are actually “proven” negligent by this effort. If you only heard the NPR piece, then you’ll know that seemingly-big numbers of OA journals apparently don’t do peer review: 157, oh my! But what both the NPR interview and Science article fail to do is to put those numbers in useful context. 157 sounds big if you’re talking dollars, but not so big if you’re talking pennies. When it comes to the number of OA journals, this “sting” is counting change, not bills.

As reported in the article, 187 journals of the 304 identified to receive the fake paper were listed in DOAJ as of 2 Oct 2012 (167 in DOAJ, 121 in Beall’s list, 16 in both). That accounts for only 2.27% of all DOAJ journals on that date (the article notes there were 8250 titles in DOAJ). In the results, the real researcher behind the fake paper, John Bohannon, notes that 157 accepted the paper, but he does not continue to give DOAJ/Beall’s list/both breakdowns in the text of the article at this point. Of those 157, we don’t know how many are in DOAJ without digging into his data (which, frankly, given that I know how small a percentage we’re talking about, I’m not going to expend energy doing); all we know is that “for DOAJ publishers that completed the review process, 45% accepted the bogus paper.” But apparently only “106 journals discernibly performed any review.” From the article, we don’t know how many of that 106 were DOAJ titles, so that 45% sounds bigger and badder than it may be. Generously presuming all 106 were DOAJ titles, then 48 DOAJ journals appear to have faulty peer review. Those 48 journals account for a mere 0.58% of all DOAJ titles a year ago; as of last Friday, when DOAJ had 9948 journals, that’s only 0.48%. Turns out, the numbers are available: from a graphic in the article, it shows that 35 journals in DOAJ apparently accepted the paper without any peer review; 38 performed some level of peer review and still accepted the fake paper. So 73 journals in DOAJ have peer review problems. That’s a mere 0.88% of all DOAJ titles a year ago; as of last Friday, when DOAJ had 9948 titles, that’s only 0.73%. That’s not good, but it’s also not alarmist.

But even if all 157 journals (now, 158, per the author’s NPR interview Friday morning) that accepted the fake paper were DOAJ titles (again, I’m being intentionally generous), that’s still only 1.58% of all current DOAJ titles. I daresay in the traditional publishing realm, there are more than 1.58% of print journals that would be “proven” to have shoddy-to-no peer review, too.

Oh, and the journal behind this “sting,” Science? Yeah, it published the discredited and widely-maligned arsenic DNA paper in 2011, which slipped through its own peer review process (h/t, Michael Eisen’s response below). That lessens the sting a bit, doesn’t it?

Finally, a few final quibbles – and one concession – with the NPR piece, as that was my first introduction to the controversy, as it was for many:

  • The total number of journals listed in DOAJ was never noted, so the numbers context above was lost on the audience (and it’s murky enough in the actual article);
  • At no point in the interview it is mentioned that journals owned by Sage, Elsevier, and Wolters Kluwer – big name publishers of primarily traditional journals – were among the 157 accepting the fake paper; and,
  • To be fair, Bohannon did make a clear effort not to malign all OA journals while discussion his “sting,” which I appreciate, but unfortunately, most researchers who’ve been resistant to OA likely only half-heard that defense.

As Bohannon rightly pointed out during his NPR interview, we do need to identify disreputable publishers who are taking advantage of the low-to-no capital needed to launch an e-journal using the OA article processing fee model, who through negligence are casting OA publishing in poor light. But more than that, we need to address issues in the peer review process for all journals.

Responses worth reading:

Updated 8 Oct 2013 to correct numbers above, and to link to the following, many of which unpack the flawed “methodology” behind the “sting”:

Wake Forest supports FASTR access to research

Monday, September 30, 2013 11:21 am

On Friday, September 27, 2013, Provost Rogan Kersh made a public commitment to Wake Forest University’s ongoing support of open access by signing the “Open Letter to the Higher Education Community,” in support of the proposed Fair Access to Science and Technology Research Act (FASTR) bill. FASTR, introduced to both the House and Senate in February with bipartisan support, seeks to ensure that Federal agencies with $100 million+ of annual extramural research funding make the outcomes of taxpayer-funded research openly, publicly available. FASTR builds upon the success of the National Institutes of Health’s Public Access Policy, the first U.S. public access policy enacted in 2008, and compliments the White House Office of Science and Technology Policy’s February 2013 memorandum directing Federal agencies with $100 million+ annual R&D budgets to provide public access to publications and data, plans for which were submitted by agencies last month.

Provost Kersh’s signature is the latest in a series of public commitments made by Wake Forest Provosts to encouraging broader open access to research, building on former Interim Provost Mark Welker’s signing of the Berlin Declaration in November 2011, and former Provost Jill Tiefenthaler’s signature of a similar letter supporting the Federal Research Public Access Act, the earlier iteration of the FASTR legislation, in May 2010. While Congress’s energies are currently focused on the looming midnight deadline to halt a government shutdown, hope remains that once we are past this current showdown, strong bipartisan support of FASTR will continue to grow, boosted by the support of institutions across the academic community. Proudly, Wake Forest is once again among them.

Fair use infographic

Friday, August 23, 2013 3:19 pm

Do you love infographics? Do you love fair use? Do you love libraries? Then you’re going to love the new infographic from ARL, American University’s (AU) Washington College of Law, and AU’s School of Communication about the Code of Best Practices in Fair Use for Academic and Research Libraries. You can find the full-size PDF and 8.5×11″ PDF files here.

Impact of impact

Monday, July 8, 2013 2:08 pm

If you’ve been around the scholarly journal publishing scene, either as an author or a librarian, you’ve likely heard talk of impact. Often, that’s impact with a big “I” – the Journal Impact Factor. Published annually each June by Thomson Reuters, the impact factor is a metric that ranks journals by the average number of citations to articles published in that journal in the preceding two years. Researchers look to it when identifying top journals in their field. Librarians look to it when making collection decisions. P&T review committees look to it when assessing journal quality and influence.

The 2012 impact factors were recently announced, and one journal in particular saw a precipitous 16% drop in its impact score from 2011 to 2012. The journal? PLoS ONE. Now, PLoS ONE is a revolutionary journal, and not just because it is open access. It is a massive, multidisciplinary, rapid peer-reviewed journal that has been making waves since 2006. It publishes tens of thousands of articles a year, and its early success – its impact – is only serving to increase annual publication rates. As one blogger explains, the fall in PLoS ONE’s impact factor is the victim in PLoS ONE’s own success.

But is it a victim? Notice that when I quickly explained the impact factor above, I noted that it is a metric, not the metric. Too often, impact factors are (mis)used as an evaluative tool to quickly assess a journal’s influence, and by extension, the presumed influence of an author’s article/research. But it only counts citation rates, and citations can be gamed. Citations also aren’t the only measure of impact, as not every use of an article will be cited in a formal publication. What about articles that get blogged? Tweeted? Facebooked? Picked up by news media? Shared by patient advocacy groups? Incorporated into classroom teaching? How are those legitimate impacts being measured? Not by impact factors, that’s for sure.

Enter altmetrics. Recognizing that the scholarly ecosystem has evolved beyond paper-based journal publication and citation to include online tools for discovering, indexing, and sharing influential articles, altmetrics “reflect the broad, rapid impact of scholarship in this burgeoning ecosystem.” ImpactStory and Plum Analytics are two altmetrics tools to aid researchers in assessing the broader, richer impact of their research. I don’t know much about either one, yet, but what I do know is exciting! In the coming months, I’ll be exploring altmetrics in greater detail, and expect to share what I learn with you here. Stay tuned!

My MOOC Experience

Monday, May 6, 2013 2:07 pm

Earlier this semester, I completed my first MOOC, An Introduction to the U.S. Food System, offered through Coursera by the Johns Hopkins Bloomberg School of Public Health. While I didn’t learn as much new information as I assume others did (not a criticism of the course, more that I’ve been learning about food issues for a number of years now), learning about our food system was only a partial goal in signing up. Given all the hoopla around MOOCs, I wanted to personally experience what some seem to see as the shiny new future of higher ed.

MOOCs are Massive Open Online Courses, and they have captured the fancy of many in higher ed, and beyond. MOOCs offer students the opportunity to learn from noted scholars, without incurring tuition costs, from the comfort of wherever they happen to be. While MOOCs may not be the future of higher ed reform – and there’s been some recent criticism – they can’t be overlooked entirely.

I embarked on my MOOC hoping to be wowed into tempering my cynicism that they are the saving grace of higher ed’s current problems. That didn’t happen (criticisms below), but there was one surprising insight that only occurred once I became a student myself: if MOOCs are going to stick around in some variation, it is imperative that researchers make sure their articles and books can be used for course content; aka, scholarship must be openly accessible. I came to this realization myself while working through my MOOC assignments, but was reassured to learn that Kevin Smith at Duke University also sees the need for a MOOC-Open Access correlation.

My MOOC involved weekly video lectures, required and optional readings, discussion forums, and quizzes. The readings are where I felt the closest affinity to a traditional course, as both my MOOC and a Geography course on food systems I took as an undergraduate relied on articles in lieu of a textbook. However, the nature of those readings were quite different. In my MOOC, of 55 readings (the vast majority optional), only 8 were scholarly articles, 7 OA. One optional reading was an article that explicitly required institutional access (alas, WFU didn’t subscribe), and two were non-research articles from scholarly journals. The rest were newspaper articles, reports, websites, and other pertinent but not scholarly readings.

What strikes me about the non-scholarly nature of the majority of the readings is that: a) this is a topic that can be covered by non-scholarly articles and reports, but that b) only ONE of the required readings was an OA scholarly article. I cannot speak to whether or not the reliance on non-scholarly readings would be similarly reflected on the syllabus of a correlated course taught at JHU, but my guess is that if this had been a traditional course, more than one of my assigned readings would have been from a scholarly source.

Setting aside my additional criticisms (simplistic assignments, no real feedback opportunities, unwieldy forums) that cast MOOCs* in a poorer light than traditional courses, college classes incorporate scholarship through assigned and supplemental course readings and student research. But when the availability of scholarly readings are limited, as they are in MOOCs, I fail to see how it can be argued that they are an acceptable substitute for credit-based classes.

Does this make my MOOC inherently bad? Of course not! I quite enjoyed my learning experience, despite a strong base of knowledge coming in. But having experienced both a MOOC and a traditional course on essentially the same topic, I cannot buy into the claim that MOOCs are the future of higher ed, without substantial shifts to open scholarly publishing and distribution systems.

___

* Admittedly, I’m making a broad generalization here, as I’ve only taken one MOOC, so my experience is quite limited. I’ve heard that other have more rigorous assignment. My second MOOC starts today – perhaps I’ll have more “homework” to do this time around!

Looking back, looking forward

Friday, February 15, 2013 4:05 pm

Until yesterday, 2013 has not been a year of positive movement in the scholarly communication realm. In early January, many were shocked and saddened to learn of the death of Aaron Swartz, a leading advocate for open access. Although his methodology has been questioned by some, the principles that inspired his actions are sound. In late January, updates I heard from various lawyer-librarians on the Kirtsaeng v. Wiley case before the Supreme Court and the implications for ALL, not just libraries, if Wiley gets even a partial “win” (Miss Pollyanna here refuses to believe an outright “win” is possible) were disheartening.

February didn’t get any better, as the publishers in the Georgia State copyright case filed their appeal at the end of January, and February greeted us with the chilling news that the US Dept. of Justice requested additional time to determine if they were going to file an amicus brief not in support of GSU, but either for the publishers or for neither party. (See Nancy Sims’s take for an excellent analogy to eating fast food burgers.) Then we learned that an academic librarian is being sued by a publisher, and another academic librarian has been threatened with litigation. After a particularly uplifting 2012 – Research Works Act squashed; researchers rebelled; victories one, two & three rolled in – this has been a difficult year to stomach.

But yesterday, on the 11th anniversary of the Budapest Open Access Initiative, which launched the Open Access movement (and also Valentine’s Day no less), hope glimmered again with the introduction of the Fair Access to Science and Technology Research Act (FASTR) in both the US House and Senate. This is an updated version of the Federal Research Public Access Act, which was introduced in the past but never brought for a vote. FASTR would require all Federal agencies that fund $100 million+ in extramural awards annually to make publications stemming from funded research available to the public within 6 months of publication. But FASTR is better in two critical ways: 1) it would also require that the publications be in formats open to reuse for “computational analysis by state-of-the-art technologies” (that’s text-mining, baby!), and 2) it was introduced simultaneously in both the House and Senate, with bipartisan support, in a non-election year. SPARC and InfoDocket have additional information.

May the glimmer become glitter this year!

Open Access Explained

Friday, October 26, 2012 4:48 pm

It’s international Open Access Week 2012, when librarians and researchers around the globe are talking about OA and related issues. Here at Wake Forest, I led a workshop for faculty and staff on fair use and copyright in teaching and scholarship Thursday afternoon (slides). On Tuesday, I gave a guest lecture on OA to faculty at Furman University (slides).

The guys behind PhD Comics have created a fun video explaining OA. While it focuses only on OA journal publishing, and is science-specific, it’s a wonderful, quick primer on the principles of OA. Enjoy!

 

Busy week across the courts

Thursday, October 11, 2012 5:20 pm

It has been a busy – and exciting! – week in courts across the country, with important implications for libraries and fair use. Last Thursday, it was reported that Google and the Association of American Publishers (AAP) reached a settlement in their long-running case surrounding the Google Books project. Although the terms of the settlement have not been disclosed, we do know that publishers will have the option to opt-out of Google Books for out-of-print yet still in-copyright content, and the digital scans Google created will be removed. For those publishers who elect to stay in, Google will continue to provide 20% views via Google Books, sell full-text access via Google Play (with revenue sharing), and provide the publisher with a digital copy. On the whole, this changes things very little for users in practical terms – and does not end the class action lawsuit by authors still pending against Google – but ends one aspect of a 7 year lawsuit that libraries have been monitoring closely.

We also learned last week that for a second time, the lawsuit brought against UCLA for digital video streaming has been dismissed. Details are scant (and hat-tip to Kevin Smith for cluing me in, as I’d missed this until today), and the dismissal does not mean that what UCLA was doing was either fair use or covered by the licensing terms. Perhaps UCLA’s use is fair, perhaps it’s covered by licensing terms, perhaps the plaintiff did not have the rights to bring the suit in the first place. So like the Google settlement, little practically changes with this dismissal.

The big, fabulous, fist bump-worthy news for fair use and libraries came last night with the ruling that the Authors Guild’s lawsuit against HathiTrust has been dismissed on summary judgement by Judge Harold Baer. In 2011, the Authors Guild, two American authors, and several similar authors’ groups overseas filed suit against HathiTrust and several member institutions on the grounds that the in-copyright digitization that HathiTrust was coordinating, which enables new scholarship opportunities via searching and indexing and greater print-disabled patron access, exceeded the library-specific exceptions outlined in Section 108 of the Copyright Act. Furthermore, the plaintiffs claimed that because of the provisions outlined in Section 108, HathiTrust could not rely on fair use as provided in Section 107 to cover uses that fell beyond 108. Convoluted, no?

In a huge win for libraries, Judge Baer granted the defendants’ motion for summary judgement (MSJ). Nancy Sims provides an excellent explanation of the importance of an MSJ ruling, but the bottom line is that the judge felt that HathiTrust’s fair use defense was so air-tight, a trial was unnecessary as the outcome was a foregone conclusion. As Judge Baer wrote, “I cannot imagine a definition of fair use that would not encompass the transformative uses made by Defendants’ MDP and would require that I terminate this invaluable contribution to the progress of science and cultivation of the arts that at the same time effectuates the ideals espoused by the [Americans with Disabilities Act]” (see pg. 22 of the ruling).

Fair use wins!!

As always, there are deeper nuances in Judge Baer’s ruling, including the role of transformativeness and the option for foreign authors’ groups to pursue a suit based on copyright law in their respective countries, and for more on those I point you to:

 

Publishers to appeal the GSU decision

Tuesday, September 11, 2012 4:41 pm

Less than a month after Judge Orinda Evans declared Georgia State University to be the prevailing party in the four-year long lawsuit brought against GSU by three scholarly publishers–Cambridge University Press, Oxford University Press, and SAGE–which hinged on alleged copyright infringement in e-reserves and course management systems, the plaintiffs have announced that they will appeal the ruling. This appeal rather frustratingly comes as little surprise to those who have kept tabs on this long-running case, as it has been clear throughout that the publishers’ interpretation of fair use is fundamentally different than that of GSU, which correlates to the interpretation of many academic libraries, and ultimately that which Judge Evans thoroughly examined in her ruling.

For the publishers, who articulate their dissatisfaction and disagreement in a statement released yesterday announcing the decision to appeal, there is little to lose and potentially much to gain in an appeal. As Brandon Butler, the Association of Research Libraries director of public-policy initiatives, was quoted as saying in the Chronicle of Higher Education’s coverage, “the plaintiffs’ situation [is similar] to that of gamblers who have little left to lose. ‘They’ve been gambling all day, they’re way, way in the hole, they can make one last gamble and win everything back.’”

Interesting times ahead, it seems.

Additional coverage:

GSU is the prevailing party

Tuesday, August 14, 2012 3:27 pm

Last Friday, August 10, Judge Orinda Evans issued her final ruling in the GSU e-reserves copyright infringement case. In her final court order, she dismissed the plaintiff publishers’ proposed injunction, which would have placed severely burdensome oversight and reporting requirements on Georgia State, and found GSU to be the prevailing party in the case. Judge Evans furthermore ordered the plaintiffs to pay Georgia State’s attorney’s fees, which is estimated to cost $2-3million. Although many had presumed GSU to be the prevailing party when Judge Evans issued her decision on the case in May, in which she found only 5 of 99 instances to be infringing, it was not until this past Friday that our presumptions were confirms. And now we wait to see if the publishers will appeal…

Read more on Inside Higher Ed.


Categories
Digital Humanities Pedagogy
Digital Humanities Research
General
the-future-of
Tags
altmetrics blogging copyright fair use MOOC open access public access publishing research scholarly communication scholarship
Archives
September 2014
July 2014
June 2014
April 2014
January 2014
November 2013
October 2013
September 2013
August 2013
July 2013
May 2013
February 2013
October 2012
September 2012
August 2012
July 2012
May 2012
February 2012
July 2010
March 2010
Subscribe
Entries
Comments

Powered by WordPress.org, protected by Akismet. Blog with WordPress.com.