Raising the Bar at Directory of Open Access Journals

Dominic Mitchell has worked extensively with the publisher and librarian communities both as a publisher, with the BMJ Publishing Group (UK), and as an account and project manager at HighWire Press – Stanford University (USA). He is responsible for collaboration with Directory of Open Access Journals’ (DOAJ’s) publishers, for DOAJ’s technical development, social media presence and its blog, the DOAJ News Service.

The Open Access movement has changed the publishing landscape in a number of ways, some of which were quite unexpected. For instance, the emergence of the author as customer has resulted in a new level of understanding of the needs of researchers while enabling new entrants into the publishing market. It’s no longer necessary to convince a critical mass of libraries that a new journal should exist – publishers can effectively ‘sell’ their journal one author at a time. While this lowering of the barrier has enabled tremendous innovation, it also creates new challenges in terms of publishing ethics.

In 2013, DOAJ’s Managing Director, Lars Bjørnshauge sat down with a small DOAJ team and put together a list of features and mechanisms that they believed describes the characteristics of a legitimate  publisher who is serious about disseminating peer reviewed research. The initial list was created using a mixture of experience in academic publishing and knowledge gathered from hundreds of instances of applications to DOAJ from both legitimate and non-serious journals and publishers. The list was given to our Advisory Board and then put out for public consultation in June 2013. It was finalized after several iterations. In March 2014, DOAJ launched a reformed  application process, using that consultation document as a template. Today, all journals that apply for entry to the DOAJ must fill in a form containing 58 questions, the majority of which are obligatory. It was a timely change and one that was already taking shape in other areas.

DOAJ raised the bar for entry for several reasons:

1. The open access market had matured and becoming more diverse and complex than it was when DOAJ was launched in 2003 with 300 journals. The old criteria were simply no longer adequate to expose the information that users need to assess open access journals, in particular article processing charges (APCs).
2. The old application process was a two-stage affair which was both time consuming and resource heavy for our small team of reviewers.
3. There was a need to proactively tackle the problem of applications from questionable journals with a set of criteria that dissuaded non-serious applicants from applying and gave the DOAJ reviewers the tools they needed to quickly identify and weed out questionable journals.

The new form was launched in March 2014 but left DOAJ with a problem: there were already 9,900 journals in DOAJ that had been accepted under the old criteria that, in the interests of consistency and quality assurance, would have to re-apply.

The Re-application Project is a huge, ongoing process. We began re-application in January 2015 and it came with several challenges. Our technical partners, Cottage Labs LLP, engineered a system that generated re-application records for all 9,900 DOAJ members, allowing publishers to update them with over 50 new pieces of data, overwriting the existing records. (From a database engineering perspective, that’s a huge table with a lot of mapping!) The DOAJ publisher  demographic is consistent with publishing industry as a whole in that it is extremely diverse in terms of size: we have several very large publishers or platforms sitting alongside thousands of single journal publishers, so we had to devise a system that would smoothly facilitate the re-application process for everyone, regardless of size.

Finally, the only way to directly engage people is via email and that in itself presents two huge challenges: people don’t update their details with us (some accounts in DOAJ were created as long ago as 2003); and a surprisingly large number of DOAJ emails get caught in spam filters and organizations’ firewalls. In January 2015, we sent out emails to the 56 accounts that had 11 or more journals linked to them (accounting for >1/3rd of all journals in DOAJ) and asked them to submit updated record data. In June, we sent out emails to the remaining 5,800 accounts. Over 500 emails bounced immediately which precipitated a 5 month clean-up program of our user data. Today, almost 4,000 re-applications have been submitted. We have already accepted 695 back into DOAJ and will be processing the rest once a critical mass of members have re-applied. In addition, 1,748 new journals have been accepted since March 2014. If you consider that we have over 5,000 re-applications still to be submitted and that we receive 80 new applications every week, you begin to understand the scale of our operation.

We wrote in more detail about the history of DOAJ, the reasoning behind the criteria and some stats about our progress in an article recently published by Science Open.

The ‘perfect’ quality control mechanism for Open Access Journals

I believe quite firmly that the approach that DOAJ is taking is the key to solving the issue of quality and thereby the reputation of open access publishing. I would even go so far as to state that all academic publishing could benefit from elements of our processes. There have been many calls recently for a definitive whitelist of reliable open access journals that researchers can use as a guide to where to publish with confidence. That is exactly DOAJ’s mission. However, in order to cope with the growth of open access publishing, DOAJ needs to be bigger. To remain vital and relevant, DOAJ needs to be faster without the precision of the review process suffering. DOAJ must transform into a fully community-driven initiative where the funders and sponsors become the contributors to a program driven by researchers and librarians who are on the receiving end of today’s system. Institutions should directly support those who volunteer for DOAJ and should actively and regularly raise awareness around the importance and relevance of the DOAJ criteria, and how they are significant in today’s academic publishing system. There are already tools out there to do this but there needs to be more awareness absorbed into workflows and processes.

At DOAJ, we struggle to climb over three hurdles, the first two being  prestige and questionable publishers. The former takes people away from open access because they believe that there are no prestigious open access journals. The latter draws their attention to it for all the wrong reasons. Questionable publishers are a drain on our resources, an annoyance that distracts DOAJ from focussing on its third hurdle: helping genuine publishers improve, become more visible, and thereby more reliable. While we can never stop questionable publishers from applying to be included, as a community, we can prevent them from having any real presence in academic publishing by excluding them and marginalising them, thereby reducing their visibility. I believe that there are 2 ways to do this: raise awareness amongst researchers and librarians, on the ground, in the institutions, in the territories that need it most; and by dedicating more people power to help build and maintain the whitelist that the community is calling out for. DOAJ is already engaged in two projects that will take steps to achieving more awareness in targeted territories all over the globe. One is a partnership with Research4Life where we will assist the three member programmes to ensure that they only include quality open access journals in their offering. The other is still being formalized and we hope to be able to announce something in early 2016.

As a community, our focus needs to be on: helping researchers decide where to publish; the indicators to look out for that signify a reliable journal, rather than ringing alarm bells on where not to publish. DOAJ’s criteria and re-application project are already laying the foundations for these indicators. Now we need a concerted effort to get the message out there where it is needed most.