The Story of an Article

"Publications and their citations are the coin of the academic realm with which tenure and research grants are obtained. In a way, an article in a peer-reviewed journal is an “advertisement” of a researcher’s scientific prowess." –Jan Velterop at Open and Shut?

For the past two years, I have been working on a thread of research together with my colleague and friend Jani Kotakoski. Our toils finally bore fruit earlier this week when the article presenting our results appeared in the web(behind a paywall unfortunately; send me an email if you're interested). The journal is ACS Nano, one of the top 5 journals in the field with an impact factor of 11.4. Although the dismerits of journal impact factors are well-known and widely decried, I won't be a hypocrite: since my aim is to succeed in the academic rat race that means playing by the rules as they are. Currently it just doesn't make sense to aim for anything lower than what the research merits. Although it didn't end up affecting our choice this time around, apart from the obvious positive reasons, the claim that open access gives a boost to the number of citations an article receives, which although still controversial, certainly warrants further study. In any case, research-wise we had a solid story to tell and the rather impressive results to support it. And, besides the high impact, a lot of research personally interesting to us is published in ACS Nano.

I am indebted to all of the co-authors (10 of them from 5 countries!) for their crucial contributions to the end result, as well as to our group leader, Prof. Kauppinen, for making it possible for me to pursue the topic despite not being strictly paid to do so. In a nutshell, we looked at how the energetic electron beams used in transmission electron microscopes(basically the same idea as an optical microscope, but the images are formed by electron waves instead of light) affects carbon-based nanomaterials (namely graphene and carbon nanotubes) that had been doped with nitrogen atoms. The image below illustrates the simulated ejection of a carbon atom neighboring a nitrogen dopant (in blue) by an electron impact. The exciting result was that we could directly see events like this using sufficiently advanced electron microscopy, and what we saw matched our simulations. As a side note, the concept of doping in nanosystems is similar to the traditional semiconductor case, but the reduced dimensionality affects things in interesting ways. But that's a story for another day.

Electron impact

The story I'd like to tell today is the story of that article. How the idea was born, how the collaboration got started, and how the manuscript was written, submitted (resubmitted, re-resubmitted...), and finally accepted for publication. I told the main plot of the story up until June 2012 in a talk I gave in the 5th ScienceSLAM Helsinki event, embedded above (thanks to Christian!). It was a lot of fun to give the talk, and I was quite happy to lose the competition to Anna-Maria Lahesmaa-Korpinen and her team of dancers with their performance “The choreography of the cancer proteome”, which was rather awesome.

I'd like to invite you to discuss the technical aspects of the article using the interesting new Plasmyd platform, and everything else in the comments!

Furthermore, I'd like to expound on the details of my story in more depth here as I feel the story illustrates in a nice way how scientific collaboration can work. However, what follows is a good deal more professionally oriented, so I'll leave it to the reader to decide if he's interested enough in the nitty-gritty details :) I'll admit the idea was influenced by the fascinating guest post on the Cosmic Variance blog which I mentioned in a previous post, and also by the ideals of open science.

As I mentioned in the talk, since we were working on the manuscript very closely together with Jani who had moved to Vienna by the time we started writing, Google Docs was an indispensable tool. Having real-time collaborative editing capabilities and a chat in one browser window was really handy, as was the reasonably good export compatibility to Word. Of course, support for Endnote citations would had really made my day, but still, I shudder to think how many more emails we would had needed to send without it.

Of course, email was the main form of communication between the rest of the co-authors. To capture this, I analyzed all the emails exchanged about the article (a whopping 720 of them!) by copying the lot onto an empty Google Mail account and using Mail Trends. I stripped the emails of attachments, since I hadn't kept all of them from the beginning. The transferring seemed to mess up the send dates of some emails, which I had to delete to avoid skewing the results. With these caveats, here are the results of the analysis (please note that the vertical scale showing the number of emails varies):

The year breakdown unsurprisingly shows that the bulk of the email communication happened this year along with the manuscript writing. The day of the week and time of the day plots clearly confirm what is well-known: scientists are workaholics. Of course, time zone differences during travel contribute somewhat, but since all authors are from Europe, those cases should be a minority. The monthly distribution clearly shows the 3-month break during my India trip in late 2011, and the summer holidays in 2011 and 2012.

The distribution of email senders to me for 2012, although somewhat inaccurate due to the deletion of incorrectly dated emails and not fully representative due to face-to-face and non-email communication, has several interesting features. In the start of 2012, ongoing experimental work with microscopist Hua Jiang made up the bulk of the correspondence (keep in mind that this is a proportional plot; see the monthly breakdown above for the absolute numbers of emails). After that, work on the manuscript began, and I was happy to find that the order of senders matches very nicely the final author order of the article (keeping in mind that the seniors are counted backwards from the end). Additional results that joined the paper later on were discussed intensively with Raul Arenal between April and June. Finally, the relatively large number of emails from the ACS Nano editorial office was due to the number of automated confirmation and copyright agreement emails per each submission.

Speaking of which, the submission and editorial history of the article was rather interesting. We submitted the first incarnation of the manuscript to ACS Nano on March 21, 2012. At this time we didn't yet have the electron energy loss spectroscopy (EELS) results from Raul, and the sections on the nanotube results were leaner; in total, the manuscript file was a tightly written 19 pages (single column, double spacing, 12 pt font size as per the ACS template). Only three days later – rarely a good sign – the editor sent back his decision: he considered the research relevant and interesting, but too short for a full article.

This was the first time we had encountered a situation like this, and were left with a tough decision: expand the manuscript (artificially, we felt at the time; we had written compactly on purpose), or submit somewhere else? After some back-and-forth with the other co-authors, we decided with Jani that our work was strong enough to warrant giving a go at another high impact journal that publishes shorter letter-length articles. We settled on the illustrious Journal of the American Chemical Society, with an impact factor 9.9. Since JACS is more chemistry-oriented, we modified the manuscript to emphasize the chemical aspects of our findings, and had to trim it down to fit the journal's 4-page document template (double column, single spacing, 9.5 pt font size). To do so, we decided to cut away all of the nanotube results and concentrate only on the simulations and the graphene data. The removed material we planned to submit to the real-journal-publication-proceedings of the yearly IWEPNM workshop we had both attended that year (I go every year – apart from the extremely high scientific level, the skiing is fantastic :) Thus we submitted the heavily revised manuscript on March 26, crossed our fingers...

...and waited. Since we didn't get a rejection from the editor straight away, we surmised the manuscript had been sent for peer review. What felt like a long time was actually rather fast as these things go: we got the referee reports and the editor's decision back on April 24: the editor ruled that the manuscript should be published elsewhere. The reason for the decision was made abundantly clear by the referee reports: all three referees made that recommendation. Our long shot had fallen short – rather deservedly, and really as a testament to the very high quality of peer review at JACS. In hindsight the whole JACS episode probably should had been avoided. But on the other had, we were heartened by the near-unanimous assessment of the work as of high importance and novelty, and did receive useful constructive criticism on several aspects of the work.

Again we faced a choice: either lower our goals and submit the manuscript somewhere else (such as the entirely respectable Physical Review B), or expand it and resubmit to ACS Nano. Some of the more senior co-authors recommended the easier way, which is quite understandable considering the amount of manuscripts they are involved with every year. Me and Jani, however, were not about to give up: we felt we had a really nice story to tell, and didn't mind going the extra mile to get it to the most prestigious outlet warranted by the results. In the meantime, the nanotube results cut from the JACS submission had matured: we had made additional analyses of our experimental dataset, considered the effect of curvature for the simulation results in order to support the nanotube results better. Most importantly, Raul Arenal had gotten involved in digging up nanotube EELS data to find the spectroscopic correlates for the structural changes we had directly observed in graphene. Thus we convinced others to give ACS Nano another go with a significantly expanded manuscript.

Some of the analyses took time to finish, and people were busy with other work. We also wanted to improve the manuscript in every way possible, taking into account all of the previous referee reports and making sure that the expansion was sufficient to convince the editor to send it out to peer review on the second time around. The expanded manuscript text ended up 31 pages long, and the number of figures increased from 4 to 7. Raul's EELS results turned out to be important enough to unanimously bump him to third place in the author list, even though he joined much later. Everything finally came together in mid-June, and we resubmitted the manuscript (technically as a new submission) to ACS Nano on June 19. After a week or so we surmised the manuscript was sent for outside peer review, so we set out to enjoy the summer and patiently wait for the reports to roll in and get the editor's decision.

The peer review process was not unproblematic this time around (though certainly not as frustrating as in some cases nor anywhere as bad as discussed here). We got the referee reports and the editor's decision on July 24: to our dismay the manuscript was rejected, although resubmission after significant modification, additional data, and analyses was offered as a possibility. Typically if a manuscript has merit and is submitted to the right journal, it is not that common to get an outright rejection. Normally a recommendation for a major revision is sufficient to address any sorts of technical issues and common criticism. After the initial letdown, we started delving into the (only) two referee reports. The first referee recommended that the work be published after a major revision – par for the course when a manuscript is considered of sufficiently high impact and broad interest for the journal in question, as was the case here. However, the actual issues the referee raised were very minor: he basically only wanted to see more details of the materials studied and the computational methodology. We already had all of this in the manuscript via references to previous work where the details were available, but it seems this was too implicit. Making the required changes was very straightforward and only required a few extra sentences.

The report of Referee #2 was the problem here. His recommendation was to "Reject, but encourage resubmission if the specific points reviewer makes are addressed". Clearly this was the basis of the editor's decision. Not to go too much into the specifics, the bottom line was that the second referee had not understood our work, and did not seem at all familiar with all of the previous research in the field on which it was based. This could, of course, had been remedied by actually going and checking out the detailed references we gave in the manuscript, but in all reality very rarely is this feasible due to time constraints. Thus the second report was far off-base in it's criticism and actually offered very little concrete suggestions that were relevant for the methodology we had used. Declining the request to referee would had clearly been the correct course of action, and one that I would personally had taken without second thought.

This episode highlights a common problem with the present peer-review system: there is no way for an editor to be expert in all of the specific topics and techniques of every paper that is submitted to a journal, least of all one that covers such a wide field as ACS Nano. Thus the editor by necessity needs to rely on the expertise of his reviewers and make decisions based on the reports he receives. Equally important it is to acknowledge that the unpaid referees do this as a service to science. However, a situation like this can lead to objectively incorrect decisions if the referee reports do not accurately reflect the work in question.

Again there was a discussion amongst the co-authors on the best way to proceed. We were still convinced the work was sound and fit for ACS Nano, but the decision had been rather negative, and it is not always easy to convince editors to change their minds. However, we felt we could write a response, which while being polite, would justify our displeasure at the basis of the decision that we felt unwarranted by the facts of the matter. In this we relied on the experience of the more senior co-authors; these are skills one picks up along the way. Our letter detailed a response point-by-point, highlighting the changes made to the manuscript upon revision.

The actual changes made were rather small, amounting to less than a dozen extra sentences. This was a bit of a gamble: the editor had asked for a major revision with additional analyses and new data, but we were convinced these were not really warranted by the actual merits of the referee reports. Our hope was that the editor would reassess his decision and send the manuscript to a third referee, who hopefully would be more familiar with the topic. Since most of us were on or going for our summer holidays when the decision came, drafting the reply was delayed for a bit. We finally submitted our response and the revised manuscript on August 28. After not hearing back from the editor within a week, we felt confident our response had been successful and the manuscript had been sent for additional review.

On September 17 we got the new decision, and it was immediately apparent that the manuscript was finally going to be accepted. The first referee of the previous submission was fully satisfied by our response, and recommended publication without additional changes. The new third referee also considered the manuscript to be of high quality and impact, and suggested only minor pertinent clarifications and changes. These were simple to implement, and we sent the final revision a few days later on September 20.

After clearing that final hurdle, things proceeded impressively quickly – the editorial processes of ACS Nano are clearly well honed. We had to submit two more versions to correct some minor formatting issues (like capitalization of the title and references), and received the final approval and galley proofs on September 26. All authors went through the proofs very carefully, and we ended up submitting nearly 30 minor corrections and edits, which we sent to the editorial office on September 28. And finally, as mentioned in the beginning, our article appeared on the web earlier this week (the publication date was set for the accepted manuscript, not the fully proofed version).

It's been a long road and we are pleased as punch to finally have our work join the scientific literature. As a final thought I'd like to stress that even though our peer review process was slightly problematic, the professionalism of the editor and his willingness to consider a rational argument and get a third referee opinion was really all any author could ask for in the system as it stands.