I am deep into the write up of my PhD thesis at the moment. For the past couple of years I have been using Scrivener to write my thesis. Scrivener is a really awesome piece of software for writing large reports. The learning curve is not inconsequential, but when you figure out how to use it, Scrivener is fantastic. At least, that’s what I would have said before I temporarily lost a lot of my work, and permanently lost some of it, after I moved my thesis.scriv document to Google Drive.
I used to adore Scrivener and would recommend it to anyone who asked me what I was using to write my thesis. Only 2 weeks ago I was having lunch with a colleague and we were talking about writing workflows and I got my laptop out of my bag to show them how I was writing my thesis with Scrivener and why I thought it was a really amazing piece of software. I had been planning on writing a post about my writing workflow with Scrivener, and I am sad that I am now having to write a more critical review.
So, why have I fallen out of love with Scrivener? Well, yesterday I lost 4000 words of my literature review and 10 days worth of edits to the thesis. After much panic and stress I did manage to recover most of the missing literature review words, but I know that I have definitely lost some content and I am uncertain if my other edits are included in the current version of my thesis document or not. At the moment, the modification date of each section of my thesis is set to 2 weeks ago, so my guess is the thesis contains old content and not the refined version I spent my Annual Leave from my job as a lecturer working really flipping hard on.
When I realised I had lost the work, my initial reaction was to cry. I was finally making some really good progress on a really difficult part of my thesis, and then my software failed and I lost a huge section of writing that I thought was actually decent quality. Although I did manage to find most of the content, I definitely lost some even though I thought I was being sensible by storing my work on the cloud rather than on an ageing laptop.
I want to save people from the pain and panic of lost words, so if you use Scrivener, or are thinking about using Scrivener please heed the following warning.
Scrivener does not play nicely when your project is stored on the cloud.
That is, if you save your .scriv file in your Google Drive you should stop immediately. On their Scrivener support pages, Literature and Latte (the company that make Scrivener) state that:
“At this time we strongly discourage the use of Google Drive for synchronising live projects. While this service appears to be stable for the transfer of simple files, complex formats like Scrivener have been known to cause it to fail, reverting, corrupting or even erasing months of work. In the cases we have handled, there has been no way to recover the data if local backups were not available. Please consider using another service for your synchronisation.”
So, even the company that makes Scrivener says don’t synchronise using Google Drive. The suggestion seems to be that this is a “Google Drive” problem, rather than a Scrivener problem. And whilst they might be right in some sense, I don’t think it’s an unusual user requirement to be able to store work using a cloud service nowadays. And Google Drive is one of the most popular cloud services.
The above quote does imply that another cloud synchronisation service should be suitable. Interestingly, there are no suggestions as to which syncing products are safe to use with Scrivener. But, another immensely popular syncing service is Dropbox. So, what about Dropbox? Will Dropbox play nicely with Scrivener? Well, if you use Dropbox with Scrivener you should read these instructions very carefully.
To summarise the key points from the link above, if you want to use Dropbox to synchronise your Scrivener project then you need to ensure you don’t try to open the same project on two (or more) different computers simultaneously. If you do open a project on multiple computers, or aren’t too sure if you left it open on another device, they have implemented an error recovery process which allows you to open a copied version instead and prevent conflicts. So that’s sort of okay and probably manageable. It’s not ideal, but there is a fail safe option which won’t override the original document or cause conflicts if you did happen to leave it open on another computer. If this was the only accommodation that had to be made to use Scrivener with Dropbox then I would happily do so.
However, if you continue reading from the guidance above, the 3rd point states that you cannot shut down or sleep your computer before Dropbox finishes uploading. This, for me, is untenable. I frequently work from a laptop which I take to classes to teach from, and a desktop which sits in my office. When it comes to working on my laptop, I regularly shut the screen (i.e. put the computer to sleep) without closing documents. I know that I shouldn’t, but I do. Even if I did manage to teach myself to close every document before I shut my computer screen, the idea that I would have to wait until Dropbox has finished syncing before I shut the screen is not something that I can guarantee.
The nature of my job is that I get interrupted frequently. Often, I will be working on my laptop and someone comes to talk to me so I shut the screen, or if I’m in my office I may quickly put my desktop to sleep. There is absolutely no way I can guarantee that my computer won’t go to sleep before Dropbox has finished uploaded.
What if a student knocks on my door and wants to talk with me? I am supposed to tell them to wait because I can’t allow my desktop to sleep, or shut my laptop screen, until Dropbox has finished syncing? This is an unreasonable requirement for me, as it is not something I can guarantee will always happen.
And, even if I attempt to follow this workflow with Dropbox and carefully manage when my computer goes to sleep, I am human and I get things wrong, especially if I am in a hurry. It seems that if I get this wrong, there is a risk that I may lose my work which is simply not something I am willing to accept.
Finally, what if the network goes down during the syncing process? That is absolutely not something you can guarantee won’t happen, or something that you can predict and mitigate for.
You may be thinking, “But Anna, you’ve been using Scrivener for a couple of years and you’ve only lost your work once, so this is unlikely to happen to me!” And yes, I have been using Scrivener for a couple of years. But, before 3 weeks ago, I was only working on my thesis from my laptop computer. I had my thesis stored locally on my hard drive which was frequently backed up to my wireless external hard drive at home. When I was working on my thesis at the office, I would connect my laptop to my desktop as a second screen, i.e. I was still only storing my Scrivener file locally. I didn’t use cloud storage for syncing, or backup. I used to work locally only.
So what changed? Well, my laptop is getting a bit old now, and I am starting to move all of my work to cloud storage so that I can use the iMac in my office and my MacBook Pro. Three weeks ago, I happily and merrily moved my thesis.scriv file to my Google Drive folder thinking it would be safe and it would actually be sensible to store it on the cloud rather than locally. Sadly, I was very, very wrong. Moving to the cloud caused a conflict somehow, I’m still not entirely sure how. Last time I shut my thesis document, my Lit Review chapter was sitting pretty on 16k words. The next time I opened it on the same device, a strange error message came up that I didn’t really understand. And then, when I looked at the Lit Review word count it had reduced to 12k words. If I hadn’t looked at the word count the last time I saved my work and again when I opened the file yesterday I could easily still be none the wiser about the lost content. I am sure I would have eventually noticed the large amount of content missing, but possibly not for a few days or a week or so, and probably not in a way that meant I could recover the content.
It only took me 3 weeks to encounter this problem. To me, that suggests it is something that is likely to happen again if I was to continue to save my .scriv file using Google Drive.
Anyways, amid the panic of a missing 4k words, I remembered that one of the last things I had been working on was adding a complex table to the document to provide a summary of my literary analysis. So, I found the location where that table should have been and it was gone. At this point, I started looking at the modification dates of all aspects of the file and noticed the most recent modification was on 1st April. That was two weeks ago.
What happened next? Well, I vented on Twitter and did some investigating and found the links above about the limitations of Scrivener with regard to cloud storage. And then I had to go and give a lecture on Software Requirements and Design where we discussed how important it was to understand User Requirements before development and be able to manage, mitigate the impact of and respond to changing requirements. Ah, the irony.
The good news is that I did manage to find a large section of content which had been deleted in this debacle. By accessing the Docs folder of my Scrivener project on Google Drive I found that file 609.rtf had been created recently. It turns out, that file 609.rtf contained the 4000 words that had been deleted. Unfortunately, the Activity panel only contained updates from Saturday and before the 1st April. So that’s still 1 week of missing content and I never did manage to recover my missing table. The fact that Google Drive cloud storage was simultaneously the cause of the problem, and the solution to the problem is probably somewhat amusing. But right now, I’m still too bitter to be amused by anything.
So, am I going to continue to use Scrivener? Well, the jury is still out. I’m kind of mad at Scrivener right now – and yes, I blame Scrivener and not Google Drive. Perhaps that is unfair, but in 2016 I think software developers should be able to create software that is safe when used with cloud storage. Scrivener is not. If my Reference Management software (I use Papers) can safely keep my pdfs, notes, comments etc. etc. etc. up to date via Dropbox so that I can use Papers across my laptop, desktop AND tablets then it is not unreasonable to expect that Scrivener can do the same. If Scrivener could update their product to work reliably with Dropbox I would be a very happy customer. But, right now, I am pretty pissed off and my stress levels are higher than they need to be.
I really do find Scrivener to be a powerful piece of software which strongly supported my writing activities and workflow pre-cloud. At some point, I will do a review of why Scrivener is a fantastic piece of software for large writing project, such as your PhD thesis. But, being unable to reliably sync or store my thesis on the cloud is a critical flaw that I’m not sure I am willing to accommodate. From herein I will be frequently using 2 different computers to write my thesis. I am considering going back to just working locally from a single device, but with my laptop becoming creakier that device would have to be the desktop computer in my office, which for complicated reasons I cannot back up. And, would mean I cannot work from a coffee shop or the library or any other impromptu location I happen to find myself in with a spare 10 minutes to bang out some words.
I guess an alternative is writing my thesis in Word. But I am also vehemently against that option because Word. But also, because Scrivener provided a lot of very useful functionality which I would seriously miss. What a dilemma!
If anyone has any recommendations for software that supports large writing projects I would love to hear them. Or, if you have any ideas on how I can safely sync my .scriv project across multiple devices that would be great. I am looking into RSYNC, but given the problems I have encountered this week I am nervous.