Scrivener and the Case of Disappearing Content

I am deep into the write up of my PhD thesis at the moment. For the past couple of years I have been using Scrivener to write my thesis. Scrivener is a really awesome piece of software for writing large reports. The learning curve is not inconsequential, but when you figure out how to use it, Scrivener is fantastic. At least, that’s what I would have said before I temporarily lost a lot of my work, and permanently lost some of it, after I moved my thesis.scriv document to Google Drive.

I used to adore Scrivener and would recommend it to anyone who asked me what I was using to write my thesis. Only 2 weeks ago I was having lunch with a colleague and we were talking about writing workflows and I got my laptop out of my bag to show them how I was writing my thesis with Scrivener and why I thought it was a really amazing piece of software. I had been planning on writing a post about my writing workflow with Scrivener, and I am sad that I am now having to write a more critical review.

So, why have I fallen out of love with Scrivener? Well, yesterday I lost 4000 words of my literature review and 10 days worth of edits to the thesis. After much panic and stress I did manage to recover most of the missing literature review words, but I know that I have definitely lost some content and I am uncertain if my other edits are included in the current version of my thesis document or not. At the moment, the modification date of each section of my thesis is set to 2 weeks ago, so my guess is the thesis contains old content and not the refined version I spent my Annual Leave from my job as a lecturer working really flipping hard on.

When I realised I had lost the work, my initial reaction was to cry. I was finally making some really good progress on a really difficult part of my thesis, and then my software failed and I lost a huge section of writing that I thought was actually decent quality. Although I did manage to find most of the content, I definitely lost some even though I thought I was being sensible by storing my work on the cloud rather than on an ageing laptop.

I want to save people from the pain and panic of lost words, so if you use Scrivener, or are thinking about using Scrivener please heed the following warning.

Scrivener does not play nicely when your project is stored on the cloud. 

That is, if you save your .scriv file in your Google Drive you should stop immediately. On their Scrivener support pages, Literature and Latte (the company that make Scrivener) state that:

“At this time we strongly discourage the use of Google Drive for synchronising live projects. While this service appears to be stable for the transfer of simple files, complex formats like Scrivener have been known to cause it to fail, reverting, corrupting or even erasing months of work. In the cases we have handled, there has been no way to recover the data if local backups were not available. Please consider using another service for your synchronisation.”

So, even the company that makes Scrivener says don’t synchronise using Google Drive. The suggestion seems to be that this is a “Google Drive” problem, rather than a Scrivener problem. And whilst they might be right in some sense, I don’t think it’s an unusual user requirement to be able to store work using a cloud service nowadays. And Google Drive is one of the most popular cloud services.

The above quote does imply that another cloud synchronisation service should be suitable. Interestingly, there are no suggestions as to which syncing products are safe to use with Scrivener. But, another immensely popular syncing service is Dropbox. So, what about Dropbox? Will Dropbox play nicely with Scrivener? Well, if you use Dropbox with Scrivener you should read these instructions very carefully.

To summarise the key points from the link above, if you want to use Dropbox to synchronise your Scrivener project then you need to ensure you don’t try to open the same project on two (or more) different computers simultaneously. If you do open a project on multiple computers, or aren’t too sure if you left it open on another device, they have implemented an error recovery process which allows you to open a copied version instead and prevent conflicts. So that’s sort of okay and probably manageable. It’s not ideal, but there is a fail safe option which won’t override the original document or cause conflicts if you did happen to leave it open on another computer. If this was the only accommodation that had to be made to use Scrivener with Dropbox then I would happily do so.

However, if you continue reading from the guidance above, the 3rd point states that you cannot shut down or sleep your computer before Dropbox finishes uploading. This, for me, is untenable. I frequently work from a laptop which I take to classes to teach from, and a desktop which sits in my office. When it comes to working on my laptop, I regularly shut the screen (i.e. put the computer to sleep) without closing documents. I know that I shouldn’t, but I do. Even if I did manage to teach myself to close every document before I shut my computer screen, the idea that I would have to wait until Dropbox has finished syncing before I shut the screen is not something that I can guarantee.

The nature of my job is that I get interrupted frequently. Often, I will be working on my laptop and someone comes to talk to me so I shut the screen, or if I’m in my office I may quickly put my desktop to sleep. There is absolutely no way I can guarantee that my computer won’t go to sleep before Dropbox has finished uploaded.

What if a student knocks on my door and wants to talk with me? I am supposed to tell them to wait because I can’t allow my desktop to sleep, or shut my laptop screen, until Dropbox has finished syncing? This is an unreasonable requirement for me, as it is not something I can guarantee will always happen.

And, even if I attempt to follow this workflow with Dropbox and carefully manage when my computer goes to sleep, I am human and I get things wrong, especially if I am in a hurry. It seems that if I get this wrong, there is a risk that I may lose my work which is simply not something I am willing to accept.

Finally, what if the network goes down during the syncing process? That is absolutely not something you can guarantee won’t happen, or something that you can predict and mitigate for.

You may be thinking, “But Anna, you’ve been using Scrivener for a couple of years and you’ve only lost your work once, so this is unlikely to happen to me!” And yes, I have been using Scrivener for a couple of years. But, before 3 weeks ago, I was only working on my thesis from my laptop computer. I had my thesis stored locally on my hard drive which was frequently backed up to my wireless external hard drive at home. When I was working on my thesis at the office, I would connect my laptop to my desktop as a second screen, i.e. I was still only storing my Scrivener file locally. I didn’t use cloud storage for syncing, or backup. I used to work locally only.

So what changed? Well, my laptop is getting a bit old now, and I am starting to move all of my work to cloud storage so that I can use the iMac in my office and my MacBook Pro. Three weeks ago, I happily and merrily moved my thesis.scriv file to my Google Drive folder thinking it would be safe and it would actually be sensible to store it on the cloud rather than locally. Sadly, I was very, very wrong. Moving to the cloud caused a conflict somehow, I’m still not entirely sure how. Last time I shut my thesis document, my Lit Review chapter was sitting pretty on 16k words. The next time I opened it on the same device, a strange error message came up that I didn’t really understand. And then, when I looked at the Lit Review word count it had reduced to 12k words. If I hadn’t looked at the word count the last time I saved my work and again when I opened the file yesterday I could easily still be none the wiser about the lost content. I am sure I would have eventually noticed the large amount of content missing, but possibly not for a few days or a week or so, and probably not in a way that meant I could recover the content.

It only took me 3 weeks to encounter this problem. To me, that suggests it is something that is likely to happen again if I was to continue to save my .scriv file using Google Drive.

Anyways, amid the panic of a missing 4k words, I remembered that one of the last things I had been working on was adding a complex table to the document to provide a summary of my literary analysis. So, I found the location where that table should have been and it was gone. At this point, I started looking at the modification dates of all aspects of the file and noticed the most recent modification was on 1st April. That was two weeks ago.

What happened next? Well, I vented on Twitter and did some investigating and found the links above about the limitations of Scrivener with regard to cloud storage. And then I had to go and give a lecture on Software Requirements and Design where we discussed how important it was to understand User Requirements before development and be able to manage, mitigate the impact of and respond to changing requirements. Ah, the irony.

The good news is that I did manage to find a large section of content which had been deleted in this debacle. By accessing the Docs folder of my Scrivener project on Google Drive I found that file 609.rtf had been created recently. It turns out, that file 609.rtf contained the 4000 words that had been deleted. Unfortunately, the Activity panel only contained updates from Saturday and before the 1st April. So that’s still 1 week of missing content and I never did manage to recover my missing table. The fact that Google Drive cloud storage was simultaneously the cause of the problem, and the solution to the problem is probably somewhat amusing. But right now, I’m still too bitter to be amused by anything.

So, am I going to continue to use Scrivener? Well, the jury is still out. I’m kind of mad at Scrivener right now – and yes, I blame Scrivener and not Google Drive. Perhaps that is unfair, but in 2016 I think software developers should be able to create software that is safe when used with cloud storage. Scrivener is not. If my Reference Management software (I use Papers) can safely keep my pdfs, notes, comments etc. etc. etc. up to date via Dropbox so that I can use Papers across my laptop, desktop AND tablets then it is not unreasonable to expect that Scrivener can do the same. If Scrivener could update their product to work reliably with Dropbox I would be a very happy customer. But, right now, I am pretty pissed off and my stress levels are higher than they need to be.

I really do find Scrivener to be a powerful piece of software which strongly supported my writing activities and workflow pre-cloud. At some point, I will do a review of why Scrivener is a fantastic piece of software for large writing project, such as your PhD thesis. But, being unable to reliably sync or store my thesis on the cloud is a critical flaw that I’m not sure I am willing to accommodate. From herein I will be frequently using 2 different computers to write my thesis. I am considering going back to just working locally from a single device, but with my laptop becoming creakier that device would have to be the desktop computer in my office, which for complicated reasons I cannot back up. And, would mean I cannot work from a coffee shop or the library or any other impromptu location I happen to find myself in with a spare 10 minutes to bang out some words.

I guess an alternative is writing my thesis in Word. But I am also vehemently against that option because Word. But also, because Scrivener provided a lot of very useful functionality which I would seriously miss. What a dilemma!

Comments:

If anyone has any recommendations for software that supports large writing projects I would love to hear them. Or, if you have any ideas on how I can safely sync my .scriv project across multiple devices that would be great. I am looking into RSYNC, but given the problems I have encountered this week I am nervous.

Advertisements

2 thoughts on “Scrivener and the Case of Disappearing Content

  1. Great ideas in this post. Thanks for the warnings about Google Drive. I’ve been using Dropbox to sync with Scrivener ever since I first downloaded the software back in 2014. So far—knock on wood—no problems whatsoever.

    Here’s my process. (I use a MacBook Air, so if you use Windows you might be able to find a similar solution)

    – Keep the main file you’re working from in Scrivener on your computer, NOT the cloud.
    – Set up Scrivener to backup as .zip to your a Master Scrivener Backup folder on your computer, NOT the cloud. (This folder is for keeping multiple backups of various works you’re doing in Scrivener.) That way, all your files are in their original format without having to worry about being lost in translation—or, in this case, synchronization. I set it up to save every 5 backups, but you could change this in Scrivener’s Preferences.
    – Choose a folder to keep ALL your Scrivener backups in.
    – Create a folder in Dropbox to sync this folder to.
    – Download MacDropAny: http://www.zibity.com/macdropany.html. This is the important part that saves so much time and hassle—automatically, no less.
    – Set up MacDropAny to sync your offline/local folder to the one on Dropbox. MacDropAny makes this process so easy and literally takes seconds to set up.

    Now, when you close Scrivener (if you’ve enabled backups to save when closing, manually saving, etc.), it will place the backup in your local computer’s folder, which will then sync automatically to Dropbox via MacDropAny.

    Tip: Every once in a while, when I’ve accrued a significant amount of backups in various stages of development, I copy them to a different folder FOR THE SPECIFIC WORK. In other words, the Master Scrivener Folder I referenced above holds ALL the recent backups for ALL my works, while this folder only holds the backups for THAT SPECIFIC WORK, be it a novel, short story, blog post, etc.

    This way, I have the most recent backups synced to Dropbox and ALL backups for each specific work copied to their respective folders. It’s a lot of backups to keep up with, but I usually don’t have to worry about touching the specific backups once I’ve copied them.

    From there, if I need to open a backup on my second computer, a Mac Mini (Mid-2011), I copy the most recent .zip backup file from Dropbox to the computer and unarchive it. This places the actual .scriv file on my computer. I work on this as long as I need to, then Scrivener backs it up like I’ve noted above. When I’m ready to continue the work on my main MacBook Air laptop, I do the same process: copy the most recent .zip file, unarchive/expand it, and now this acts as my “Active” Scrivener project.

    It sounds like a lot, but it’s second nature when you get used to it. And I’m sure everyone’s process will be different. This is the one that works for me.

    Hope this helps. Best of luck to you in your future writing—and syncing—endeavors.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s