files lost, meta data remains

chrissydid · 02-14-2016, 12:17 PM

Discovered when looking for a book today that many of my library entries have no formats associated with the file.

In October 2015 I had a similar issue caused from a synch error with dropbox where I previously held my library. When the 10-15 error occurred the library was corrupted, many things unsynced. I was able to restore the library via a backup of the entire library. In the restore I moved my library completely out of dropbox to prevent this happening again. —Meant I gave up ability to access files on other devices, but worth it for safety—

At the time, when I restored, not all files were able to be matched back up with their metadata/library entry. I readded by hand several hundred files...

——> back to now

I ran a library maintenance and found that over 3,000 entries had no file format associated. Not knowing how to fix, I imported the backup I used to fix the library in October.

Now I'm stuck with cleanup trying to merge together library entries and it is going to take a ridiculously long time.

How do I avoid this again?

The library should not be syncing with anything. I cannot ascertain how or what happened. I haven't run anything particularly special with the library since the last debacle. I have only added a handful of books. I don't sync via calibre with any devices. (I use rudimentary kindle app 'send to kindle' to read through Kindle program on my ipad)

Can anyone advise on what could be going on? I just want to protect my library and not have to go through this again.

Could this have happened because of an update? That is the only action I have taken with the program. Please and thank you

eschwartz · 02-14-2016, 04:06 PM

This typically happens ass a direct result of syncing.
You CAN sync safely, you just need to follow a couple basic rules.
Always sync fully before starting calibre. Pause syncing while calibre is running. After closing calibre, fully sync all new changes.

Failure to do so can bork your database.
Or cause you to have Dropbox "conflicted copies", which you must fix by hand.

chrissydid · 02-14-2016, 07:54 PM

thanks for your reply-- indeed i think this was the issue with the error in october. However, I'm concerned b/c it has happened again despite there being no sync established ...

Could there be another cause? My library is an awful mess right now. It will take me several days to manually fix it. I just want to prevent it happening again.

eschwartz · 02-14-2016, 09:33 PM

Well, it didn't happen on its own... something other than calibre must have been changing those files.

theducks · 02-14-2016, 11:46 PM

I will echo: Didn't happen on its own.

Calibre does not tolerate OS level messing below the skirts with filenames or folders.

(that includes direct Adding via the OS) Only Calibre should be used to add/remove/rename books

Calibre was not designed for Multi-access (2+ simultaneous computers)

Active Sync can fight changes. Always pause before running Calibre and allow it to complete before connecting to another/same Calibre again (your connection may be lots slower than your local drive. Many changes can take TIME to sync)

disable the active A/V scan of the Calibre Library folder (those books were checked when you downloaded them, anyway). That can slow file updates.

itimpi · 02-15-2016, 03:52 AM

Quote:

Originally Posted by chrissydid

Could there be another cause? My library is an awful mess right now. It will take me several days to manually fix it. I just want to prevent it happening again.

Another common cause is using the library stored on a NAS or networked computer. Only libraries stored on local drives are officially supported.

chrissydid · 02-19-2016, 09:10 AM

Again, I appreciate the response. I don't mean to imply it magically acted this way on its own.

Yet, to reiterate, I am not knowingly synching the library at all. I am also not on a network or shared drive—that's a good catch, I could imagine this potential if I was in a network. I use calibre exclusively for maintaining my library—no outside fiddling, I promise. (When I send a file to kindle app steps are: open calibre, select book that has mobi format, open "Send to Kindle", drag calibre book entry to "send to kindle", it auto detects mobi format/lets me adjust title-author, then it uploads to kindle cloud as 'document', from that point on i'm reading/notating the kindle cloud file not my personal copy)

I am working off of a single iMac running OS 10.11.2.

I appreciate the tips if I were to choose to tempt fate and try to connect a device to calibre or sync it in some way.

However, as I have not been syncing and an error occurred, can anyone advise? Is there perhaps a diagnostic to run to check if anything is attempting to sync?

Again, the only thing I had given permission in the past was dropbox and I have since moved the library out of dropbox's reach. Many, many calibre versions previous I attempted to use Marvin and Calibre, but I was never able to make it work appropriately without causing shutdowns, so I uninstalled it.
¯\_(ツ)_/¯

Quote:

Originally Posted by theducks

I will echo: Didn't happen on its own.

Calibre does not tolerate OS level messing below the skirts with filenames or folders.

(that includes direct Adding via the OS) Only Calibre should be used to add/remove/rename books

Calibre was not designed for Multi-access (2+ simultaneous computers)

Active Sync can fight changes. Always pause before running Calibre and allow it to complete before connecting to another/same Calibre again (your connection may be lots slower than your local drive. Many changes can take TIME to sync)

disable the active A/V scan of the Calibre Library folder (those books were checked when you downloaded them, anyway). That can slow file updates.

02-14-2016, 12:17 PM	#1
chrissydid Junior Member Posts: 8 Karma: 10 Join Date: Aug 2014 Device: ipad mini	files lost, meta data remains Discovered when looking for a book today that many of my library entries have no formats associated with the file. In October 2015 I had a similar issue caused from a synch error with dropbox where I previously held my library. When the 10-15 error occurred the library was corrupted, many things unsynced. I was able to restore the library via a backup of the entire library. In the restore I moved my library completely out of dropbox to prevent this happening again. —Meant I gave up ability to access files on other devices, but worth it for safety— At the time, when I restored, not all files were able to be matched back up with their metadata/library entry. I readded by hand several hundred files... ——> back to now I ran a library maintenance and found that over 3,000 entries had no file format associated. Not knowing how to fix, I imported the backup I used to fix the library in October. Now I'm stuck with cleanup trying to merge together library entries and it is going to take a ridiculously long time. How do I avoid this again? The library should not be syncing with anything. I cannot ascertain how or what happened. I haven't run anything particularly special with the library since the last debacle. I have only added a handful of books. I don't sync via calibre with any devices. (I use rudimentary kindle app 'send to kindle' to read through Kindle program on my ipad) Can anyone advise on what could be going on? I just want to protect my library and not have to go through this again. Could this have happened because of an update? That is the only action I have taken with the program. Please and thank you

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
meta data	kcmccombs	Library Management	3	12-07-2013 05:29 AM
Lost Kindle Data files	Alastair_Lack	Amazon Kindle	5	06-28-2013 03:31 PM
set meta data with ebook-meta and ebook-convert	krischik	Calibre	6	01-19-2010 11:40 AM

02-14-2016, 04:06 PM	#2
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	This typically happens ass a direct result of syncing. You CAN sync safely, you just need to follow a couple basic rules. Always sync fully before starting calibre. Pause syncing while calibre is running. After closing calibre, fully sync all new changes. Failure to do so can bork your database. Or cause you to have Dropbox "conflicted copies", which you must fix by hand.

02-14-2016, 07:54 PM	#3
chrissydid Junior Member Posts: 8 Karma: 10 Join Date: Aug 2014 Device: ipad mini	thanks for your reply-- indeed i think this was the issue with the error in october. However, I'm concerned b/c it has happened again despite there being no sync established ... Could there be another cause? My library is an awful mess right now. It will take me several days to manually fix it. I just want to prevent it happening again.

02-14-2016, 09:33 PM	#4
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Well, it didn't happen on its own... something other than calibre must have been changing those files.

02-14-2016, 11:46 PM	#5
theducks Well trained by Cats Posts: 29,944 Karma: 55705602 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I will echo: Didn't happen on its own. Calibre does not tolerate OS level messing below the skirts with filenames or folders. (that includes direct Adding via the OS) Only Calibre should be used to add/remove/rename books Calibre was not designed for Multi-access (2+ simultaneous computers) Active Sync can fight changes. Always pause before running Calibre and allow it to complete before connecting to another/same Calibre again (your connection may be lots slower than your local drive. Many changes can take TIME to sync) disable the active A/V scan of the Calibre Library folder (those books were checked when you downloaded them, anyway). That can slow file updates.

Advert

Advert