New Kobo Libra 2 -- setup questions

[chaley]

My old Boox reader died and I decided to try a Kobo Libra 2. So far so good.

Two questions:

1. I keep my authors as LN, FN. Clearly the Kobo wants them as FN LN. I think I solved the problem with the following plugboard for authors. Is there something else I should do?
   

   program:
   	comma = '';
   	res = '';
   	for author in $authors separator '&':
   		res = strcat(res, comma, swap_around_comma(author));
   		comma = ', '
   	rof;
   	res

2. For collections: I found the configuration to make collections from values in columns. What I want is something that I could do with the old Sony PRS 350: prefix the column name to the collection value (see the tweak "Specify renaming rules for SONY collections"). With this ability and assuming the columns #genre, tags, and #cbhpri, the result would something like
   CBH Pri: high
   Genre: Gobble
   Tags: foo
   Tags: Bar
   
   Is there already some way to do this?

EDIT: Is there any big downside to not entering my Kobo account into the reader other than not being able to buy books from the reader itself?

EDIT 2: Is there a way to give a different device name in the Kobo Utilities plugin devices tab? If this goes well then my wife will also get the same reader and it would be nice to tell them apart other than with the serial number.

[theducks]

I just edited the json value 'name' (inserted a V
It did not seem to barf. BTW Utilities does show Serial number
I also have my devices configured to use different Backup-database folders

[chaley]

Originally Posted by theducks

I just edited the json value 'name' (inserted a V
It did not seem to barf. BTW Utilities does show Serial number
I also have my devices configured to use different Backup-database folders

Thanks. I looked at the json shown by the utilities plugin and didn't see that. I did find it in the json in the config folder.

Now I am fighting with "Database is locked." EDIT: this seems to be my fault. The kobo desktop was running. I killed it, ejected, reconnected, ran chkdsk (as it wanted) and now it works.

[theducks]

It is in Kobo Utilities.json (plugin folder)
Maybe you have to Configure options for each device ticked

[davidfor]

Originally Posted by chaley

My old Boox reader died and I decided to try a Kobo Libra 2. So far so good.

Two questions:

1. I keep my authors as LN, FN. Clearly the Kobo wants them as FN LN. I think I solved the problem with the following plugboard for authors. Is there something else I should do?
   

   program:
   	comma = '';
   	res = '';
   	for author in $authors separator '&':
   		res = strcat(res, comma, swap_around_comma(author));
   		comma = ', '
   	rof;
   	res

Kobo only reads the first dc:creator tag it finds. And assumes the authors are "FN LN" and that multiple authors "FN1 LN1, FN2, LN2, ...". I usually recommend:

program: raw_list('authors', ', ')

But, what you have looks like it should work.

• For collections: I found the configuration to make collections from values in columns. What I want is something that I could do with the old Sony PRS 350: prefix the column name to the collection value (see the tweak "Specify renaming rules for SONY collections"). With this ability and assuming the columns #genre, tags, and #cbhpri, the result would something like
  CBH Pri: high
  Genre: Gobble
  Tags: foo
  Tags: Bar
  
  Is there already some way to do this?

No, there isn't. I remember looking at that when I was writing the collections code, but, decided not to use it. You should be able to do it in a custom column. I have been thinking about allowing the use of a template instead of the list of columns. I am already using them elsewhere, so it would make sense to use them here as well. The main thing is to make sure it is backwardly compatible.

EDIT: Is there any big downside to not entering my Kobo account into the reader other than not being able to buy books from the reader itself?

In the past, you had to do that by updating the database. Kobo added a sideloaded mode in firmware 4.31 which works well. I have one of my devices setup like this, and apart from syncing books to Kobo, I don't see anything missing. I do buy books from Kobo, so I don't use this for the devices I use daily.

EDIT 2: Is there a way to give a different device name in the Kobo Utilities plugin devices tab? If this goes well then my wife will also get the same reader and it would be nice to tell them apart other than with the serial number.

As @theducks said, the device can be renamed in the device list on the Device tab. But, and I had completely forgotten this, you can only rename the device when it is connected. And it is only used by the plugin, not the driver. What I have just noticed is that this is not shown in the tooltip for the button. I'll fix this.

And I wonder if it would be safe for the plugin to set the name used by the driver. I'll have to experiment.

[chaley]

Thanks for the "kobo newbie" help.

Originally Posted by davidfor

Kobo only reads the first dc:creator tag it finds. And assumes the authors are "FN LN" and that multiple authors "FN1 LN1, FN2, LN2, ...". I usually recommend:

program: raw_list('authors', ', ')

But, what you have looks like it should work.

I had seen your recommendation but it only works if the authors are already "FN LN". Mine are "LN, FN" and I don't want to change it. I considered using the following to change the individual authors then change the separator, but decided that the "more traditional" program was easier to read.

program:
	a = list_re_group($authors, '&', '.', '(^.*$)', 'program: swap_around_comma($)');
	re(a, '&', ', ')

No, there isn't. I remember looking at that when I was writing the collections code, but, decided not to use it. You should be able to do it in a custom column. I have been thinking about allowing the use of a template instead of the list of columns. I am already using them elsewhere, so it would make sense to use them here as well. The main thing is to make sure it is backwardly compatible.

Using a template is a great idea. As for compatibility and assuming you don't want to add a checkbox "Columns box contains a template", you can assume it is a template if it starts with "program:" or contains a '{'. No list of lookup names can match that.

Using a template and setting the device name (below) solves another problem, using different collection columns for different devices. The template could condition its output on the value of current_device_name().

[chaley]

I looked at the code for the driver and the utilities and I think that setting the device name will be a giant pain in the *^&. The problem is the sequencing of the operations. The calibre device driver gets and caches the device_info before the utilities can get involved.

I also looked at the code for generating collection values (get_collections). Changing that to use a template will also be a pain because so much of it is built around lookup/collection names.

Fortunately for me, neither of the above is needed. The following solves my problem albeit with a bit of brute force:

• In the utilities plugin, add the ability to copy the device serial number so the user can get it to use in the template.
• Add another box to the driver where I can put a template to format values for collection attributes.
• Change kobo.books.get_collections() to invoke the template if it is defined. If it is defined then you would:
  • Initialize a 'globals' dict with the device serial number, the lookup name being processed, the value generated, and the column heading. You can get the serial number at device-open time without involving the utilities plugin.
  • Initialize a new_vals dict.
  • For each value in the vals dict, call the template. If the answer is non-empty add it to the new_vals dict.
  • Replace vals with new_vals
  Attached is a patch file showing an example of the kobotouch driver changes to do the above. It works for me. NB: I didn't make the utilities change.

[davidfor]

Firstly, looking at the code for the collections, I remember how much I hated it. And, honestly, I didn't understand enough about calibre at the time to do it properly. Plus, it was pretty much my first time coding Python. I think that's why I dropped the SONY options as it was doing my head in at the time. But, looking at it, and the version of "get_collections" in CollectionsBookList, I do not think that it would be that hard to put it back in.

But, I think I'd prefer to add template support.

Can you post the template you are using? Your code isn't what I was thinking of. I think your intention is to use both the collection column list and a template. And the template is applied to the values from the collection columns. When I thought of this, my intention was use the template to return a list of collection names. And they would be used without any further changes.

Which explains why you were adding a field in the config. After yesterday's posts, I was thinking of adding a "Use template" checkbox. When selected the current collection columns field would be replaces by the TemplateConfig widget that is in the configuration code. That is the field plus template editor button and validation. That would be safer than parsing for whether it was a template or not.

I'll add the serial number. The driver is already reading the version file, so it should be getting the serial number.

I didn't mention it yesterday, but, someone asked something similar to this recently. For that, I suggested using the template function "connected_device_uuid". That can be gotten from the driveinfo.calibre file and is how both the Kobo Utilities and Reading List plugins identify a device. Passing that and possibly the firmware version in might be useful.

[chaley]

Originally Posted by davidfor

Firstly, looking at the code for the collections, I remember how much I hated it. And, honestly, I didn't understand enough about calibre at the time to do it properly. Plus, it was pretty much my first time coding Python. I think that's why I dropped the SONY options as it was doing my head in at the time.

By way of some explanation for how the collection code is as it is:

I started using calibre when I bought my first Sony PRS 350 in 2010 (IIRC). I wanted better collections support, specifically collections from calibre fields. I opened the idea with Kovid and he said "Go ahead. No promise I will take it though." Given that it was my first foray into python and calibre code, his position was wise. It is also worth noting that this effort precedes my custom column work by a couple of years.

So, as is usual in these cases I built what I wanted: automatic collections management. By this time Kovid and I decided we worked well enough together and he gave me (more-or-less) free rein. I released it, and oh my did the excrement hit the rotating blades.

There were four general responses:

1. Take it out! Don't you dare touch my collections. I know where you live!
2. Add collections when I send a book but otherwise never change or delete them. I also know where you live!
3. I can see the use. Update (add & remove) the collections for a book when I send it to the device, but otherwise don't touch them.
4. I like it. Do everything manually so I never need to touch the collections.

The above explains where I ended up, adding "Metadata management". You can see the correspondence between the management choices and the requests. Adding the management choices made the code much more complex, and my experience with python didn't help. I had to differentiate between collections the user made and collections the driver made and I had to implement the semantics of the management choice. This shows up in the collections code as the '_new_book' and 'device_collections' stuff.

The Sony shows collection entries in the order they were added, which by the way I really wish I could do on a Kobo. This permitted me to control the display order of collections, in particular the series collection. It also explains why the collection code passes around sort values.

Around this time (2012?) we started working on custom columns, which expanded the universe of collections possible to create. Some issues:

• The same value could appear in different columns, with different meaning. This is why the prefix/suffix stuff tweak was added.
• The sorting "problem" grew bigger. Some data was best presented using a different sort. For example, The user could sort a collection by date or some other value. Finally, some columns were best combined into a single collection, for example multiple series columns. I added a tweak to control how a collection was sorted. By default series-style collections were sorted by the associated series index, and other collections were sorted by title_sort.

All this explains some of the complexity in the collections code.

But, looking at it, and the version of "get_collections" in CollectionsBookList, I do not think that it would be that hard to put it back in.

But, I think I'd prefer to add template support.

I agree. There is no reason to re-add the sony tweaks. There are better ways to do it.

Can you post the template you are using? Your code isn't what I was thinking of. I think your intention is to use both the collection column list and a template. And the template is applied to the values from the collection columns. When I thought of this, my intention was use the template to return a list of collection names. And they would be used without any further changes.

The difficulty here is metadata management. If you support all three choices, and it appears that you do, then you must still distinguish between 'device_collections' and collections generated from metadata.

The technique I used avoids this problem by leaving all the metadata management and device collections stuff as it is. Instead, when collection names are being generated from columns I can remove or rename them.

The template below shows what I mean. I have (actually, will have) two libra 2 readers, mine and my wife's. For mine I want the following collections:

• Authors by first letter of last name. My brain doesn't deal well with lists in FN LN order, sorted by LN. It also doesn't deal well with lists sorted by FN LN. This collection solves the problem by giving me an alternate way to get closer to an author using the first letter of the last name.
• My curated tags.

My wife just wants her curated tags.

The "Collection columns" box contains all the columns we both want to use: "#dthtags, authors, #cbhtags". The template cleans things up using the device serial number to select what to do. For columns not wanted on the device it returns '', removing that collection value. It does this by checking the lookup_name and the serial number.

• For my wife's collections it passes through only values for the #dthtags columns, without a prefix.
• For my collections it:
  • adds a prefix to the collection name. For authors I use the heading defined in calibre. For my tags I use "Tags"
  • for authors it renames the entry to be the first letter of the last name.
  • It removes values for the column #dthtags.

NB: I haven't yet received our second Libra 2 so the existing serial number is used twice. I test it by changing the value in the first 'if'.

program:
	globals(val='Last, First', lookup_name='authors', serial_number='N4181B5019336', 
		    heading='Authors');
	if serial_number == 'N4181B5019336' then
		if lookup_name == 'authors' then return strcat(heading, ': ', substr(val, 0, 1)) fi;
		if lookup_name == '#cbhtags' then return strcat('Tags: ', val) fi;
		return ''
	fi;
	if serial_number == 'N4181B5019336' then
		if lookup_name == '#dthtags' then return val fi
	fi;
	return ''

Which explains why you were adding a field in the config. After yesterday's posts, I was thinking of adding a "Use template" checkbox. When selected the current collection columns field would be replaces by the TemplateConfig widget that is in the configuration code. That is the field plus template editor button and validation. That would be safer than parsing for whether it was a template or not.

I certainly have no objection to having a template that produces the list of collection names. I suspect this won't be easy if you continue to respect metadata management.

EDIT: assuming the template returns a list you will probably need to use a funky list separator like ':::'. There is no reason to believe that values don't contain commas or other simple separators.

I'll add the serial number. The driver is already reading the version file, so it should be getting the serial number.

Getting the serial number in the driver was easy to do. Currently I get the value to put in the template by looking at the version file. It would be nice to have a way to get it from the utilities plugin.

I didn't mention it yesterday, but, someone asked something similar to this recently. For that, I suggested using the template function "connected_device_uuid". That can be gotten from the driveinfo.calibre file and is how both the Kobo Utilities and Reading List plugins identify a device. Passing that and possibly the firmware version in might be useful.

Yes, it could be. That said, I have been mildly unhappy with the UUID system for a long time because the UUID changes if driveinfo.calibre is deleted. For the Kobo the serial number will never change.

[chaley]

Here is a screen capture of the Libra 2 collection page, generated using the template in the previous post.

[chaley]

@davidfor: looking again at the code, I now think I was being way too pessimistic about using templates. If you add the checkbox you suggested then:

• If the box isn't checked then "collection_attributes" is set to the contents of the box as is done today.
• If the box is checked then collection_attributes is set to something like "template", which cannot otherwise appear because there is no standard lookup name 'template'.
• When getting the values for a "collection attribute" (a lookup name), if the name is "template" then get the list of values from the template, otherwise from the column.
• The rest of the processing is unchanged.

This idea leaves open the possibility of intermixing values from columns (lookup names) and results of a template. If instead of the checkbox you add a template box then the code could process any keys in the standard box then process the results of the template. Doing this would support using 'standard' values unchanged along with adding 'custom' values without using custom columns.

Apologies for being a bit dense.

EDIT: if you want me to send you proof-of-concept code for the above, let me know. It is a small change from what I have already done.