What do you mean by reinstalling? What was the exact sequence of events?

Sequence of steps.

1.) Upgraded MariaDB database server (with spider engine installed) from version 10.2.13 to 10.2.14 following the steps recommended in
https://mariadb.com/kb/en/library/upgrading-between-minor-versions-on-linux/

2.) Reinstalling the spider storage engine post-upgrade fails with the error;
ERROR 1062 (23000) at line 365: Duplicate entry 'spider' for key 'PRIMARY'

Okay, let me be more precise. Please provide the exact sequence of command lines that you used for (1) and (2). Thanks.

1.) Steps to deinstall & reinstall MariaDB server 10.2 (spider engine already installed)
apt-get remove --purge mysql-server mysql-client mysql-common
apt-get autoremove
apt-get install mariadb-server
mysql_upgrade -p

2.) Steps to reinstall the spider storage engine.
apt-get install mariadb-plugin-spider

Unpacking mariadb-plugin-spider (10.2.14+maria~xenial) ...
Setting up mariadb-plugin-spider (10.2.14+maria~xenial) ...
ERROR 1062 (23000) at line 365: Duplicate entry 'spider' for key 'PRIMARY'

Unfortunately, this algorithm doesn't work.

1) apt-get remove --purge mysql-server mysql-client mysql-common does remove the server (and actually all mariadb packages), simply because everything depends on mysql-common, and is removed as a dependency. mysql-server and mysql-client are of course not recognized;

2) regardless of (1): even apt-get remove --purge mariadb-server mariadb-client mysql-common (or apt-get purge mariadb-server mariadb-client mysql-common) does not remove /var/lib/mysql, at some point during the process it says this:

Purging configuration files for mysql-common (10.2.15+maria~xenial) ...

dpkg: warning: while removing mysql-common, directory '/etc/mysql' not empty so not removed

and indeed doesn't remove it; however it completely removes all packages, so you can't run purge again afterwards (and subsequent autoremove doesn't purge anything either);

3) when one runs a "normal" remove, like sudo apt-get remove mariadb-server mariadb-client mariadb-common mysql-common, the packages are removed, but they leave

rc  mariadb-client-10.2                10.2.15+maria~xenial            amd64        MariaDB database client binaries

rc  mariadb-common                     10.2.15+maria~xenial            all          MariaDB database common files (e.g. /etc/mysql/conf.d/mariadb.cnf)

rc  mariadb-server-10.2                10.2.15+maria~xenial            amd64        MariaDB database server binaries

rc  mysql-common                       10.2.15+maria~xenial            all          MariaDB database common files (e.g. /etc/mysql/my.cnf)

and if you run sudo apt-get purge mariadb-client-10.2 mariadb-common mariadb-server-10.2 mysql-common afterwards – that's when the /var/lib/mysql gets removed, with a big shiny warning dialog.

I'm reassigning it to otto, the debian packaging expect, to determine if the above is expected behavior, and if not, fix it.

Dermot.Brereton,
So, given the above, when you removed/purged your packages, the vardir with mysql.plugin containing spider remained in place, and subsequent installation of spider failed as it tried to insert it there again.

jacob-mathew, could you please fix spider install script not to fail when spider is already present in the mysql.plugin table?

The Spider install script installs Spider in mysql.plugin only if it is not already present in INFORMATION_SCHEMA.plugins. It is indeed possible to run the script repeatedly without failures.

Note that the error that would occur upon attempting to install Spider when Spider is already installed is the following:

ERROR 1968 (HY000): Plugin 'spider' already installed

jacob-mathew,
I think it's not the point. The script should be able to handle the situation when the plugin is not installed, but an orphan record exists in mysql.plugin.

I now understand the inconsistency that results after upgrades to MariaDB and Spider on Ubuntu. I was able to artifcially create this situation on my development machine and verified that the reported error occurs upon re-installing Spider. I have fixed the problem by making changes to the Spider installation script for recognizing and correcting the inconsistency.

Kentoku, please review my fix for this problem in commit 0897d81 on my branch.

LGTM
It's O.K. to push.

Fix is pushed to 10.3 and 10.2.