To clarify: it might not make sense to create separate repositories for older releases that we don't expect users to use much anymore, like 1.2 and 1.3, but I think it might make sense to start using separate repositories for future new releases. Maybe we could start this with MaxScale 2.0?

Another option: instead of using separate repositories for different MaxScale versions, maybe we could keep them all in the same repository, but use different package names. e.g. maxscale20 for MaxScale 2.0 and maxscale 21 for MaxScale 2.1. I believe Percona does something like this with XtraBackup's package names. For example, they have percona-xtrabackup-22 for XtraBackup 2.2 and percona-xtrabackup-24 for XtraBackup 2.4: http://repo.percona.com/release/7Server/RPMS/x86_64/

I'm not sure which approach would be better.

GeoffMontee, pinning a particular release of MaxScale doesn't really make sense like it does with the server. Typically, there's a single "current" release of MaxScale at any one time. There isn't concurrent development or bugfixing for multiple branches of MaxScale.

When the first release of MaxScale 1.3 was made, no further updates were made to the MaxScale 1.2 line, so users of that software were running buggy software with potential unfixed security issues.

Hopefully the upgrade process for MaxScale will smooth out over time, making problems like the upgrade from 1.2 to 1.4 less serious in the future.

If a user needs to stay on a specific version of MaxScale, they can put a specific version into the repository URL in their .repo or .list file. This can be done on installation by adding a path component like "/mariadb-maxscale=1.4.3/" (for example /generate/10.1/mariadb-maxscale=1.1/mariadb-enterprise-repository.rpm) when fetching the repository configuration package.

The MaxScale release process at present does not include symlinks such as 1.4 -> 1.4.3, though adding those would allow pinning to a specific release family/branch, so perhaps that's a task to be proposed to the MaxScale team.

Percona's XtraBackup package naming works a little differently than what you suggest. The newest GA release gets the package name "percona-xtrabackup". If XtraBackup 2.2 is the newest GA line, a user will install percona-xtrabackup to get, say, XtraBackup 2.2.11. When their package manager updates packages, they'll get XtraBackup 2.2.12. Once XtraBackup 2.3 is GA, the package manager will upgrade to 2.3.0 for example, because now XtraBackup 2.3 packages will be named "percona-xtrabackup". The next release of the old 2.2 line will have a package name of percona-xtrabackup-22, so if a user wants XtraBackup 2.2.13 they need to install percona-xtrabackup-22 instead of percona-xtrabackup. This of course only works if there are additional releases in an older line after a newer line has been declared GA.

Thanks for the response, kolbe.

Hopefully the upgrade process for MaxScale will smooth out over time, making problems like the upgrade from 1.2 to 1.4 less serious in the future.

Yep, I hope so!

If a user needs to stay on a specific version of MaxScale, they can put a specific version into the repository URL in their .repo or .list file. This can be done on installation by adding a path component like "/mariadb-maxscale=1.4.3/" (for example /generate/10.1/mariadb-maxscale=1.1/mariadb-enterprise-repository.rpm) when fetching the repository configuration package.

Hmm, I'm not sure that this solves the primary problem. The primary problem is that users expect upgrades of "Enterprise" packages to be backward compatible unless they explicitly want to upgrade to a new major release--especially user who are used to RHEL, which is all about stability. But these users still want to make sure they have the latest patches for their minor release, so they still want to try upgrading periodically.

If I understand your solution correctly, it sounds like your solution to pin a specific release would mean the user is consciously making a decision to not upgrade at all.

The MaxScale release process at present does not include symlinks such as 1.4 -> 1.4.3, though adding those would allow pinning to a specific release family/branch, so perhaps that's a task to be proposed to the MaxScale team.

The ability to pin a specific release family sounds like it would be better than pinning a release.

Percona's XtraBackup package naming works a little differently than what you suggest. The newest GA release gets the package name "percona-xtrabackup". If XtraBackup 2.2 is the newest GA line, a user will install percona-xtrabackup to get, say, XtraBackup 2.2.11. When their package manager updates packages, they'll get XtraBackup 2.2.12. Once XtraBackup 2.3 is GA, the package manager will upgrade to 2.3.0 for example, because now XtraBackup 2.3 packages will be named "percona-xtrabackup".

Yes, I'm aware that Percona XtraBackup also has a "percona-xtrabackup" package that points to the latest GA release.

The next release of the old 2.2 line will have a package name of percona-xtrabackup-22, so if a user wants XtraBackup 2.2.13 they need to install percona-xtrabackup-22 instead of percona-xtrabackup. This of course only works if there are additional releases in an older line after a newer line has been declared GA.

This is exactly the point that I wanted to make: users have the option to stick to an older XtraBackup release if they want to. If users want to make sure that they stick with Percona XtraBackup 2.2 to prevent backward compatability issues, they can install "percona-xtrabackup-22" instead of "percona-xtrabackup." It sounds like some users might appreciate the same kind of capability for MaxScale.

My point was that the percona-xtrabackup-22 package doesn't exist until XtraBackup 2.3 is released — until then XtraBackup 2.2 is just called percona-xtrabackup (there was no such thing as "percona-xtrabackup-22" until XtraBackup 2.3 was GA). So unless the user makes a conscious choice, when XtraBackup 2.3 becomes available, to replace percona-xtrabackup with percona-xtrabackup-22, they'll be upgraded to 2.3. That's pretty much the same situation that I'm suggesting, where a user would make a conscious choice to pin their repository to a specific release (major, minor, or patch) of MaxScale. (That requires the MaxScale folks to create and update a symlink of, for example, 1.4->1.4.3, as part of their release process, which is not done today.)

Conceptually, AFAIK, a MaxScale "release family" such as 1.3 or 1.4 is effectively EOL as soon as the next release is made. (In some cases, only a single release was made before the version was incremented, such as 1.3.0, the last release before 1.4.0, which introduced the upgrade problem in question.) As long as MaxScale's release process includes a single active release at any time, I can't see any way to effectively provide the kind of backwards compatibility guarantee you suggest by default.

I think this is ultimately an issue of MaxScale upgrade stability and release planning rather than an issue that can be effectively solved by changes to package repositories.

My point was that the percona-xtrabackup-22 package doesn't exist until XtraBackup 2.3 is released — until then XtraBackup 2.2 is just called percona-xtrabackup (there was no such thing as "percona-xtrabackup-22" until XtraBackup 2.3 was GA).

Sure, good point. That might be how Percona does it, but that doesn't necessarily mean that we need to emulate exactly how they do it. Unless I'm missing something, I don't think there's anything stopping us from creating the "maxscale20" package as soon as MaxScale 2.0 is released.

So unless the user makes a conscious choice, when XtraBackup 2.3 becomes available, to replace percona-xtrabackup with percona-xtrabackup-22, they'll be upgraded to 2.3. That's pretty much the same situation that I'm suggesting, where a user would make a conscious choice to pin their repository to a specific release (major, minor, or patch) of MaxScale. (That requires the MaxScale folks to create and update a symlink of, for example, 1.4->1.4.3, as part of their release process, which is not done today.)

If the MaxScale team creates and updates that symlink and if this "release pinning" technique is documented, users might be fine with that.

Conceptually, AFAIK, a MaxScale "release family" such as 1.3 or 1.4 is effectively EOL as soon as the next release is made. (In some cases, only a single release was made before the version was incremented, such as 1.3.0, the last release before 1.4.0, which introduced the upgrade problem in question.) As long as MaxScale's release process includes a single active release at any time, I can't see any way to effectively provide the kind of backwards compatibility guarantee you suggest by default.

One possible way to do it would be to always keep the latest release of the past 3 or so MaxScale release families in the repository at a time. For example, when MaxScale 2.2 is released at some point in the distant future, we could also keep the latest MaxScale 2.1 and 2.0 releases in the repository.

Of course, we would have to have a disclaimer for users that MaxScale 2.1 and 2.0 probably wouldn't get any more bug fixes after the release of 2.2. But if users are sticking with an older version, it's probably because they have already tested that that particular version works well for them, and they do not want to upgrade until they have performed their tests with the new version.

I think this is ultimately an issue of MaxScale upgrade stability and release planning rather than an issue that can be effectively solved by changes to package repositories.

It would be great if MaxScale upgrades were more stable, but I'm skeptical that that will happen anytime soon with how quickly MaxScale has been changing recently (which isn't really a bad thing).

Do we know yet if MaxScale 1.4 configuration files, password files, privileges, etc. will work out-of-the-box with MaxScale 2.0? Or will users have to perform more post-upgrade steps to get their MaxScale instance working, like they did from 1.3 to 1.4?

We actually keep all historical MaxScale releases in the repository. You can replace "latest" in the repository URL (in the .repo or .list file) with the version number of any MaxScale release you want, and you should get the packages for that release.

Kolbe> "There isn't concurrent development or bugfixing for multiple branches of MaxScale."

There has not been, but that may change now that the license is BSL.

Suppose we have released MaxScale 2.0 and 2.1 and are working on 2.2. All of them are still BSL, but with different change dates. If now a severe security flaw is found in 1.4.3 (GPL) we may decide to fix it and release a 1.4.4.

If that security flaw is still present in 2.2, it is not sufficient to fix it there, but we have to forward port the 1.4.4 fix to 2.0 and 2.1 as well. Otherwise 2.0 (and 2.1 as well) will, when it becomes GPL, be "inferior" to 1.4.4.

And if we do the above, that may trigger new packages and new releases of the 2.0 and 2.1 branches.

Johan, good points. Let's make sure that whatever plans are made are communicated to people who handle package & repository publishing.

solution from Kolbe:MaxScale$

ls -ld 1.4 2.0 2.1 beta latest
lrwxrwxrwx 1 kkegel employees 5 Apr 21 21:45 1.4 -> 1.4.5
lrwxrwxrwx 1 kkegel employees 5 Apr 21 21:45 2.0 -> 2.0.5
lrwxrwxrwx 1 kkegel employees 5 Apr 21 21:45 2.1 -> 2.1.2
lrwxrwxrwx 1 kkegel employees 3 Apr 21 21:45 beta -> 2.1
lrwxrwxrwx 1 kkegel employees 3 Apr 21 21:45 latest -> 2.0