[MDEV-16590] Add Push Notification Plugin similar to LISTEN/NOTIFY in PostgreSQL Created: 2018-06-27  Updated: 2022-07-08

Status: Open
Project: MariaDB Server
Component/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Francisco Dueñas Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: None

Issue Links:
Relates
relates to MDEV-23545 Add Trigger-View capability Open

 Description   

Hi, A good feature that PostgreSQL has is a way to enable PUSH Notifications in the client, so when a process need to update or send info to the client it can be done with this mechanism

Currently there no way to execute external commands to send info to an app, unless we use the https://github.com/mysqludf/lib_mysqludf_sys plugin, but this coudl lead to security issues, The PUSH Notification option could be a secure way to send messages to a certain client/app

This could be achieved by adding a plugin at server side and an addon DLL at client side
More info:

https://www.postgresql.org/docs/current/static/sql-listen.html

https://www.postgresql.org/docs/current/static/sql-notify.html



 Comments   
Comment by Christopher Hartsough [ 2018-08-21 ]

I think this needs to be upgraded in priority. This feature would open MariaDB to very quick response applications. Naturally, games come to mind, more importantly, real-time process monitoring would be easy (easier ?) to do. PostgreSQL has a very fast and elegant solution, its only problem I see is that notification is dependent on the updating of the primary backup server. The bright side is the processes fails it doesn't appear to affect the primary server.

DISCLAIMER: I am NOT an expert; if I get wrong, PLEASE make the correction.

Comment by Daniel Black [ 2018-08-21 ]

I'm of the opinion that message queue services (like RabbitMQ) have nicer APIs and a storage/engine architecture around queues that is better suited to this problem as I understand it. Even if you just use if for pushing the primary key of elements and let the app retrieve it over SQL.

Comment by Francisco Dueñas [ 2019-09-20 ]

This should be take into account in the future, Other databases additional to PostgreSQL like FireBird hasta also that feature, and will open new ways for MariaDB to be used in other kind of developments.

Please Give your vote also to this option

Comment by Sergei Golubchik [ 2019-11-21 ]

fduenas, could you please explain how you're using or plan to use this feature? And what for?

Comment by Francisco Dueñas [ 2019-11-26 ]

I Plan to use it as a mechanism for Publish/subscribe/queing application that can implement in an easy way that can of architecture.
For example, Normally without this I have to put a Thread in my app to read a table in mariaDB for the Next Task to Do.  For now I use a Library named  https://github.com/mysqludf/lib_mysqludf_sys plugin, which one of those let you execute external applications, but those could lead you to security issues.

This kind of Publish/Notify architecture can be used in games, or helper applications, console tools, to execute certain tasks or process, like sending emails, processing and generating reports on demand, etc.

My proposal is for trying to implement a way top interact with external environment tools, Without compromising MariaDB Security and needed external plugins that we must compile or patch to keep them working.

Also another useful feature to add let execute HttpPost and Httprequest, this is another way to interact with external apps.

Comment by Sergei Golubchik [ 2019-12-01 ]

Do I understand correctly, that you want to subscribe to, say, table changes? To get notified when a "next task to do" is inserted into a queue table or something?

Comment by Francisco Dueñas [ 2019-12-10 ]

Hi Sergei. Well that will be one of the uses, but it can be also used for Publish/subscribe purposes, where I can send a notification with a Paypload (extra data, that could be In JSON format or on any other text representation) to another external app that has been subscribed for a specific notification/message will execute a specific task depending on the notification data.

It could be database related or maybe a push notifcation to excecute specific tasks (send emails, process and generate reports, execute Shell task, etc)

Comment by Sergei Golubchik [ 2019-12-12 ]

But if it's not related to tables or anything, just sending a message to another application, why would not do it in a database? It seems that a database is not a generic message passing broker.

Being notified when a table is updated is, in a way, what a database could do. And one can [ab]use this feature by creating a table message_queue and letting an application to subscribe to it. But a fully generic message passing service between two applications — I'm not sure that it belongs in a database.

Comment by Francisco Dueñas [ 2019-12-17 ]

No I was thinking just message from mariaDB server to another app, never to be used as a gateway between two apps.

And Yes It will be related to tables, for example if could be used by SQL Monitors or Benchmark tools.
But also could help to send infor for other stuff, but always related to data that is being updated in tables.

Comment by Francisco Dueñas [ 2020-01-20 ]

Another useful example that can be used with MariaDB is that, there is no built-in commands to execute OS commands, this is for security reasons.
But we as develoeprs can Create an application that its only purpose will be only to run commands. for example create folders in the server storage device, so we can generate files from a stored procedure and store them to a specific folder. Inside that stored procedure we can send a Push notification, so we can tell the App to execute certain command. It can be to create a folder, or to move a file from a specific folder to a new one.
A special execution permission can be added so only Granted users can execute a Push notifications.

Comment by Todd Hubers [ 2020-08-23 ]

I added a related task request - MDEV-23545.

But I would still like to see this Notify/Listen mechanism implemented in MariaDB. Postgres isn't the only database to implement this kind of feature, there is also:

Comment by Todd Hubers [ 2020-08-25 ]

This could be implemented as per comment https://jira.mariadb.org/browse/MDEV-23545?focusedCommentId=164026&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-164026 on https://jira.mariadb.org/browse/MDEV-23545 in the Mechanics Progress section - using TableWait/TableSignal.

That is:

  • No data to send - simpler to implement - followup with SELECT
  • Partitioned per-table (and per schema) - enabling fine-grained security enforcement. Should User B know that a new Sale record was inserted?

Ideally this would become some sort of standard SQL too:

CREATE SIGNAL Inserted ON mydb.Sales 
GRANT WAIT * On mydb.Sales TO 'User B'
GRANT WAIT * On mydb.* TO 'User C'
GRANT WAIT Inserted On mydb.* TO 'User D'
GRANT PULSE * On mydb.Sales TO 'User E'
GRANT PULSE * On mydb.* TO 'User F'
GRANT PULSE Inserted On mydb.* TO 'User G'
 
WAIT Inserted ON mydb.Sales TIMEOUT 300 -- or the default, NO TIMEOUT
-- Somewhere else
PULSE Inserted On mydb.Sales

Comment by Todd Hubers [ 2020-09-08 ]

A gust post from Percona - https://blog.engineyard.com/5-subtle-ways-youre-using-mysql-as-a-queue-and-why-itll-bite-you

First, let’s look at how to avoid polling. I wish that MySQL had listen/notify functionality, the way that PostgreSQL and Microsoft SQL Server do (just to mention two). Alas, MySQL doesn’t, but you can simulate it. Here are three ideas: [i] use GET_LOCK() and RELEASE_LOCK(), or [ii] write a plugin to communicate through Spread, or [iii] make the consumers run a SLEEP(100000) query, and then kill these queries to “signal” to the worker that there’s something to do

Generated at Thu Feb 08 08:30:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.