Hmm, I don't quite understand what your question is about, can you give examples?

Your Openweathermap link displays an XML document. CONNECT supports XML and JSON files, but what do you mean by API? Do you mean that CONNECT should be able to access documents on the Net resulting from REST commands? I am not familiar with REST but I think it is mostly used from JavaScript. Perhaps it is possible to execute JavaScript from C++.

Sorry for long answer, I mean, can I create table type for example CSV, and in file_name type: http://solr.ad.vz/solr/dictionary/select?wt=csv
If I open this url in browser I get CSV file.

Presently you cannot. Allowing this would be a new development that is not planned yet. But why not?

Understood. If you have such functionality, I think it will be another big step in Connect Engine, it turns out you can use web services with such ease!
The only thing that happens is how to interpret the where filters in the parameters of the web service, i.e. for example

WHERE first_name LIKE 'alex'

will be identical

http://solr.ad.vz/solr/staff/select?first_name=alex

And also for great versatility, the web service can be mandatory authorization via username and password

In any case, thanks for the current functionality! You have made a very handy tool!

Adding this is probably possible but complicated. It requires adding a package to make C++ able to use Rest API, for instance CASABLANCA (Microsoft cpprestsdk) with installation dependent on the operating system. The first test I tried to do on Windows failed to compile the tutorial sample! But I keep it under my mind because it seems to be a way to connect MariaDB to the cloud.

Thank you very much for your interest in the idea! I'll be watching and ready to test if something will be!

Doing some tests I was able to get the Openweathermap file but the solr.ad.vz server cannot be found.

this is normal, since I showed a link to a locally deployed Apache Solr for example this domain on the Internet is actually not Have you already managed to connect to openweather?

Yes I did with a small sample program here attached but not from MariaDB/CONNECT.

I'm unfortunately a bad programmer, and I'm not very good at it . Is there at least one chance to implement this in Connect Engine?

Firstly not into the CONNECT engine itself, because it could not be qualified as GA anymore.
However, this can be added as an OEM module and I am working on it.

I apologize for my amateurishness, and that means GA and OEM?

General Availability means that a product is not into a (alpha, beta or gamma) test status anymore. For CONNECT this is valuable for the current functions and would not be for new additional features until fully tested.

The OEM table type is tables that are not directly supported by CONNECT but are implemented in additional code that CONNECT can use by connecting to an external library. This is documented in https://mariadb.com/kb/en/library/connect-table-types-oem/

This enables me to implement a new table type without modifying the current CONNECT code.

AAA, ok, thanks!

Bingo! I was able to write and execute a REST module and to create a CONNECT table working on a file retrieved from the Web. It is a simplistic first implementation that just copies the web content to a file and makes CONNECT work with it.

The create table is something like:

CREATE OR REPLACE TABLE webd (

  name char(6) NOT NULL `FIELD_FORMAT`='location/name',

  country char(2) NOT NULL `FIELD_FORMAT`='location/country',

  altitude char(1) DEFAULT NULL FIELD_FORMAT='location/location/@altitude',

  latitude char(7) DEFAULT NULL FIELD_FORMAT='location/location/@latitude',

  longitude char(8) DEFAULT NULL FIELD_FORMAT='location/location/@longitude',

  geobase char(8) DEFAULT NULL FIELD_FORMAT='location/location/@geobase',

  geobaseid char(7) DEFAULT NULL FIELD_FORMAT='location/location/@geobaseid',

  calctime char(6) DEFAULT NULL `FIELD_FORMAT`='meta/calctime',

  sunrise char(19) DEFAULT NULL FIELD_FORMAT='sun/@rise',

  sunset char(19) DEFAULT NULL FIELD_FORMAT='sun/@set',

  time_from char(19) DEFAULT NULL FIELD_FORMAT='forecast/time/@from',

  time_to char(19) DEFAULT NULL FIELD_FORMAT='forecast/time/@to'

) ENGINE=CONNECT TABLE_TYPE=OEM  MODULE='Rest.dll' SUBTYPE=REST

FILE_NAME='weatherdata.xml'

CONNECTION='https://samples.openweathermap.org/data/2.5/forecast?q=London,us&mode=xml&appid=b6907d289e10d714a6e88b30761fae22'

OPTION_LIST='Ftype=XML,rownode=weatherdata';

The result (after fixing some bugs due to unrecognized file format):

Format can be XML, JSON or CSV.

It is not much but demonstrates the feasibility of it.

Wow! You're magic! Now I seem to be beginning to understand the essence of OEM. Correctly I understand that this execution at first as though downloads the XML file, and then the standard method CONNECT works with this file?

What will be the difficulty to do so, for example, in the example above the APPID passed through WHERE

I probably have strange questions I just really want to try to make friends Connect(MariaDB) with the search engine Apache Solr, ie with its API, so to access the index Apache Solr directly dialect SQL

It would work the same way. Any URI (http://...) that returns a file (currently XML, JSON or CSV but more could be added) should work. By the way, discovery now works, meaning you are not obliged to define the columns or you can query for what are the columns of the remote file.

About the WHERE clause, it seems difficult to implement it automatically because the syntax may be specific to the web server. However, the whole file will be retrieved and MariaDB will apply the where clause as for any other table type.

To avoid retrieving a huge file entirely, you can also create a table that will use as URI the one specifying the restricted selection. But this one will always return the same restricted contain.

I'd like to make some more tests with different file types. Have you some examples (like the XML one) or how could I use this Apache Solr?

It would be nice if you could do some tests yourself with this new OEM module. However it is not so easy to do. You have first to install the Microsoft cpprestsdk package, compile the OEM source, and place the result in the MariaDB plugin directory. The current version I wrote is for Windows but the package can also be used on LINUX. What operating system are you using. I could make a version for ubuntu and test it but I have no other LINUX versions available.

I use Windows 10.
And can I just use your DLL file, ie how do I understand just copy it to a specific folder MariaDB, or should I do some more preparatory work?
I would just test on the example of Apache Solr

Looking ahead, Apache Solr has the ability of BASIC AUTHENTICATION, specifically its anyway have to use, at least to protect from an outside of the index, is it possible your method is also to use it. I all these details REST authorization not know, in your example,or rather from Open Weather there is issued a TOKEN, and Apache Solr, I do not remember such possibility, there's just a pair of USERNAME/PASSWORD

I also use Windows 10 however, it's not so simple:

You must install the Microsoft Casablanca support that my OEM module uses. This is described in https://github.com/Microsoft/cpprestsdk and for Windows do:

vcpkg install cpprestsdk cpprestsdk:x64-windows

(assuming you have a 64 bits machine)

When I tried it I realized that I did not have vcpkg and I had to install it from https://github.com/Microsoft/vcpkg
These sites explain how to get and install the packages. When installing cpprestsdk (Casablanca) one step install a lot of boost utilities and this can last almost half an hour.

In addition, I had to add the path of where the Casablanca DLLs have been loaded in the PATH environment variable and restart my system.

Then you must compile the EOM module. I cannot just send you mine because it was compiled and linked against my own version of CONNECT. By the way I was obliged to fix a few bug in CONNECT to have discovery working. But with another version, it should work but without discovery, meaning you would have to define manually the table columns. This can be done with Visual Studio if you know how to use it. The issue is that the compilation and link is done against the ha_connect.lib library that is now distributed with MariaDB and should be found in the Plugin directory.

Perhaps I can do this for you if you upload and attach here your ha_connect.lib file of your present distribution (by the way tell me what version of MariaDB you are using) If so, I will be able to send you the pre-compiled EOM module (DLL)

Good day, Olivier!
I tried at myself to make all this (to establish, compile, etc.) but unfortunately nothing turned out. Can I ask you to compile=) I have version 10.3.9

Of course you cannot compile, you do not have the source code of the OEM module!

So I have installed MariaDB 10.3.9 and compiled the module. I have uploaded it here and a Readme.txt file explaining how to use it.

Let me know about failure or success!

bertrandop told me that:

The REST requests are now implemented in CONNECT and allow you to create tables based on web APIs, without even describing the columns.