Hi all
Sorry, I still have to complain, ggghhh 
I'm still seeing high RAM usage.
And sorry, I did not do any "clean" tests yet - I just felt I had to post what I knew so far (so many stability problems with MariaDB so far => starting to turn my attention towards Apache Kudu & Yandex Clickhouse...).
It's definitely not as obviously visible as with versions equal/lower than 10.2.15, but in my opinion the problem is still there.
Attaching the file "stefano-mariadb-10_02_16-ram-20180825.png":
The graph shows the RSS allocation from a clean start up to ~2.5 billion inserts (2'537'827'547) (some very few selects & updates were executed as well, but at most 0.001% of all DMLs).
My current setting is"tokudb_cache_size = 12G", so I wouldn't expect it to go a lot beyond that - 15GB is definitely beyond my expectations. Sometimes it went up to 26 GBs, but I've always been able to shut it down before it started doing some serious swapping.
I admit that the graph isn't very exciting, but I've seen numbers grow a lot faster under certain conditions (e.g. concurrent queries like "insert into [my_isam_table] select a_single_column from [my_tokudb_tbl]>" + "insert into [my_VEC_table] select some, other, cols from [my_tokudb_tbl]" made my 32GB server start swapping within minutes - sorry, I did not repeat the experiment.
MariaDB 10.1 (used mostly 10.1.34) did not have this problem but totally hanged after ~1 days of continuous "insert"s with no kind of hint of "why" anywhere => I usually had to "kill -9 xxx" the mariadb-process & start it & wait for a recovery", so from this point of view the 10.2-series is better (no hard hangs even after having processed the same amount of data as the 10.1-series) but for me it's still ultimately unstable.
(btw. when using TokuDB the insert-rate with the 10.1-series is still 10-20% faster than with 10.2/3-series; looks like 10.2/3 uses max 25-50-75% of the available vCPUs for the checkpoints, but this is another issue...)
Question:
is anybody doing at least ~1 million "inserts" upwards per minute for an extended amount of time (hours, days, weeks) and if yes, do you see any suspicious RAM allocation/behaviour?
(asking this because the factor of "time elapsed vs. amount of inserts" might have something to do with the problem)
Thanks a lot
Stefano
After another day the mem usage has grown to: 7.3GB of mem and 3.8GB of swap.