[MDEV-6552] TokuDB requires HUGE setting,... Created: 2014-08-08  Updated: 2014-09-15  Due: 2014-09-13  Resolved: 2014-09-15

Status: Closed
Project: MariaDB Server
Component/s: None
Affects Version/s: 10.0.10-galera
Fix Version/s: N/A

Type: Bug Priority: Trivial
Reporter: Joni-Pekka Kurronen Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: None
Environment:

Ubuntu Tryusty

Attachments: File my.cnf    

 Description   

hi,

I moved from 12.04 Ubuntu 14.04.
Installed Mariadb - galera from ditribution and moved old datbases and settings to new one. Old one was build from sources and not galera.

I had to add line:

exec "echo never >/sys/kernel/mm/transparent_hugepage/enabled"

before exec mysqld line at startup script to get mariadb functioning.

That's needed to get TokuDB functionin and maraiadb goin up.

Other abnormal behaviour faced were:

  • mysqld server did hang and prevented normal start and stop,
    no error log messages , THAT WAS BEFORE SCRIPT CHANGE
  • after killing mysqld by KILL and got mysqld -v to show problem
    I COULD NOT TERMINATE started mysqld -v on terminal whit Ctrl-X-C !
    Complains that error.log is old format ! , WHILE GETTING PROBLEM SORTED DUE NO ERROR LOG

 Comments   
Comment by Joni-Pekka Kurronen [ 2014-08-08 ] 

mysql < bacula.sql command run's abt. 8 hours ending TOTAL CRASH OF UBUNTU Trusty desktop whit 32GB ram, 1T RADI1 ( 2x1T ),... after

it it take 3 hours to sync RAID1 disk.

bacula.sql size is abt 3G and useing Tokudb

Log say's very litle:

/usr/sbin/mysqld, Version: 10.0.12-MariaDB-1~trusty-wsrep-log (mariadb.org binary distribution, wsrep_25.10.r4002). started with:
 
Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
 
Time                 Id Command    Argument

This error can be reproduced but take's 8 hours.

Manualy started to starup, need's HUGE SETTING
before start.

joni@mpi1:~$ sudo su
 
root@mpi1:/home/joni# echo never >/sys/kernel/mm/transparent_hugepage/enabled
 
root@mpi1:/home/joni# exit
 
exit
 
joni@mpi1:~$ sudo mysqld
 
140808 18:35:02 [Warning] option 'table_definition_cache': unsigned value 100 adjusted to 400
 
140808 18:35:02 [Warning] An old style --language or -lc-message-dir value with language specific part detected: /usr/local/mysql/share/english/
 
140808 18:35:02 [Warning] Use --lc-messages-dir without language specific part instead.
 
140808 18:35:02 [ERROR] Error message file 'errmsg.sys' is probably from and older version of MariaDB / MYSQL as it doesn't contain all error messages
 
2014-08-08 18:35:02 7f61397067c0 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
 
140808 18:35:02 [Note] InnoDB: Using mutexes to ref count buffer pool pages
 
140808 18:35:02 [Note] InnoDB: The InnoDB memory heap is disabled
 
140808 18:35:02 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
 
140808 18:35:02 [Note] InnoDB: Compressed tables use zlib 1.2.8
 
140808 18:35:02 [Note] InnoDB: Using Linux native AIO
 
140808 18:35:02 [Note] InnoDB: Using CPU crc32 instructions
 
140808 18:35:02 [Note] InnoDB: Initializing buffer pool, size = 8.0M
 
140808 18:35:02 [Note] InnoDB: Completed initialization of buffer pool
 
140808 18:35:02 [Note] InnoDB: Highest supported file format is Barracuda.
 
140808 18:35:02 [Note] InnoDB: 128 rollback segment(s) are active.
 
140808 18:35:02 [Note] InnoDB: Waiting for purge to start
 
140808 18:35:02 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.17-65.0 started; log sequence number 17630569340
 
140808 18:35:02 [Note] Plugin 'FEEDBACK' is disabled.
 
Fri Aug  8 18:35:03 2014 Tokudb recovery starting in env /var/lib/mysql/
 
Fri Aug  8 18:35:04 2014 Tokudb recovery scanning backward from 2822092
 
Fri Aug  8 18:35:12 2014 Tokudb recovery bw_end_checkpoint at 1524828 timestamp 1407507854498401 xid 1524629 (bw_newer)
 
Fri Aug  8 18:35:12 2014 Tokudb recovery bw_begin_checkpoint at 1524629 timestamp 1407507854497794 (bw_between)
 
Fri Aug  8 18:35:12 2014 Tokudb recovery turning around at begin checkpoint 1524629 time 607
 
Fri Aug  8 18:35:12 2014 Tokudb recovery starts scanning forward to 2822092 from 1524629 left 1297463 (fw_between)
 
Fri Aug  8 18:35:19 2014 Tokudb recovery scanning forward to 2822092 at 2030629 left 791463 (fw_newer)
 
Fri Aug  8 18:35:29 2014 Tokudb recovery closing 194 dictionaries
 
Fri Aug  8 18:35:31 2014 Tokudb recovery making a checkpoint
 
Fri Aug  8 18:35:31 2014 Tokudb recovery done
 
140808 18:35:31 [Note] Recovering after a crash using /var/log/mysql/mysql-bin
 
140808 18:35:32 [ERROR] Error in Log_event::read_log_event(): 'Event too small', data_len: 0, event_type: 0
 
140808 18:35:32 [Note] Starting crash recovery...
 
140808 18:35:32 [Note] Crash recovery finished.
 
140808 18:35:32 [Note] Server socket created on IP: '0.0.0.0'.
 
140808 18:35:32 [Note] Event Scheduler: Loaded 0 events
 
140808 18:35:32 [Note] WSREP: Read nil XID from storage engines, skipping position init
 
140808 18:35:32 [Note] WSREP: wsrep_load(): loading provider library 'none'
 
140808 18:35:32 [Note] [Debug] WSREP: dummy_init
 
140808 18:35:32 [Note] mysqld: ready for connections.
 
Version: '10.0.12-MariaDB-1~trusty-wsrep-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution, wsrep_25.10.r4002

mysql

System start's and no entry's at bacula database but all tabel's are created.

MariaDB [(none)]> \s
 
--------------
 
mysql  Ver 15.1 Distrib 10.0.10-MariaDB, for Linux (x86_64) using readline 5.1
 
 
 
Connection id:		83
 
Current database:	
Current user:		root@localhost
 
SSL:			Not in use
 
Current pager:		stdout
 
Using outfile:		''
 
Using delimiter:	;
 
Server:			MariaDB
 
Server version:		10.0.12-MariaDB-1~trusty-wsrep-log mariadb.org binary distribution, wsrep_25.10.r4002
 
Protocol version:	10
 
Connection:		Localhost via UNIX socket
 
Server characterset:	latin1
 
Db     characterset:	latin1
 
Client characterset:	utf8
 
Conn.  characterset:	utf8
 
UNIX socket:		/var/run/mysqld/mysqld.sock
 
Uptime:			5 min 6 sec
 
 
 
Threads: 1  Questions: 158  Slow queries: 0  Opens: 175  Flush tables: 1  Open tables: 88  Queries per second avg: 0.516
 
--------------
 
 
 
MariaDB [(none)]> 
 
 
MariaDB [bacula]> SELECT concat(TABLE_NAME,':',ENGINE) FROM Information_schema.TABLES  WHERE TABLE_SCHEMA='bacula';
 
+-------------------------------+
 
| concat(TABLE_NAME,':',ENGINE) |
 
+-------------------------------+
 
| BaseFiles:TokuDB              |
 
| CDImages:InnoDB               |
 
| Client:TokuDB                 |
 
| Counters:TokuDB               |
 
| Device:TokuDB                 |
 
| File:InnoDB                   |
 
| FileSet:TokuDB                |
 
| Filename:TokuDB               |
 
| Job:InnoDB                    |
 
| JobHisto:InnoDB               |
 
| JobMedia:InnoDB               |
 
| Location:InnoDB               |
 
| LocationLog:InnoDB            |
 
| Log:InnoDB                    |
 
| Media:InnoDB                  |
 
| MediaType:InnoDB              |
 
| Path:InnoDB                   |
 
| PathHierarchy:InnoDB          |
 
| PathVisibility:InnoDB         |
 
| Pool:InnoDB                   |
 
| RestoreObject:InnoDB          |
 
| Status:InnoDB                 |
 
| Storage:InnoDB                |
 
| UnsavedFiles:InnoDB           |
 
| Version:InnoDB                |
 
+-------------------------------+
 
25 rows in set (0.01 sec)
 
 
 
MariaDB [bacula]>

Comment by Elena Stepanova [ 2014-08-08 ]

Hi,

exec "echo never >/sys/kernel/mm/transparent_hugepage/enabled"
That's needed to get TokuDB functionin and maraiadb goin up.

Yes, this is a documented TokuDB requirement which comes from the engine: https://mariadb.com/kb/en/mariadb/documentation/storage-engines/tokudb/how-to-enable-tokudb-in-mariadb/ .
And in your case the whole MariaDB server won't get started, because you set TokuDB as a defalut storage engine, and without the default engine server does not run.

mysqld server did hang and prevented normal start and stop,
no error log messages , THAT WAS BEFORE SCRIPT CHANGE

It's difficult to say anything about hanging without any error messages or a stack trace. As said above, the server was not expected to start, but it should not hang, rather just go down with the failure.

I COULD NOT TERMINATE started mysqld -v on terminal whit Ctrl-X-C !

Over many versions and many years by now, mysqld only handles SIGINT (Ctrl+C) when it's started with --gdb option:

mysqld now only adds an interrupt handler for the SIGINT signal if you start it with the new --gdb option. This is because some MySQL users encountered strange problems when they accidentally sent SIGINT to mysqld threads

http://dev.mysql.com/doc/refman/5.0/fr/news-4-0-14.html

Complains that error.log is old format ! , WHILE GETTING PROBLEM SORTED DUE NO ERROR LOG

I don't quite understand this part. Please copy-paste the exact message that you received.

After crash mysqld start's, but dose not allow connections, at this time i will wait two hours,...
Log say's very litle:

What you quoted looks like the general log. You need to find the error log. It might be a separate file or in your syslog or alike.

Comment by Elena Stepanova [ 2014-08-08 ] 

140808 18:35:02 [Warning] An old style --language or -lc-message-dir value with language specific part detected: /usr/local/mysql/share/english/
 
140808 18:35:02 [Warning] Use --lc-messages-dir without language specific part instead.
 
140808 18:35:02 [ERROR] Error message file 'errmsg.sys' is probably from and older version of MariaDB / MYSQL as it doesn't contain all error messages

These two warnings (and quite possibly the error too) are caused by this line in your cnf:

language = /usr/local/mysql/share/english

Try to comment it and see if the server is able to find lc-messages-dir on its own.

140808 18:35:31 [Note] Recovering after a crash using /var/log/mysql/mysql-bin
 
140808 18:35:32 [ERROR] Error in Log_event::read_log_event(): 'Event too small', data_len: 0, event_type: 0
 
140808 18:35:32 [Note] Starting crash recovery...

Maybe you really have corrupted binary logs?
Do you even need them?
Do you have replication (apart from Galera), or use them for backup?

Comment by Joni-Pekka Kurronen [ 2014-08-09 ]

hi,

Changes to my.cnf removed total crash.

So I assume you know reason for TOTAL CRASH and at any case
it is not acceptable.

Thank you Elena to workaround !

yours,
joni

Comment by Elena Stepanova [ 2014-08-09 ]

Which TOTAL CRASH do you mean? Please specify, since you described several mutually unrelated issues, mainly caused by misconfiguration.
If there was a real crash and not just server going down gracefully due to a wrong configuration, please provide the error log and/or stack trace.

Comment by Joni-Pekka Kurronen [ 2014-08-28 ]

REAL CRASH. TOTAL. power off 4 seconds and start again.

Comment by Joni-Pekka Kurronen [ 2014-08-28 ]

Solution: stop useing tokudb !

Comment by Elena Stepanova [ 2014-09-15 ]

Closing as incomplete for now; if you have any new information (namely, explanation which TOTAL/REAL CRASH you are talking about, with the quote from the error log including the stack trace), please comment to re-open the issue.

Generated at Thu Feb 08 07:12:47 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.