[MXS-1633] Need remove mutex in sqlite Created: 2018-01-29  Updated: 2018-01-30  Resolved: 2018-01-30

Status: Closed
Project: MariaDB MaxScale
Component/s: Core
Affects Version/s: None
Fix Version/s: 2.2.2

Type: Bug Priority: Major
Reporter: dapeng huang Assignee: Johan Wikman
Resolution: Fixed Votes: 0
Labels: None

Sprint: MXS-SPRINT-51

 Description   

In MXS-1529, each thread maintain a sqlite instance, openwith SQLITE_OPEN_NOMUTEX,
But I saw a stack :

#0 0x00007f1c1067df4d in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f1c10679d02 in _L_lock_791 () from /lib64/libpthread.so.0
#2 0x00007f1c10679c08 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f1c0a52d9fc in sqlite3Malloc () from /lib64/libsqlite3.so.0
#4 0x00007f1c0a52db41 in sqlite3DbMallocRaw () from /lib64/libsqlite3.so.0
#5 0x00007f1c0a56d5cf in exprAnalyze () from /lib64/libsqlite3.so.0
#6 0x00007f1c0a56e238 in exprAnalyzeAll () from /lib64/libsqlite3.so.0
#7 0x00007f1c0a56541d in sqlite3WhereBegin () from /lib64/libsqlite3.so.0
#8 0x00007f1c0a568755 in sqlite3Select () from /lib64/libsqlite3.so.0
#9 0x00007f1c0a57ab1a in sqlite3Parser () from /lib64/libsqlite3.so.0
#10 0x00007f1c0a57eb29 in sqlite3RunParser () from /lib64/libsqlite3.so.0
#11 0x00007f1c0a57f12a in sqlite3Prepare () from /lib64/libsqlite3.so.0
#12 0x00007f1c0a57f425 in sqlite3LockAndPrepare () from /lib64/libsqlite3.so.0
#13 0x00007f1c0a57f6c8 in sqlite3_prepare_v2 () from /lib64/libsqlite3.so.0
#14 0x00007f1c0a57f785 in sqlite3_exec () from /lib64/libsqlite3.so.0
#15 0x00007f1c0a7ce19c in validate_mysql_user (instance=instance@entry=0x7174e0, dcb=dcb@entry=0x7f1b7ff4f210, session=session@entry=0x7f1b7ff4f650, scramble=scramble@entry=0x7f1b7ff4f4d0 "@1QYXE+i?ZT-*?>8U\037
H", scramble_len=scramble_len@entry=20) at /home/wuzang.hdp/Code/MaxScale/server/modules/authenticator/MySQLAuth/dbusers.c:208
#16 0x00007f1c0a7cd18d in mysql_auth_authenticate (dcb=0x7f1b7ff4f210) at /home/wuzang.hdp/Code/MaxScale/server/modules/authenticator/MySQLAuth/mysql_auth.c:288
#17 0x00007f1c0a10a91d in gw_read_do_authentication (nbytes_read=<optimized out>, read_buffer=0x7f1b7f

Found in query_classifier/qc_sqlite/sqlite-src-3110100/src/main.c

/* Remove harmful bits from the flags parameter
**

    • The SQLITE_OPEN_NOMUTEX and SQLITE_OPEN_FULLMUTEX flags were
    • dealt with in the previous code block. Besides these, the only
    • valid input flags for sqlite3_open_v2() are SQLITE_OPEN_READONLY,
    • SQLITE_OPEN_READWRITE, SQLITE_OPEN_CREATE, SQLITE_OPEN_SHAREDCACHE,
    • SQLITE_OPEN_PRIVATECACHE, and some reserved bits. Silently mask
    • off all other flags.
      */
      flags &= ~( SQLITE_OPEN_DELETEONCLOSE |
      SQLITE_OPEN_EXCLUSIVE |
      SQLITE_OPEN_MAIN_DB |
      SQLITE_OPEN_TEMP_DB |
      SQLITE_OPEN_TRANSIENT_DB |
      SQLITE_OPEN_MAIN_JOURNAL |
      SQLITE_OPEN_TEMP_JOURNAL |
      SQLITE_OPEN_SUBJOURNAL |
      SQLITE_OPEN_MASTER_JOURNAL |
      SQLITE_OPEN_NOMUTEX |
      SQLITE_OPEN_FULLMUTEX |
      SQLITE_OPEN_WAL
      );

So set flag SQLITE_OPEN_NOMUTEX is useless;

 Comments   
Comment by Johan Wikman [ 2018-01-29 ]

Just for the record, the code in query_classifier/qc_sqlite/sqlite-src-3110100/ is only used by qc_sqlite; the user management of MaxScale uses the system provided sqlite3 library.

I think that particular locking can be removed by setting SQLITE_CONFIG_MEMSTATUS to 0 using sqlite3_config.

Comment by dapeng huang [ 2018-01-30 ]

Yes, you are right, I tired add

sqlite3_config(SQLITE_CONFIG_MEMSTATUS, 0)

in function mysql_auth_init, it works; and the performance improved 100% in our env using our simple connect/disconnect test;

Comment by Johan Wikman [ 2018-01-30 ]
  • The collecting of memory statistics is turned off.
  • It is ensured that Sqlite3 is used in Multi-thread mode. Necessary as there are MaxScale components (e.g. PAM authentication) where multiple threads access the same database.
Generated at Thu Feb 08 04:08:15 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.