Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-22214

Windows: crash when mariadbd.exe tries to load plugin which is linked to mysqld.exe

Details

    Description

      Example : https://ci.appveyor.com/project/rasmushoj/server/builds/32088322

      There is a weird mix of mariadbd.exe and mysqld.exe in the crashes, in scenarious where plugins are used.

      Take a look how Create_tmp_table::add_schema_fields() from mariadbd.exe calls Type_handler_longlong::make_schema_field() from mysqld.exe below

      [00:23:38] Attempting backtrace. You can use the following information to find out
      		 
      [00:23:38] where mysqld died. If you see no messages after this, something went
      		 
      [00:23:38] terribly wrong...
      		 
      [00:23:38] 
      [00:23:38] mysqld.exe!my_malloc()[my_malloc.c:88]
      		 
      [00:23:38] mysqld.exe!alloc_root()[my_alloc.c:243]
      		 
      [00:23:38] mysqld.exe!Type_handler_longlong::make_schema_field()[sql_type.cc:3951]
      		 
      [00:23:38] mariadbd.exe!Create_tmp_table::add_schema_fields()[sql_select.cc:19111]
      		 
      [00:23:38] mariadbd.exe!create_tmp_table_for_schema()[sql_select.cc:19182]
      		 
      [00:23:38] mariadbd.exe!create_schema_table()[sql_show.cc:8097]
      		 
      [00:23:38] mariadbd.exe!mysql_schema_table()[sql_show.cc:8308]
      		 
      [00:23:38] mariadbd.exe!open_and_process_table()[sql_base.cc:3622]
      		 
      [00:23:38] mariadbd.exe!open_tables()[sql_base.cc:4215]
      		 
      [00:23:38] mariadbd.exe!open_and_lock_tables()[sql_base.cc:5110]
      		 
      [00:23:38] mariadbd.exe!execute_sqlcom_select()[sql_parse.cc:6089]
      		 
      [00:23:38] mariadbd.exe!mysql_execute_command()[sql_parse.cc:3901]
      		 
      [00:23:38] mariadbd.exe!mysql_parse()[sql_parse.cc:7958]
      		 
      [00:23:38] mariadbd.exe!dispatch_command()[sql_parse.cc:1842]
      		 
      [00:23:38] mariadbd.exe!do_command()[sql_parse.cc:1358]
      		 
      [00:23:38] mariadbd.exe!threadpool_process_request()[threadpool_common.cc:356]
      		 
      [00:23:38] mariadbd.exe!tp_callback()[threadpool_common.cc:195]
      		 
      [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
      		 
      [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
      		 
      [00:23:38] KERNEL32.DLL!BaseThreadInitThunk()
      		 
      [00:23:38] ntdll.dll!RtlUserThreadStart()
      		 
      [00:23:38] 
      [00:23:38] Trying to get some variables.
      		 
      [00:23:38] Some pointers may be invalid and cause the dump to abort.
      		 
      [00:23:38] Query (0x174659aa420): SELECT lock_mode FROM information_schema.metadata_lock_info WHERE lock_type='Backup lock'
      		 
      [00:23:38] Connection ID (thread ID): 5
      		 
      [00:23:38] Status: NOT_KILLED

      Interestingly, this does not happen on buildbot, which has a slightly more recent Windows
      and VS version (not sure whether VS is involved)

      What happens is that one executable (mariadbd.exe) , e.g loads a shared library (plugin), which is "linked" to another executable(mysqld.exe), and on some reason some functions will then be taken from another exe. This can't work well, but on some reason it does on buildbot. Perhaps more recent Windows OS loader figures out that mysqld.exe is a hardlink to mariadbd.exe.

      Attachments

        Issue Links

          is caused by

          Task - A task that needs to be done. MDEV-21303 Make executables MariaDB named

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. MDEV-22010 use executables MariaDB named in scripts

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            wlad Vladislav Vaintroub created issue -
            wlad Vladislav Vaintroub made changes -
            Field Original Value New Value
            wlad Vladislav Vaintroub made changes -
            Component/s Platform Windows [ 10138 ]
            Component/s Platform Windows [ 14964 ]
            Fix Version/s 10.5 [ 23123 ]
            Fix Version/s 10.5 [ 23608 ]
            Key MENT-709 MDEV-22214
            Affects Version/s 10.5 [ 23123 ]
            Affects Version/s 10.5 [ 23608 ]
            Project MariaDB Enterprise [ 11500 ] MariaDB Server [ 10000 ]
            wlad Vladislav Vaintroub added a comment -

            delayloading server exe, with custom loader, that does not try to load a new ".exe", but uses a new one could be a workaround
            But it is not, because the bad plugins use server variables, together with functions. VS custom delayloader does not work with data, only with functions.

            However using a large DLL, with the whole server functionality, to which both the "stub" server exe , and all bad plugins link, would eliminate the issue. We should also get rid of various GetProcAddress(GetModuleHandle(NULL), "name") random hacks for determining variable addresses, and replace it with regular MYSQL_PLUGIN_IMPORT. the hacks are random, because they are selectively used by some plugins, and not by the others. It will also simplify the coding, and get rid of possibility of accessing null pointers, on Windows.

            wlad Vladislav Vaintroub added a comment - delayloading server exe, with custom loader, that does not try to load a new ".exe", but uses a new one could be a workaround But it is not, because the bad plugins use server variables, together with functions. VS custom delayloader does not work with data, only with functions. However using a large DLL, with the whole server functionality, to which both the "stub" server exe , and all bad plugins link, would eliminate the issue. We should also get rid of various GetProcAddress(GetModuleHandle(NULL), "name") random hacks for determining variable addresses, and replace it with regular MYSQL_PLUGIN_IMPORT. the hacks are random, because they are selectively used by some plugins, and not by the others. It will also simplify the coding, and get rid of possibility of accessing null pointers, on Windows.
            wlad Vladislav Vaintroub made changes -
            Description Example : https://ci.appveyor.com/project/rasmushoj/server/builds/32088322

            There is a weird mix of mariadbd.exe and mysqld.exe in the crashes, all invlving plugins

            Example
            {noformat}
            [00:23:38] Attempting backtrace. You can use the following information to find out
            [00:23:38] where mysqld died. If you see no messages after this, something went
            [00:23:38] terribly wrong...
            [00:23:38]
            [00:23:38] mysqld.exe!my_malloc()[my_malloc.c:88]
            [00:23:38] mysqld.exe!alloc_root()[my_alloc.c:243]
            [00:23:38] mysqld.exe!Type_handler_longlong::make_schema_field()[sql_type.cc:3951]
            [00:23:38] mariadbd.exe!Create_tmp_table::add_schema_fields()[sql_select.cc:19111]
            [00:23:38] mariadbd.exe!create_tmp_table_for_schema()[sql_select.cc:19182]
            [00:23:38] mariadbd.exe!create_schema_table()[sql_show.cc:8097]
            [00:23:38] mariadbd.exe!mysql_schema_table()[sql_show.cc:8308]
            [00:23:38] mariadbd.exe!open_and_process_table()[sql_base.cc:3622]
            [00:23:38] mariadbd.exe!open_tables()[sql_base.cc:4215]
            [00:23:38] mariadbd.exe!open_and_lock_tables()[sql_base.cc:5110]
            [00:23:38] mariadbd.exe!execute_sqlcom_select()[sql_parse.cc:6089]
            [00:23:38] mariadbd.exe!mysql_execute_command()[sql_parse.cc:3901]
            [00:23:38] mariadbd.exe!mysql_parse()[sql_parse.cc:7958]
            [00:23:38] mariadbd.exe!dispatch_command()[sql_parse.cc:1842]
            [00:23:38] mariadbd.exe!do_command()[sql_parse.cc:1358]
            [00:23:38] mariadbd.exe!threadpool_process_request()[threadpool_common.cc:356]
            [00:23:38] mariadbd.exe!tp_callback()[threadpool_common.cc:195]
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] KERNEL32.DLL!BaseThreadInitThunk()
            [00:23:38] ntdll.dll!RtlUserThreadStart()
            [00:23:38]
            [00:23:38] Trying to get some variables.
            [00:23:38] Some pointers may be invalid and cause the dump to abort.
            [00:23:38] Query (0x174659aa420): SELECT lock_mode FROM information_schema.metadata_lock_info WHERE lock_type='Backup lock'
            [00:23:38] Connection ID (thread ID): 5
            [00:23:38] Status: NOT_KILLED
            {noformat}

            Interestingly, this does not happen on buildbot, which has a slightly more recent Windows
            and VS version (not sure whether VS is involved)

            What happens is that one executable (mariadbd.exe) , e.g loads a shared library (plugin), which is "linked" to another executable(mysqld.exe), and on some reason some functions will then be taken from another exe. This can't work well, but on some reason it does on buildbot. Perhaps more recent Windows OS loader figures out that mysqld.exe is a hardlink to mariadbd.exe.

            		Example : https://ci.appveyor.com/project/rasmushoj/server/builds/32088322

            There is a weird mix of mariadbd.exe and mysqld.exe in the crashes, all invlving plugins

            Take a look how Create_tmp_table::add_schema_fields() from mariadbd.exe calls Type_handler_longlong::make_schema_field() from mysqld.exe below

            {noformat}
            [00:23:38] Attempting backtrace. You can use the following information to find out
            [00:23:38] where mysqld died. If you see no messages after this, something went
            [00:23:38] terribly wrong...
            [00:23:38]
            [00:23:38] mysqld.exe!my_malloc()[my_malloc.c:88]
            [00:23:38] mysqld.exe!alloc_root()[my_alloc.c:243]
            [00:23:38] mysqld.exe!Type_handler_longlong::make_schema_field()[sql_type.cc:3951]
            [00:23:38] mariadbd.exe!Create_tmp_table::add_schema_fields()[sql_select.cc:19111]
            [00:23:38] mariadbd.exe!create_tmp_table_for_schema()[sql_select.cc:19182]
            [00:23:38] mariadbd.exe!create_schema_table()[sql_show.cc:8097]
            [00:23:38] mariadbd.exe!mysql_schema_table()[sql_show.cc:8308]
            [00:23:38] mariadbd.exe!open_and_process_table()[sql_base.cc:3622]
            [00:23:38] mariadbd.exe!open_tables()[sql_base.cc:4215]
            [00:23:38] mariadbd.exe!open_and_lock_tables()[sql_base.cc:5110]
            [00:23:38] mariadbd.exe!execute_sqlcom_select()[sql_parse.cc:6089]
            [00:23:38] mariadbd.exe!mysql_execute_command()[sql_parse.cc:3901]
            [00:23:38] mariadbd.exe!mysql_parse()[sql_parse.cc:7958]
            [00:23:38] mariadbd.exe!dispatch_command()[sql_parse.cc:1842]
            [00:23:38] mariadbd.exe!do_command()[sql_parse.cc:1358]
            [00:23:38] mariadbd.exe!threadpool_process_request()[threadpool_common.cc:356]
            [00:23:38] mariadbd.exe!tp_callback()[threadpool_common.cc:195]
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] KERNEL32.DLL!BaseThreadInitThunk()
            [00:23:38] ntdll.dll!RtlUserThreadStart()
            [00:23:38]
            [00:23:38] Trying to get some variables.
            [00:23:38] Some pointers may be invalid and cause the dump to abort.
            [00:23:38] Query (0x174659aa420): SELECT lock_mode FROM information_schema.metadata_lock_info WHERE lock_type='Backup lock'
            [00:23:38] Connection ID (thread ID): 5
            [00:23:38] Status: NOT_KILLED
            {noformat}

            Interestingly, this does not happen on buildbot, which has a slightly more recent Windows
            and VS version (not sure whether VS is involved)

            What happens is that one executable (mariadbd.exe) , e.g loads a shared library (plugin), which is "linked" to another executable(mysqld.exe), and on some reason some functions will then be taken from another exe. This can't work well, but on some reason it does on buildbot. Perhaps more recent Windows OS loader figures out that mysqld.exe is a hardlink to mariadbd.exe.

            wlad Vladislav Vaintroub made changes -
            serg Sergei Golubchik added a comment -

            This is pushed already. Should the issue be closed or did the push happen by mistake?

            serg Sergei Golubchik added a comment - This is pushed already. Should the issue be closed or did the push happen by mistake?
            wlad Vladislav Vaintroub made changes -
            issue.field.resolutiondate 2020-04-12 11:22:22.0 2020-04-12 11:22:22.3
            wlad Vladislav Vaintroub made changes -
            Fix Version/s 10.5.3 [ 24263 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]
            wlad Vladislav Vaintroub added a comment -

            Nope, not a mistake.

            wlad Vladislav Vaintroub added a comment - Nope, not a mistake.
            wlad Vladislav Vaintroub made changes -
            Description Example : https://ci.appveyor.com/project/rasmushoj/server/builds/32088322

            There is a weird mix of mariadbd.exe and mysqld.exe in the crashes, all invlving plugins

            Take a look how Create_tmp_table::add_schema_fields() from mariadbd.exe calls Type_handler_longlong::make_schema_field() from mysqld.exe below

            {noformat}
            [00:23:38] Attempting backtrace. You can use the following information to find out
            [00:23:38] where mysqld died. If you see no messages after this, something went
            [00:23:38] terribly wrong...
            [00:23:38]
            [00:23:38] mysqld.exe!my_malloc()[my_malloc.c:88]
            [00:23:38] mysqld.exe!alloc_root()[my_alloc.c:243]
            [00:23:38] mysqld.exe!Type_handler_longlong::make_schema_field()[sql_type.cc:3951]
            [00:23:38] mariadbd.exe!Create_tmp_table::add_schema_fields()[sql_select.cc:19111]
            [00:23:38] mariadbd.exe!create_tmp_table_for_schema()[sql_select.cc:19182]
            [00:23:38] mariadbd.exe!create_schema_table()[sql_show.cc:8097]
            [00:23:38] mariadbd.exe!mysql_schema_table()[sql_show.cc:8308]
            [00:23:38] mariadbd.exe!open_and_process_table()[sql_base.cc:3622]
            [00:23:38] mariadbd.exe!open_tables()[sql_base.cc:4215]
            [00:23:38] mariadbd.exe!open_and_lock_tables()[sql_base.cc:5110]
            [00:23:38] mariadbd.exe!execute_sqlcom_select()[sql_parse.cc:6089]
            [00:23:38] mariadbd.exe!mysql_execute_command()[sql_parse.cc:3901]
            [00:23:38] mariadbd.exe!mysql_parse()[sql_parse.cc:7958]
            [00:23:38] mariadbd.exe!dispatch_command()[sql_parse.cc:1842]
            [00:23:38] mariadbd.exe!do_command()[sql_parse.cc:1358]
            [00:23:38] mariadbd.exe!threadpool_process_request()[threadpool_common.cc:356]
            [00:23:38] mariadbd.exe!tp_callback()[threadpool_common.cc:195]
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] KERNEL32.DLL!BaseThreadInitThunk()
            [00:23:38] ntdll.dll!RtlUserThreadStart()
            [00:23:38]
            [00:23:38] Trying to get some variables.
            [00:23:38] Some pointers may be invalid and cause the dump to abort.
            [00:23:38] Query (0x174659aa420): SELECT lock_mode FROM information_schema.metadata_lock_info WHERE lock_type='Backup lock'
            [00:23:38] Connection ID (thread ID): 5
            [00:23:38] Status: NOT_KILLED
            {noformat}

            Interestingly, this does not happen on buildbot, which has a slightly more recent Windows
            and VS version (not sure whether VS is involved)

            What happens is that one executable (mariadbd.exe) , e.g loads a shared library (plugin), which is "linked" to another executable(mysqld.exe), and on some reason some functions will then be taken from another exe. This can't work well, but on some reason it does on buildbot. Perhaps more recent Windows OS loader figures out that mysqld.exe is a hardlink to mariadbd.exe.

            		Example : https://ci.appveyor.com/project/rasmushoj/server/builds/32088322

            There is a weird mix of mariadbd.exe and mysqld.exe in the crashes, in scenarious where plugins are used.

            Take a look how Create_tmp_table::add_schema_fields() from mariadbd.exe calls Type_handler_longlong::make_schema_field() from mysqld.exe below

            {noformat}
            [00:23:38] Attempting backtrace. You can use the following information to find out
            [00:23:38] where mysqld died. If you see no messages after this, something went
            [00:23:38] terribly wrong...
            [00:23:38]
            [00:23:38] mysqld.exe!my_malloc()[my_malloc.c:88]
            [00:23:38] mysqld.exe!alloc_root()[my_alloc.c:243]
            [00:23:38] mysqld.exe!Type_handler_longlong::make_schema_field()[sql_type.cc:3951]
            [00:23:38] mariadbd.exe!Create_tmp_table::add_schema_fields()[sql_select.cc:19111]
            [00:23:38] mariadbd.exe!create_tmp_table_for_schema()[sql_select.cc:19182]
            [00:23:38] mariadbd.exe!create_schema_table()[sql_show.cc:8097]
            [00:23:38] mariadbd.exe!mysql_schema_table()[sql_show.cc:8308]
            [00:23:38] mariadbd.exe!open_and_process_table()[sql_base.cc:3622]
            [00:23:38] mariadbd.exe!open_tables()[sql_base.cc:4215]
            [00:23:38] mariadbd.exe!open_and_lock_tables()[sql_base.cc:5110]
            [00:23:38] mariadbd.exe!execute_sqlcom_select()[sql_parse.cc:6089]
            [00:23:38] mariadbd.exe!mysql_execute_command()[sql_parse.cc:3901]
            [00:23:38] mariadbd.exe!mysql_parse()[sql_parse.cc:7958]
            [00:23:38] mariadbd.exe!dispatch_command()[sql_parse.cc:1842]
            [00:23:38] mariadbd.exe!do_command()[sql_parse.cc:1358]
            [00:23:38] mariadbd.exe!threadpool_process_request()[threadpool_common.cc:356]
            [00:23:38] mariadbd.exe!tp_callback()[threadpool_common.cc:195]
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] ntdll.dll!RtlReleaseSRWLockExclusive()
            [00:23:38] KERNEL32.DLL!BaseThreadInitThunk()
            [00:23:38] ntdll.dll!RtlUserThreadStart()
            [00:23:38]
            [00:23:38] Trying to get some variables.
            [00:23:38] Some pointers may be invalid and cause the dump to abort.
            [00:23:38] Query (0x174659aa420): SELECT lock_mode FROM information_schema.metadata_lock_info WHERE lock_type='Backup lock'
            [00:23:38] Connection ID (thread ID): 5
            [00:23:38] Status: NOT_KILLED
            {noformat}

            Interestingly, this does not happen on buildbot, which has a slightly more recent Windows
            and VS version (not sure whether VS is involved)

            What happens is that one executable (mariadbd.exe) , e.g loads a shared library (plugin), which is "linked" to another executable(mysqld.exe), and on some reason some functions will then be taken from another exe. This can't work well, but on some reason it does on buildbot. Perhaps more recent Windows OS loader figures out that mysqld.exe is a hardlink to mariadbd.exe.

            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 107001 ] MariaDB v4 [ 157591 ]

            People

              wlad Vladislav Vaintroub
              wlad Vladislav Vaintroub
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.