Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-15333

MariaDB (still) slow start

    XMLWordPrintable

Details

    Description

      Still having issues with startup with 10.2.13. MDEV-13869 did not fix my issue.

      FYI, my 30 +/- minute start time is for a DB roughly 290GB and roughly 991K tables

      2018-02-16  7:35:02 140326823504064 [Note] InnoDB: Highest supported file format is Barracuda.
      2018-02-16  8:02:01 140326823504064 [Note] InnoDB: 128 out of 128 rollback segments are active.
      

      During the startup, gdb shows this:

      Attaching to program: /usr/sbin/mysqld, process 26586
      [New LWP 26587]
      [New LWP 26588]
      [New LWP 26598]
      [New LWP 26599]
      [New LWP 26600]
      [New LWP 26601]
      [New LWP 26602]
      [New LWP 26603]
      [New LWP 26604]
      [New LWP 26605]
      [New LWP 26606]
      [New LWP 26607]
      [New LWP 26608]
      [New LWP 26609]
      [New LWP 26610]
      [New LWP 26611]
      [Thread debugging using libthread_db enabled]
      Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
      0x00007f1636cebd43 in pread64 () at ../sysdeps/unix/syscall-template.S:84
      84	../sysdeps/unix/syscall-template.S: No such file or directory.
       
      Thread 17 (Thread 0x7f08c2805700 (LWP 26611)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005587d9931f36 in os_event::wait (this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:166
      #2  os_event::wait_low (reset_sig_count=1, this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:336
      #3  os_event_wait_low (event=0x5588274b9ca0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:535
      #4  0x00005587d9a6edab in buf_flush_page_cleaner_worker (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0flu.cc:3507
      #5  0x00007f1636ce26ba in start_thread (arg=0x7f08c2805700) at pthread_create.c:333
      #6  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 16 (Thread 0x7f08c3006700 (LWP 26610)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005587d9931f36 in os_event::wait (this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:166
      #2  os_event::wait_low (reset_sig_count=1, this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:336
      #3  os_event_wait_low (event=0x5588274b9ca0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:535
      #4  0x00005587d9a6edab in buf_flush_page_cleaner_worker (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0flu.cc:3507
      #5  0x00007f1636ce26ba in start_thread (arg=0x7f08c3006700) at pthread_create.c:333
      #6  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 15 (Thread 0x7f08c3807700 (LWP 26609)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005587d9931f36 in os_event::wait (this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:166
      #2  os_event::wait_low (reset_sig_count=1, this=0x5588274b9ca0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:336
      #3  os_event_wait_low (event=0x5588274b9ca0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:535
      #4  0x00005587d9a6edab in buf_flush_page_cleaner_worker (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0flu.cc:3507
      #5  0x00007f1636ce26ba in start_thread (arg=0x7f08c3807700) at pthread_create.c:333
      #6  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 14 (Thread 0x7f08c4008700 (LWP 26608)):
      #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
      #1  0x00005587d9931f36 in os_event::wait (this=0x5587dbfab2e0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:166
      #2  os_event::wait_low (reset_sig_count=1, this=0x5587dbfab2e0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:336
      #3  os_event_wait_low (event=0x5587dbfab2e0, reset_sig_count=reset_sig_count@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0event.cc:535
      #4  0x00005587d9a6f42a in buf_flush_page_cleaner_coordinator () at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0flu.cc:3179
      #5  0x00007f1636ce26ba in start_thread (arg=0x7f08c4008700) at pthread_create.c:333
      #6  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 13 (Thread 0x7f08c4809700 (LWP 26607)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c4808ae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c4808ae0, m1=m1@entry=0x7f08c4808b90, m2=m2@entry=0x7f08c4808b98, request=request@entry=0x7f08c4808ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c4808ba0, m2=0x7f08c4808b98, m1=0x7f08c4808b90, global_segment=9)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=9, m1=m1@entry=0x7f08c4808b90, m2=m2@entry=0x7f08c4808b98, request=request@entry=0x7f08c4808ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=9) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c4809700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 12 (Thread 0x7f08c500a700 (LWP 26606)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c5009ae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c5009ae0, m1=m1@entry=0x7f08c5009b90, m2=m2@entry=0x7f08c5009b98, request=request@entry=0x7f08c5009ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c5009ba0, m2=0x7f08c5009b98, m1=0x7f08c5009b90, global_segment=8)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=8, m1=m1@entry=0x7f08c5009b90, m2=m2@entry=0x7f08c5009b98, request=request@entry=0x7f08c5009ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=8) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c500a700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 11 (Thread 0x7f08c580b700 (LWP 26605)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c580aae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c580aae0, m1=m1@entry=0x7f08c580ab90, m2=m2@entry=0x7f08c580ab98, request=request@entry=0x7f08c580aba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c580aba0, m2=0x7f08c580ab98, m1=0x7f08c580ab90, global_segment=7)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=7, m1=m1@entry=0x7f08c580ab90, m2=m2@entry=0x7f08c580ab98, request=request@entry=0x7f08c580aba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=7) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c580b700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 10 (Thread 0x7f08c600c700 (LWP 26604)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c600bae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c600bae0, m1=m1@entry=0x7f08c600bb90, m2=m2@entry=0x7f08c600bb98, request=request@entry=0x7f08c600bba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c600bba0, m2=0x7f08c600bb98, m1=0x7f08c600bb90, global_segment=6)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=6, m1=m1@entry=0x7f08c600bb90, m2=m2@entry=0x7f08c600bb98, request=request@entry=0x7f08c600bba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=6) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c600c700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 9 (Thread 0x7f08c680d700 (LWP 26603)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c680cae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c680cae0, m1=m1@entry=0x7f08c680cb90, m2=m2@entry=0x7f08c680cb98, request=request@entry=0x7f08c680cba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c680cba0, m2=0x7f08c680cb98, m1=0x7f08c680cb90, global_segment=5)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=5, m1=m1@entry=0x7f08c680cb90, m2=m2@entry=0x7f08c680cb98, request=request@entry=0x7f08c680cba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=5) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c680d700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 8 (Thread 0x7f08c700e700 (LWP 26602)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c700dae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c700dae0, m1=m1@entry=0x7f08c700db90, m2=m2@entry=0x7f08c700db98, request=request@entry=0x7f08c700dba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c700dba0, m2=0x7f08c700db98, m1=0x7f08c700db90, global_segment=4)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=4, m1=m1@entry=0x7f08c700db90, m2=m2@entry=0x7f08c700db98, request=request@entry=0x7f08c700dba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=4) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c700e700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 7 (Thread 0x7f08c780f700 (LWP 26601)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c780eae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c780eae0, m1=m1@entry=0x7f08c780eb90, m2=m2@entry=0x7f08c780eb98, request=request@entry=0x7f08c780eba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c780eba0, m2=0x7f08c780eb98, m1=0x7f08c780eb90, global_segment=3)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=3, m1=m1@entry=0x7f08c780eb90, m2=m2@entry=0x7f08c780eb98, request=request@entry=0x7f08c780eba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=3) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c780f700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 6 (Thread 0x7f08c8010700 (LWP 26600)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c800fae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c800fae0, m1=m1@entry=0x7f08c800fb90, m2=m2@entry=0x7f08c800fb98, request=request@entry=0x7f08c800fba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c800fba0, m2=0x7f08c800fb98, m1=0x7f08c800fb90, global_segment=2)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=2, m1=m1@entry=0x7f08c800fb90, m2=m2@entry=0x7f08c800fb98, request=request@entry=0x7f08c800fba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=2) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c8010700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 5 (Thread 0x7f08c8811700 (LWP 26599)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c8810ae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c8810ae0, m1=m1@entry=0x7f08c8810b90, m2=m2@entry=0x7f08c8810b98, request=request@entry=0x7f08c8810ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c8810ba0, m2=0x7f08c8810b98, m1=0x7f08c8810b90, global_segment=1)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=1, m1=m1@entry=0x7f08c8810b90, m2=m2@entry=0x7f08c8810b98, request=request@entry=0x7f08c8810ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=1) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c8811700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 4 (Thread 0x7f08c9012700 (LWP 26598)):
      #0  0x00007f1637e0664a in ?? () from /lib/x86_64-linux-gnu/libaio.so.1
      #1  0x00005587d992b358 in LinuxAIOHandler::collect (this=this@entry=0x7f08c9011ae0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1893
      #2  0x00005587d992b8a2 in LinuxAIOHandler::poll (this=this@entry=0x7f08c9011ae0, m1=m1@entry=0x7f08c9011b90, m2=m2@entry=0x7f08c9011b98, request=request@entry=0x7f08c9011ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2038
      #3  0x00005587d99307c9 in os_aio_linux_handler (request=0x7f08c9011ba0, m2=0x7f08c9011b98, m1=0x7f08c9011b90, global_segment=0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:2092
      #4  os_aio_handler (segment=segment@entry=0, m1=m1@entry=0x7f08c9011b90, m2=m2@entry=0x7f08c9011b98, request=request@entry=0x7f08c9011ba0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5720
      #5  0x00005587d9ac4685 in fil_aio_wait (segment=segment@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5330
      #6  0x00005587d99c6958 in io_handler_thread (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:343
      #7  0x00007f1636ce26ba in start_thread (arg=0x7f08c9012700) at pthread_create.c:333
      #8  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 3 (Thread 0x7f1625d93700 (LWP 26588)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005587d984e68f in inline_mysql_cond_timedwait (src_file=0x5587d9cf2158 "/home/buildbot/buildbot/build/mariadb-10.2.13/storage/maria/ma_servicethread.c", src_line=116, abstime=0x7f1625d92c80, 
          mutex=0x5587dacb0860 <LOCK_checkpoint>, that=0x5587dacb0820 <COND_checkpoint>) at /home/buildbot/buildbot/build/mariadb-10.2.13/include/mysql/psi/mysql_thread.h:1186
      #2  my_service_thread_sleep (control=control@entry=0x5587da3f46e0 <checkpoint_control>, sleep_time=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/maria/ma_servicethread.c:115
      #3  0x00005587d9846157 in ma_checkpoint_background (arg=0x1e) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/maria/ma_checkpoint.c:709
      #4  0x00007f1636ce26ba in start_thread (arg=0x7f1625d93700) at pthread_create.c:333
      #5  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 2 (Thread 0x7f1634a7e700 (LWP 26587)):
      #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
      #1  0x00005587d9bd5762 in inline_mysql_cond_timedwait (that=0x5587dad0af60 <COND_timer>, mutex=0x5587dad0afa0 <LOCK_timer>, 
          src_file=0x5587d9d3a168 "/home/buildbot/buildbot/build/mariadb-10.2.13/mysys/thr_timer.c", src_line=292, abstime=0x7f1634a7dd10)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/include/mysql/psi/mysql_thread.h:1186
      #2  timer_handler (arg=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/mysys/thr_timer.c:292
      #3  0x00007f1636ce26ba in start_thread (arg=0x7f1634a7e700) at pthread_create.c:333
      #4  0x00007f163638d41d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
       
      Thread 1 (Thread 0x7f16383ac8c0 (LWP 26586)):
      #0  0x00007f1636cebd43 in pread64 () at ../sysdeps/unix/syscall-template.S:84
      #1  0x00005587d992bca5 in pread64 (__offset=0, __nbytes=16384, __buf=0x7f0ad43e8000, __fd=1817) at /usr/include/x86_64-linux-gnu/bits/unistd.h:99
      #2  SyncFileIO::execute (request=..., this=<synthetic pointer>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:1573
      #3  os_file_io (in_type=..., file=file@entry=1817, buf=buf@entry=0x7f0ad43e8000, n=n@entry=16384, offset=offset@entry=0, err=err@entry=0x7ffd2786b44c)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:4877
      #4  0x00005587d992c850 in os_file_pread (err=0x7ffd2786b44c, offset=0, n=16384, buf=0x7f0ad43e8000, file=1817, type=...) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5046
      #5  os_file_read_page (type=..., file=file@entry=1817, buf=buf@entry=0x7f0ad43e8000, offset=offset@entry=0, n=n@entry=16384, o=o@entry=0x0, exit_on_err=true)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5083
      #6  0x00005587d992e9f1 in os_file_read_func (n=16384, offset=0, buf=0x7f0ad43e8000, file=<optimized out>, type=...) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:5493
      #7  os_aio_func (type=..., mode=mode@entry=24, name=name@entry=0x5588327ed280 "./ddx_lab_802532/quickbooks_references.ibd", file=..., buf=buf@entry=0x7f0ad43e8000, offset=0, n=16384, read_only=false, 
          m1=0x5588327ed1d0, m2=0x7f0acd19e5a0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/os/os0file.cc:6683
      #8  0x00005587d9acba01 in pfs_os_aio_func (src_file=0x5587d9d27f60 "/home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc", src_line=5290, m2=0x7f0acd19e5a0, m1=0x5588327ed1d0, 
          read_only=<optimized out>, n=16384, offset=0, buf=0x7f0ad43e8000, file=..., name=0x5588327ed280 "./ddx_lab_802532/quickbooks_references.ibd", mode=24, type=...)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/include/os0file.ic:256
      #9  fil_io (type=..., sync=sync@entry=true, page_id=..., page_size=..., byte_offset=byte_offset@entry=0, len=16384, buf=0x7f0ad43e8000, message=0x7f0acd19e5a0, ignore_missing_space=false)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:5285
      #10 0x00005587d9a7e5fb in buf_read_page_low (ignore_missing_space=false, unzip=false, page_size=..., page_id=..., mode=132, type=0, sync=true, err=0x7ffd2786b9e0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0rea.cc:192
      #11 buf_read_page (page_id=..., page_size=...) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0rea.cc:460
      #12 0x00005587d9a56e08 in buf_page_get_gen (page_id=..., page_size=..., rw_latch=rw_latch@entry=2, guess=<optimized out>, guess@entry=0x0, mode=mode@entry=10, 
          file=file@entry=0x5587d9d27f60 "/home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc", line=4639, mtr=0x7ffd2786c050, err=0x0)
          at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/buf/buf0buf.cc:4335
      #13 0x00005587d9ac97d4 in fsp_flags_try_adjust (space_id=space_id@entry=550663, flags=flags@entry=0) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:4637
      #14 0x00005587d9aca0f4 in fil_ibd_open (validate=<optimized out>, validate@entry=false, fix_dict=<optimized out>, purpose=purpose@entry=FIL_TYPE_TABLESPACE, id=550663, flags=<optimized out>, 
          space_name=<optimized out>, path_in=0x5588327e83f0 "./ddx_lab_802532/quickbooks_references.ibd") at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/fil/fil0fil.cc:4222
      #15 0x00005587d9aa63bb in dict_check_sys_tables (validate=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/dict/dict0load.cc:1480
      #16 dict_check_tablespaces_and_store_max_id (validate=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/dict/dict0load.cc:1535
      #17 0x00005587d99cbdc1 in innobase_start_or_create_for_mysql () at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/srv/srv0start.cc:2544
      #18 0x00005587d98afff1 in innobase_init (p=<optimized out>) at /home/buildbot/buildbot/build/mariadb-10.2.13/storage/innobase/handler/ha_innodb.cc:4367
      #19 0x00005587d966e084 in ha_initialize_handlerton (plugin=0x5587dbe68a80) at /home/buildbot/buildbot/build/mariadb-10.2.13/sql/handler.cc:520
      #20 0x00005587d94e1b05 in plugin_initialize (tmp_root=tmp_root@entry=0x7ffd27877640, plugin=plugin@entry=0x5587dbe68a80, argc=argc@entry=0x5587da481c58 <remaining_argc>, argv=argv@entry=0x5587dbe22e40, 
          options_only=options_only@entry=false) at /home/buildbot/buildbot/build/mariadb-10.2.13/sql/sql_plugin.cc:1416
      #21 0x00005587d94e24da in plugin_init (argc=argc@entry=0x5587da481c58 <remaining_argc>, argv=0x5587dbe22e40, flags=2) at /home/buildbot/buildbot/build/mariadb-10.2.13/sql/sql_plugin.cc:1697
      #22 0x00005587d943c560 in init_server_components () at /home/buildbot/buildbot/build/mariadb-10.2.13/sql/mysqld.cc:5266
      #23 0x00005587d9441437 in mysqld_main (argc=63, argv=0x5587dbe22e40) at /home/buildbot/buildbot/build/mariadb-10.2.13/sql/mysqld.cc:5858
      #24 0x00007f16362a6830 in __libc_start_main (main=0x5587d941fe00 <main(int, char**)>, argc=6, argv=0x7ffd27878238, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
          stack_end=0x7ffd27878228) at ../csu/libc-start.c:291
      #25 0x00005587d9434779 in _start ()
      Detaching from program: /usr/sbin/mysqld, process 26586
      
      

      Of possible note, 'apt upgrade' from 10.2.12 to 10.2.13 reported that it found a mysql 5.6 data in /var/lib/mysql (been 10.2.x for a month)?

      Attachments

        Issue Links

          Activity

            People

              marko Marko Mäkelä
              michaelcaplan Michael Caplan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.