[MDEV-4786] merge 10.0-monty → 10.0 Created: 2013-07-15  Updated: 2013-09-03  Resolved: 2013-08-12

Status: Closed
Project: MariaDB Server
Component/s: None
Fix Version/s: 10.0.4

Type: Task Priority: Major
Reporter: Sergei Golubchik Assignee: Sergei Golubchik
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-4744 InnoDB Fulltext indexes Closed
is blocked by MDEV-4785 merge 5.5 → 10.0-base → 10.0 Closed
PartOf
includes MDEV-4809 vcol tests merging for 10.0 merge Closed
includes MDEV-4816 rpl.rpl_trunc_temp fails in 10.0-serg Closed
is part of MDEV-3932 5.6 merge Closed
Relates
relates to MDEV-4847 Merge tests for GET DIAGNOSTICS feature Closed
relates to MDEV-4860 Buildbot upgrade tests for deb packag... Closed
relates to MDEV-4864 Merge tests for EXCHANGE PARTITION fe... Closed
relates to MDEV-4865 Change related to --log option/variab... Closed
relates to MDEV-4866 Test case for bugfix #34750 is not me... Closed

 Description   
  • fix tests
    • funcs_1.is_tables (svoj, make sure serg is aware of this patch, comment added)
    • innodb.help_url (svoj)
    • main.filesort_debug (psergey)
    • main.gis (HF)
    • main.innodb_ext_key (Igor)
    • main.innodb_mysql_sync (svoj)
    • main.system_mysql_db_fix40123 (psergey)
    • main.system_mysql_db_fix50030 (psergey)
    • main.system_mysql_db_fix50117 (psergey)
    • maria.maria (svoj)
    • perfschema.relaylog (svoj)
    • rpl.rpl_current_user (psergey, initial investigation)
    • rpl.rpl_gtid_master_promote (psergey, initial investigation)
    • rpl.rpl_temp_table_mix_row (psergey, initial investigation)
    • rpl.rpl_trunc_temp (psergey, initial investigation), MDEV-4816
    • vcol.vcol_blocked_sql_funcs_innodb (sanja)
    • vcol.vcol_column_def_options_innodb (sanja)
    • vcol.vcol_handler_innodb (sanja)
    • vcol.vcol_ins_upd_innodb (sanja)
    • vcol.vcol_keys_innodb (sanja)
    • vcol.vcol_non_stored_columns_innodb (sanja)
    • vcol.vcol_partition_innodb (sanja)
    • vcol.vcol_select_innodb (sanja)
    • vcol.vcol_supported_sql_funcs_innodb (sanja)
    • vcol.vcol_trigger_sp_innodb (sanja)
    • vcol.vcol_view_innodb (sanja)
    • connect.grant (svoj)
    • sql_discovery.simple (svoj)
    • archive.archive (svoj)
    • main.mysqldhelp (svoj)
    • innodb.innodb_mysql (big-test, svoj)
    • parts.partition_alter1_1_innodb (big-test, svoj)
    • parts.partition_alter1_1_2_innodb (big-test, svoj)
    • parts.partition_alter1_2_innodb (big-test, svoj)
    • parts.partition_alter2_1_1_innodb (big-test, svoj)
    • parts.partition_alter2_1_2_innodb (big-test, svoj)
    • parts.partition_alter2_2_1_innodb (big-test, svoj)
    • parts.partition_alter2_2_2_innodb (big-test, svoj)
    • parts.partition_alter4_innodb (big-test, svoj)
    • sys_vars.log_error_func (embedded, svoj)
    • sys_vars.log_error_func2 (embedded, svoj)
    • maria.small_blocksize (embedded, svoj)
    • sys_vars.autocommit_func3 (embedded, svoj)
    • sys_vars.autocommit_func2 (embedded, svoj)
    • sys_vars.autocommit_func4 (embedded, svoj)
    • sys_vars.autocommit_func5 (embedded, svoj)
    • sys_vars.tx_isolation_func (embedded, svoj)
    • main.pool_of_threads (embedded, svoj)
    • innodb.innodb (embedded, svoj)
    • archive.archive-big (embedded, svoj)
    • parts.part_supported_sql_func_innodb (embedded, svoj)
    • parts.partition_decimal_myisam (embedded, svoj)
    • parts.partition_decimal_innodb (embedded, svoj)
    • main.sum_distinct-big (embedded, svoj)
    • percona.innodb_sys_index (embedded, svoj)
    • percona.percona_flush_contiguous_neighbors (embedded, svoj)
    • fix "check of testcases" failure (svoj)
    • fix archive.archive non-debug failure (svoj)
    • main.partition_open_files_limit (embedded, svoj)
    • innodb.innodb_bug12400341 (embedded, svoj)
    • main.partition_cache (embedded, svoj)
    • main.partition_cache_innodb (embedded, svoj)
    • main.partition_cache_myisam (embedded, svoj)
    • main.query_cache (embedded, svoj)
    • funcs_1.is_statistics_mysql_embedded (embedded, svoj)
    • funcs_1.is_table_constraints_mysql_embedded (embedded, svoj)
    • funcs_1.is_tables_mysql_embedded (embedded, svoj)
    • funcs_1.is_columns_mysql_embedded (embedded, svoj)
    • main-test_sql_discovery.plugin (release, svoj)
    • main.plugin (release, svoj)
    • parts.partition_mgm_lc2_innodb (lc fs, svoj)
    • parts.partition_mgm_lc2_archive (lc fs, svoj)
    • parts.partition_mgm_lc2_memory (lc fs, svoj)
    • parts.partition_mgm_lc2_myisam (lc fs, svoj)
    • vcol.vcol_misc (valgrind, bar)
    • funcs_1.is_cml_memory (valgrind, bar)
    • main.ctype_ucs2_def (valgrind, bar)
    • innodb.innodb-ucs2 (valgrind, bar)
    • main.mix2_myisam_ucs2 (valgrind, bar)
    • main.ctype_uca (valgrind, bar)
    • main.ctype_ucs (valgrind, bar)
    • binlog.binlog_stm_ctype_ucs (valgrind, bar)
    • binlog.binlog_mysqlbinlog_row_myisam (valgrind, bar)
    • binlog.binlog_mysqlbinlog_row (valgrind, bar)
    • binlog.binlog_row_ctype_ucs (valgrind, bar)
    • funcs_1.storedproc (valgrind, Holyfoot)
    • main.gis (valgrind, Holyfoot)
  • ps_ddl.test (svoj, comment added)
    • test CHECK_METADATA
    • doesn't seem to do anything, perhaps the test is wrong?
  • innodb_mysql_sync.test (svoj, comment added)
    • test TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE
    • now it's the same as TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE
  • get_table_def_key()
    • start using get_table_def_key() where 5.6 does
    • by creating get_table_share(TABLE_LIST) and tdc_open_view(TABLE_LIST)
  • last argument of wait_while_table_is_used() ? (comment added, svoj)
  • simple_rename_or_index_change to use alter_table_manage_keys?
  • remove half-applied "Bug#14521864: MYSQL 5.1 TO 5.5 BUGS PARTITIONING"
  • remove HA_CAN_EXPORT
    • let extra() fail if cannot be exported
    • most engines will not fail, because they support export
  • what is HA_READ_OUT_OF_SYNC ? (comment added, svoj)
  • remove HA_BLOCK_CONST_TABLE
    • and other remnants of pushed joins (if any)
  • move all rsa/auth stuff into a plugin, grrrr!
  • check create_table_mode for correctness
  • fix engines (svoj)
    • connect (svoj)
    • sequence (svoj)
    • test_sql_discovery (svoj)
    • spider (svoj)
  • fix debian/ubuntu packages build failure (svoj)
  • fix debian/ubuntu build failure (svoj)


 Comments   
Comment by Sergei Petrunia [ 2013-07-24 ]

About filesort_debug.test failure.
We get error 1041 instead of 1038. In symbols, that is:

#define ER_OUT_OF_SORTMEMORY 1038
#define ER_OUT_OF_RESOURCES 1041

The difference comes from the following. mysql-5.6 has these lines in my_malloc():

DBUG_EXECUTE_IF("simulate_out_of_memory",
DBUG_SET("-d,simulate_out_of_memory");

added by alexey.kopytov@sun.com-20100521112348-rz0zm53wbo6g2aqr,
Bug #42064: low memory crash when importing hex strings, in
Item_hex_string::Item_hex_string

10.0-serg doesn't have them. This causes "simulate_out_of_memory" condition to remain switched on, which causes malloc failure when we're trying to allocate a memory for the error message. Which, in turn, produces a ER_OUT_OF_RESOURCES error here:

#0 sql_alloc_error_handler () at /home/psergey/dev2/10.0-serg/sql/thr_malloc.cc:49
#1 0x0000000000e6d30e in alloc_root (mem_root=0x2ed2648, length=392) at /home/psergey/dev2/10.0-serg/mysys/my_alloc.c:203
#2 0x00000000005887b1 in Sql_alloc::operator new (size=392, mem_root=0x2ed2648) at /home/psergey/dev2/10.0-serg/sql/sql_list.h:43
#3 0x000000000060ba70 in Warning_info::push_warning (this=0x2ed2648, thd=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_error.cc:692
#4 0x000000000060032f in Diagnostics_area::push_warning (this=0x2ed2420, thd=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_error.h:807
#5 0x00000000005f3ea2 in THD::raise_condition (this=0x2ecd610, sql_errno=1038, sqlstate=0xf083db "HY001", level=Sql_condition::WARN_LEVEL_ERROR, msg=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size") at /home/psergey/dev2/10.0-serg/sql/sql_class.cc:1213
#6 0x000000000057f29d in my_message_sql (error=1038, str=0x7fffe4160bc0 "Out of sort memory, consider increasing server sort buffer size", MyFlags=4166) at /home/psergey/dev2/10.0-serg/sql/mysqld.cc:3371
#7 0x0000000000e70bf2 in my_error (nr=1038, MyFlags=4166) at /home/psergey/dev2/10.0-serg/mysys/my_error.c:125
#8 0x00000000007fa578 in filesort (thd=0x2ecd610, table=0x7fffa804cb30, sortorder=0x7fffa8016760, s_length=2, select=0x7fffa8016568, max_rows=18446744073709551615, sort_positions=false, examined_rows=0x7fffe41614a0, found_rows=0x7fffe4161498) at /home/psergey/dev2/10.0-serg/sql/filesort.cc:278
.

Comment by Sergei Petrunia [ 2013-07-24 ]

I've tried to figure out which changeset exchanged the order of push_warning() and set_error_status() calls in THD::raise_condition(). It was this merge changeset (and the change was introduced by the merge cset itself, not by one of the merged-in csets) :

revno: 3774 [merge]
revision-id: sergii@pisem.net-20130721143919-7cltcw2l9g29f983
parent: sergii@pisem.net-20130718144657-oi7ql96c29knkqqi
parent: sergii@pisem.net-20130717165112-i9klgxk4enpvc09a
committer: Sergei Golubchik <sergii@pisem.net>
branch nick: 10.0
timestamp: Sun 2013-07-21 16:39:19 +0200
message:
10.0-monty merge
...

Comment by Sergey Vojtovich [ 2013-07-24 ]

Hi Sergei,

yes, that's what I was afraid of. I guess it is just merge mistake.

Regards,
Sergey

Comment by Sergey Vojtovich [ 2013-07-25 ]

TDC_RT_REMOVE_NOT_OWN_KEEP_SHARE is not same as TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE. Comment in sql_table.cc says:

Storage engine has requested exclusive lock only for prepare phase
and we are not under LOCK TABLES.
Don't mark TABLE_SHARE as old in this case, as this won't allow opening
of table by other threads during main phase of in-place ALTER TABLE.

At this moment we hold exclusive metadata lock, all we should do is purge unused TABLE objects.

Comment by Sergey Vojtovich [ 2013-08-02 ]

HA_READ_OUT_OF_SYNC was introduced with the undermentioned revision. Yet the
sole purpose of this flag is to disable RBR updates batching.

Originally it was named HA_STORES_NO_ROWS, which I find applicable to blackhole only.
I suggested two alternatives: HA_EMPTY or HA_READ_OUT_OF_SYNC.
Meaning of HA_EMPTY is the same as HA_STORES_NO_ROWS.

But HA_READ_OUT_OF_SYNC is a bit more generic: the idea is that storage engine doesn't
guarantee that table updates will be [immediately] visible.

E.g. abstract storage engine may consume N rows [on slave], but return single aggregated result row.
HA_READ_OUT_OF_SYNC is applicable to this case as well, whereas HA_STORES_NO_ROWS is
controversial.

revno: 3763 [merge]
committer: Rohit Kalhans <rohit.kalhans@oracle.com>
branch nick: mysql-trunk
timestamp: Wed 2012-05-02 17:34:42 +0530
message:
WL#5597: Using batch operations when there is no index in RBR

CONTEXT
-------

With RBR, if a table has no indexes, the slave does a full table scan for each
row changed (i.e. updated or deleted). This can be extremely time consuming
when there is a high number of rows to be updated.

SOLUTION
---------

When there is a table without a PK or an INDEX, create a temporary in-memory
index (e.g. in-memory hash table) and store the rows to be update in it. Then for
each row in the table, check if the row exists in the hash table. If there is a
match, do the operation, i.e. update or delete.

Comment by Sergey Vojtovich [ 2013-08-05 ]

In 10.0 TDC_RT_REMOVE_NOT_OWN is almost unused in favor of TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE.

But technically I failed to understand it's purpose: wait_while_table_is_used() upgrades shared meta-data lock to exclusive and no other thread is permitted to access this table. Why do we need extra protection mechanism in table definition cache?

If TDC_RT_REMOVE_NOT_OWN_AND_MARK_NOT_USABLE serves a purpose indeed, we need to restore it.

Comment by Sergey Vojtovich [ 2013-08-05 ]

CREATE TABLE t1(a INT, b VARCHAR(255));
ALTER TABLE t1 DROP COLUMN b;

Is the above alter good for in-place algorithm?
If it is, what row format shall t1 have (static or dynamic)?

If it is not allowed or (allowed and format is static): remove juggling with HA_OPTION_PACK_RECORD in mysql_create_frm_image().
else: create .frm after we know for sure which algorithm we will be using, disable juggling with copy algorithm.

Comment by Sergey Vojtovich [ 2013-08-07 ]

ps_ddl.test doesn't contain a test case for revision alexander.nozdrin@oracle.com-20111219114211-49pqi0wfs9p4o9yi
Bug#11748352 - 36002: PREPARED STATEMENTS: IF A VIEW USED IN A STATEMENT IS REPLACED, BAD DATA.

Verified that the problem is applicable to our code, test case from the above revision fails, original test case from BUG#36002 fails.
Also verified that with fully applied changeset test cases succeed.

Comment by Sergey Vojtovich [ 2013-08-07 ]

Merge pushed, remaining items will be moved to another post-10.0.4 task.

Generated at Thu Feb 08 06:59:09 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.