Details

    Description

      I don't use the critical priority frivolously, but I have a query that always works on a mysql server, but only once at the first execution every 20 to 60 minutes on a MariaDB server. I am able to reproduce, that the query gives a different (wrong) result, if executed a few seconds after it gave the correct result. It also runs about 10 times slower on MariaDB than on mysql 5.7.
      I am ready to give you more information than just the query, but the data is sensitive. I have a pseudonymized database dump (101 MB compressed) and a screen recording (87 MB) that I could send to the developers who want to look into the problem. The screen recording shows me executing the query twice with different results and a proof via query log and diff tool, that the queries are the same.
      I filed this issue under "Data Definition - Temporary" because I did not find a component for issues with query results and had a vague feeling, that the temporary table has to do with it. However, I tried renaming it every time, but to no avail! Could it be the query cache?

      This bug affects our production environment.

      I appended the query results with columns separated by tabs, since I could not display them cleanly with this editor.

      Here is the query:

      DROP TEMPORARY TABLE IF EXISTS top5ads;
       
      CREATE TEMPORARY TABLE top5ads (click_count INT(10) UNSIGNED, ad_id INT(10) UNSIGNED);
       
      INSERT INTO top5ads (click_count, ad_id)
      SELECT
      	COUNT(*) AS click_count,
      	ads.id AS ad_id
      FROM
      	employ__ad_clicks ad_clicks
      	JOIN employ__ads ads ON ads.id = ad_clicks.ad_id
      	JOIN employ__companies cmps ON cmps.id = ads.company_id
      WHERE
      	cmps.vdma_company_id > 0 AND
      	ads.job_type_id IN (1,2,3,4,5,13,19) AND
      	ad_clicks.host <> '' AND
      	-- {job_type_id} AND
      	click_date BETWEEN '2018-01-01' AND '2018-07-01'
      GROUP BY
      	ads.id
      ORDER BY
      	click_count DESC
      LIMIT 5;
       
      SELECT
      	click_count,
      	CONCAT
      	(
      		TRIM(ads.title), ' | ',
      		(
      			SELECT
      				CONCAT(pcs.postal_code, ' ', ad_pcs.city)
      			FROM
      				employ__ads_postal_codes ad_pcs
      				JOIN employ__postal_codes pcs ON pcs.id = ad_pcs.postal_code_id
      			WHERE
      				ad_pcs.ad_id = ads.id
      			LIMIT 1
      		), ' | ',
      		click_count,
      		'\n',
      		cmps.name, ' ', cmps.second_name, ' | ', DATE_FORMAT(ads.created_at, '%d.%m.%Y'), ' | ',
      		CASE WHEN
      			ads.published AND
      			CURRENT_DATE BETWEEN ads.publishes_at AND ads.ends_at
      		THEN
      			'online'
      		ELSE
      			'offline'
      		END
      	) AS ad_title,
      	ads.job_type_id,
      	ads.permalink,
      	mpgs_clicks_lists.mpgs_clicks_list,
      	'Top' AS x_label
      FROM
      	top5ads
      JOIN employ__ads ads ON ads.id = top5ads.ad_id
      JOIN employ__companies cmps ON cmps.id = ads.company_id
      JOIN
      	(
      		SELECT
      			adcs_ad_id,
      			GROUP_CONCAT(mpg_name_click_count ORDER BY mpg_name_click_count SEPARATOR '\n') AS mpgs_clicks_list
      		FROM
      		(
      			SELECT
      				adcs_ad_id,
      				CONCAT(click_counts_mpgs.name, ': ', click_counts_mpgs.click_count_mpg) AS mpg_name_click_count
      			FROM
      			(
      				SELECT
      					adcs.ad_id AS adcs_ad_id,
      					COUNT(*) as click_count_mpg,
      					mpgs.name
      				FROM
      					employ__ad_clicks adcs
      					JOIN employ__marketplace_group_hosts mpg_hosts ON adcs.host = mpg_hosts.name
      					JOIN employ__marketplace_groups mpgs ON mpgs.id = mpg_hosts.marketplace_group_id
      				WHERE
      					adcs.host <> '' AND
      					adcs.click_date BETWEEN '2018-01-01' AND '2018-07-01'
      				GROUP BY
      					mpgs.name,
      					adcs.ad_id
      			) click_counts_mpgs
      		) mpg_name_click_counts
      		GROUP BY
      			adcs_ad_id
      		ORDER BY mpg_name_click_counts.adcs_ad_id
      	)	
      	mpgs_clicks_lists ON mpgs_clicks_lists.adcs_ad_id = ads.id;
      
      

      Attachments

        1. mariadb.cnf
          0.5 kB
        2. my.cnf
          5 kB
        3. mysqld_safe_syslog.cnf
          0.0 kB
        4. query_results_mariadb.txt
          2 kB

        Issue Links

          Activity

            lm Leif Matthiessen created issue -
            alice Alice Sherepa made changes -
            Field Original Value New Value
            Description I don't use the critical priority frivolously, but I have a query that always works on a mysql server, but only once at the first execution every 20 to 60 minutes on a MariaDB server. I am able to reproduce, that the query gives a different (wrong) result, if executed a few seconds after it gave the correct result. It also runs about 10 times slower on MariaDB than on mysql 5.7.
            I am ready to give you more information than just the query, but the data is sensitive. I have a pseudonymized database dump (101 MB compressed) and a screen recording (87 MB) that I could send to the developers who want to look into the problem. The screen recording shows me executing the query twice with different results and a proof via query log and diff tool, that the queries are the same.
            I filed this issue under "Data Definition - Temporary" because I did not find a component for issues with query results and had a vague feeling, that the temporary table has to do with it. However, I tried renaming it every time, but to no avail! Could it be the query cache?

            This bug affects our production environment.

            I appended the query results with columns separated by tabs, since I could not display them cleanly with this editor.


            Here is the query:

            DROP TEMPORARY TABLE IF EXISTS top5ads;

            CREATE TEMPORARY TABLE top5ads (click_count INT(10) UNSIGNED, ad_id INT(10) UNSIGNED);

            INSERT INTO top5ads (click_count, ad_id)
            SELECT
            COUNT(*) AS click_count,
            ads.id AS ad_id
            FROM
            employ__ad_clicks ad_clicks
            JOIN employ__ads ads ON ads.id = ad_clicks.ad_id
            JOIN employ__companies cmps ON cmps.id = ads.company_id
            WHERE
            cmps.vdma_company_id > 0 AND
            ads.job_type_id IN (1,2,3,4,5,13,19) AND
            ad_clicks.host <> '' AND
            -- {job_type_id} AND
            click_date BETWEEN '2018-01-01' AND '2018-07-01'
            GROUP BY
            ads.id
            ORDER BY
            click_count DESC
            LIMIT 5;

            SELECT
            click_count,
            CONCAT
            (
            TRIM(ads.title), ' | ',
            (
            SELECT
            CONCAT(pcs.postal_code, ' ', ad_pcs.city)
            FROM
            employ__ads_postal_codes ad_pcs
            JOIN employ__postal_codes pcs ON pcs.id = ad_pcs.postal_code_id
            WHERE
            ad_pcs.ad_id = ads.id
            LIMIT 1
            ), ' | ',
            click_count,
            '\n',
            cmps.name, ' ', cmps.second_name, ' | ', DATE_FORMAT(ads.created_at, '%d.%m.%Y'), ' | ',
            CASE WHEN
            ads.published AND
            CURRENT_DATE BETWEEN ads.publishes_at AND ads.ends_at
            THEN
            'online'
            ELSE
            'offline'
            END
            ) AS ad_title,
            ads.job_type_id,
            ads.permalink,
            mpgs_clicks_lists.mpgs_clicks_list,
            'Top' AS x_label
            FROM
            top5ads
            JOIN employ__ads ads ON ads.id = top5ads.ad_id
            JOIN employ__companies cmps ON cmps.id = ads.company_id
            JOIN
            (
            SELECT
            adcs_ad_id,
            GROUP_CONCAT(mpg_name_click_count ORDER BY mpg_name_click_count SEPARATOR '\n') AS mpgs_clicks_list
            FROM
            (
            SELECT
            adcs_ad_id,
            CONCAT(click_counts_mpgs.name, ': ', click_counts_mpgs.click_count_mpg) AS mpg_name_click_count
            FROM
            (
            SELECT
            adcs.ad_id AS adcs_ad_id,
            COUNT(*) as click_count_mpg,
            mpgs.name
            FROM
            employ__ad_clicks adcs
            JOIN employ__marketplace_group_hosts mpg_hosts ON adcs.host = mpg_hosts.name
            JOIN employ__marketplace_groups mpgs ON mpgs.id = mpg_hosts.marketplace_group_id
            WHERE
            adcs.host <> '' AND
            adcs.click_date BETWEEN '2018-01-01' AND '2018-07-01'
            GROUP BY
            mpgs.name,
            adcs.ad_id
            ) click_counts_mpgs
            ) mpg_name_click_counts
            GROUP BY
            adcs_ad_id
            ORDER BY mpg_name_click_counts.adcs_ad_id
            )
            mpgs_clicks_lists ON mpgs_clicks_lists.adcs_ad_id = ads.id;

            I don't use the critical priority frivolously, but I have a query that always works on a mysql server, but only once at the first execution every 20 to 60 minutes on a MariaDB server. I am able to reproduce, that the query gives a different (wrong) result, if executed a few seconds after it gave the correct result. It also runs about 10 times slower on MariaDB than on mysql 5.7.
            I am ready to give you more information than just the query, but the data is sensitive. I have a pseudonymized database dump (101 MB compressed) and a screen recording (87 MB) that I could send to the developers who want to look into the problem. The screen recording shows me executing the query twice with different results and a proof via query log and diff tool, that the queries are the same.
            I filed this issue under "Data Definition - Temporary" because I did not find a component for issues with query results and had a vague feeling, that the temporary table has to do with it. However, I tried renaming it every time, but to no avail! Could it be the query cache?

            This bug affects our production environment.

            I appended the query results with columns separated by tabs, since I could not display them cleanly with this editor.


            Here is the query:


            {code:sql}
            DROP TEMPORARY TABLE IF EXISTS top5ads;

            CREATE TEMPORARY TABLE top5ads (click_count INT(10) UNSIGNED, ad_id INT(10) UNSIGNED);

            INSERT INTO top5ads (click_count, ad_id)
            SELECT
            COUNT(*) AS click_count,
            ads.id AS ad_id
            FROM
            employ__ad_clicks ad_clicks
            JOIN employ__ads ads ON ads.id = ad_clicks.ad_id
            JOIN employ__companies cmps ON cmps.id = ads.company_id
            WHERE
            cmps.vdma_company_id > 0 AND
            ads.job_type_id IN (1,2,3,4,5,13,19) AND
            ad_clicks.host <> '' AND
            -- {job_type_id} AND
            click_date BETWEEN '2018-01-01' AND '2018-07-01'
            GROUP BY
            ads.id
            ORDER BY
            click_count DESC
            LIMIT 5;

            SELECT
            click_count,
            CONCAT
            (
            TRIM(ads.title), ' | ',
            (
            SELECT
            CONCAT(pcs.postal_code, ' ', ad_pcs.city)
            FROM
            employ__ads_postal_codes ad_pcs
            JOIN employ__postal_codes pcs ON pcs.id = ad_pcs.postal_code_id
            WHERE
            ad_pcs.ad_id = ads.id
            LIMIT 1
            ), ' | ',
            click_count,
            '\n',
            cmps.name, ' ', cmps.second_name, ' | ', DATE_FORMAT(ads.created_at, '%d.%m.%Y'), ' | ',
            CASE WHEN
            ads.published AND
            CURRENT_DATE BETWEEN ads.publishes_at AND ads.ends_at
            THEN
            'online'
            ELSE
            'offline'
            END
            ) AS ad_title,
            ads.job_type_id,
            ads.permalink,
            mpgs_clicks_lists.mpgs_clicks_list,
            'Top' AS x_label
            FROM
            top5ads
            JOIN employ__ads ads ON ads.id = top5ads.ad_id
            JOIN employ__companies cmps ON cmps.id = ads.company_id
            JOIN
            (
            SELECT
            adcs_ad_id,
            GROUP_CONCAT(mpg_name_click_count ORDER BY mpg_name_click_count SEPARATOR '\n') AS mpgs_clicks_list
            FROM
            (
            SELECT
            adcs_ad_id,
            CONCAT(click_counts_mpgs.name, ': ', click_counts_mpgs.click_count_mpg) AS mpg_name_click_count
            FROM
            (
            SELECT
            adcs.ad_id AS adcs_ad_id,
            COUNT(*) as click_count_mpg,
            mpgs.name
            FROM
            employ__ad_clicks adcs
            JOIN employ__marketplace_group_hosts mpg_hosts ON adcs.host = mpg_hosts.name
            JOIN employ__marketplace_groups mpgs ON mpgs.id = mpg_hosts.marketplace_group_id
            WHERE
            adcs.host <> '' AND
            adcs.click_date BETWEEN '2018-01-01' AND '2018-07-01'
            GROUP BY
            mpgs.name,
            adcs.ad_id
            ) click_counts_mpgs
            ) mpg_name_click_counts
            GROUP BY
            adcs_ad_id
            ORDER BY mpg_name_click_counts.adcs_ad_id
            )
            mpgs_clicks_lists ON mpgs_clicks_lists.adcs_ad_id = ads.id;

            {code}

            alice Alice Sherepa made changes -
            Labels need_feedback
            alice Alice Sherepa made changes -
            Assignee Alice Sherepa [ alice ]
            lm Leif Matthiessen made changes -
            Attachment mariadb.cnf [ 45971 ]
            Attachment my.cnf [ 45972 ]
            Attachment mysqld_safe_syslog.cnf [ 45973 ]
            alice Alice Sherepa made changes -
            Labels need_feedback
            lm Leif Matthiessen made changes -
            Environment Ubuntu Xenial Xerus Ubuntu Xenial Xerus, Mac OS
            lm Leif Matthiessen made changes -
            Environment Ubuntu Xenial Xerus, Mac OS Ubuntu Xenial Xerus, Mac OS 10.13.6
            alice Alice Sherepa made changes -
            Assignee Alice Sherepa [ alice ] Thirunarayanan B [ thiru ]
            alice Alice Sherepa made changes -
            Status Open [ 1 ] Confirmed [ 10101 ]
            alice Alice Sherepa made changes -
            Affects Version/s 10.3.8 [ 23113 ]
            alice Alice Sherepa made changes -
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            marko Marko Mäkelä made changes -
            Affects Version/s 10.3.2 [ 22533 ]
            Affects Version/s 10.2.9 [ 22611 ]
            marko Marko Mäkelä made changes -
            Component/s Storage Engine - InnoDB [ 10129 ]
            Component/s Query Cache [ 10120 ]
            marko Marko Mäkelä made changes -
            Labels corruption flush temporary
            marko Marko Mäkelä made changes -
            issue.field.resolutiondate 2018-08-24 06:58:17.0 2018-08-24 06:58:17.485
            marko Marko Mäkelä made changes -
            Fix Version/s 10.2.18 [ 23112 ]
            Fix Version/s 10.3.10 [ 23140 ]
            Fix Version/s 10.2 [ 14601 ]
            Fix Version/s 10.3 [ 22126 ]
            Resolution Fixed [ 1 ]
            Status Confirmed [ 10101 ] Closed [ 6 ]
            marko Marko Mäkelä made changes -
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 88669 ] MariaDB v4 [ 154738 ]

            People

              thiru Thirunarayanan Balathandayuthapani
              lm Leif Matthiessen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.