Details

    Description

      Please add a RETURNING option to INSERT.

      Example from PostgreSQL

      postgres=# CREATE TABLE t1 (id SERIAL, name VARCHAR(100));
      CREATE TABLE
      postgres=# INSERT INTO t1(name) VALUES('test') RETURNING id;
       id 
      ----
        1
      (1 row)
       
      INSERT 0 1
      

      Inspired by: https://evertpot.com/writing-sql-for-postgres-mysql-sqlite/

      This could make it easier to write statements which work with both MariaDB and PostgreSQL. And this might improve compatibility with Oracle RDBMS.


      How it'll work:

      • All variants of INSERT will work — INSERT ... VALUES, INSERT ... SET, INSERT ... SELECT
      • INSERT ... ON DUPLICATE KEY UPDATE will work, will return the final row values (inserted or updated)
      • REPLACE will return what's inserted
      • INSERT IGNORE will return what it'll try to insert, (ignoring insert errors, as expected)
      • INSERT DELAYED will, obviously, not work
      • rows will be returned as they're being inserted. If the insert fails in the middle and is rolled back — rows might've been sent that didn't end up in the table

      Attachments

        Issue Links

          Activity

            dveeden Daniël van Eeden created issue -
            dveeden Daniël van Eeden added a comment - - edited

            For what I know this is not in the SQL:2003 standard.
            http://www.savage.net.au/SQL/sql-2003-2.bnf.html#insert%20statement

            This might save a roundtrip. But I do think last_insert_id sent to the client anyways, so a 'SELECT LAST_INSERT_ID()' should not be needed.

            This could also allow multiple columns like it does in PostgreSQL. This could be efficient if defaults are used.

            Example:

            postgres=# CREATE TABLE t2(id SERIAL, name VARCHAR(100), language CHAR(2) DEFAULT 'en');
            CREATE TABLE
            postgres=# INSERT INTO t2(name) VALUES('test') RETURNING id,language;
             id | language 
            ----+----------
              1 | en
            (1 row)
             
            INSERT 0 1
            

            So from a client perspective normally you would need to do this:

            • insert a new record
            • do a select based on last_insert_id to get the language for the new row
            • do something in the code which uses the language.

            Often the solution to this is to keep all defaults client side to prevent the select, but then ad-hoc sql or other scripts might use other defaults. It should be possible to keep the defaults in the database w/o losing efficiency.

            dveeden Daniël van Eeden added a comment - - edited For what I know this is not in the SQL:2003 standard. http://www.savage.net.au/SQL/sql-2003-2.bnf.html#insert%20statement This might save a roundtrip. But I do think last_insert_id sent to the client anyways, so a 'SELECT LAST_INSERT_ID()' should not be needed. This could also allow multiple columns like it does in PostgreSQL. This could be efficient if defaults are used. Example: postgres=# CREATE TABLE t2(id SERIAL, name VARCHAR(100), language CHAR(2) DEFAULT 'en'); CREATE TABLE postgres=# INSERT INTO t2(name) VALUES('test') RETURNING id,language; id | language ----+---------- 1 | en (1 row)   INSERT 0 1 So from a client perspective normally you would need to do this: insert a new record do a select based on last_insert_id to get the language for the new row do something in the code which uses the language. Often the solution to this is to keep all defaults client side to prevent the select, but then ad-hoc sql or other scripts might use other defaults. It should be possible to keep the defaults in the database w/o losing efficiency.
            stephane@skysql.com VAROQUI Stephane made changes -
            Field Original Value New Value
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            serg Sergei Golubchik made changes -
            Labels gsoc19
            serg Sergei Golubchik made changes -
            Labels gsoc19
            serg Sergei Golubchik made changes -
            Labels gsoc19
            serg Sergei Golubchik made changes -
            Assignee Oleksandr Byelkin [ sanja ]
            cvicentiu Vicențiu Ciorbaru made changes -
            serg Sergei Golubchik made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            serg Sergei Golubchik made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            serg Sergei Golubchik made changes -
            Status In Progress [ 3 ] Stalled [ 10000 ]
            serg Sergei Golubchik made changes -
            Status Stalled [ 10000 ] In Review [ 10002 ]
            serg Sergei Golubchik made changes -

            In reply to Daniël, this is extremely beneficial when you are bulk adding rows plus their relationships when innodb_autoinc_lock_mode = 2 (required for Galera clusters). As the insert IDs cannot be guaranteed with innodb_autoinc_lock_mode=2 the last insert id is not sufficient when adding multiple rows: To add 100 rows the way Daniël described it would cause 200 extra round trips.

            Also, afaik the protocol only returns the LAST auto increment id of your query, not the auto increment ids of the rows in between.

            With this feature implemented, you can multi-row insert 100 rows and get the corresponding ids back in 1 rtt.

            Another new possibility with this feature: when doing an INSERT IGNORE .... RETURNING query you can see which rows actually inserted successfully, and trigger stuff in the application based on these results.

            Very nice work rucha174 and everybody else involved, I am looking forward to it being released.

            michaeldg Michaël de groot added a comment - In reply to Daniël, this is extremely beneficial when you are bulk adding rows plus their relationships when innodb_autoinc_lock_mode = 2 (required for Galera clusters). As the insert IDs cannot be guaranteed with innodb_autoinc_lock_mode=2 the last insert id is not sufficient when adding multiple rows: To add 100 rows the way Daniël described it would cause 200 extra round trips. Also, afaik the protocol only returns the LAST auto increment id of your query, not the auto increment ids of the rows in between. With this feature implemented, you can multi-row insert 100 rows and get the corresponding ids back in 1 rtt. Another new possibility with this feature: when doing an INSERT IGNORE .... RETURNING query you can see which rows actually inserted successfully, and trigger stuff in the application based on these results. Very nice work rucha174 and everybody else involved, I am looking forward to it being released.

            sanja, serg,

            Could you please specify here in JIRA description:

            • whether INSERT IGNORE ... RETURNING (also mentioned in the previous comment) is supported, and if so, what it should return – what was attempted to be inserted, or what actually was;
            • whether REPLACE ... RETURNING is supported, and if so, what it should return – what was newly inserted or everything;
            • what should happen when certain values are ignored upon INSERT (e.g. they were attempted to be inserted into virtual columns) – should they be returned or not;
            • what should happen if INSERT failed half-way (e.g. while inserting into a non-transactional table and hitting a duplicate key, but can be on various reasons);

            probably more questions to follow

            elenst Elena Stepanova added a comment - sanja , serg , Could you please specify here in JIRA description: whether INSERT IGNORE ... RETURNING (also mentioned in the previous comment) is supported, and if so, what it should return – what was attempted to be inserted, or what actually was; whether REPLACE ... RETURNING is supported, and if so, what it should return – what was newly inserted or everything; what should happen when certain values are ignored upon INSERT (e.g. they were attempted to be inserted into virtual columns) – should they be returned or not; what should happen if INSERT failed half-way (e.g. while inserting into a non-transactional table and hitting a duplicate key, but can be on various reasons); probably more questions to follow
            serg Sergei Golubchik made changes -
            Description (set as 'Data Manipulation - Insert Delayed', because I couldn't find a component for 'regular' insert)

            Please add a RETURNING option to INSERT.

            Example from PostgreSQL
            {noformat}
            postgres=# CREATE TABLE t1 (id SERIAL, name VARCHAR(100));
            CREATE TABLE
            postgres=# INSERT INTO t1(name) VALUES('test') RETURNING id;
             id
            ----
              1
            (1 row)

            INSERT 0 1
            {noformat}

            Inspired by: https://evertpot.com/writing-sql-for-postgres-mysql-sqlite/

            This could make it easier to write statements which work with both MariaDB *and* PostgreSQL. And this might improve compatibility with Oracle RDBMS.
            Please add a RETURNING option to INSERT.

            Example from PostgreSQL
            {noformat}
            postgres=# CREATE TABLE t1 (id SERIAL, name VARCHAR(100));
            CREATE TABLE
            postgres=# INSERT INTO t1(name) VALUES('test') RETURNING id;
             id
            ----
              1
            (1 row)

            INSERT 0 1
            {noformat}

            Inspired by: https://evertpot.com/writing-sql-for-postgres-mysql-sqlite/

            This could make it easier to write statements which work with both MariaDB *and* PostgreSQL. And this might improve compatibility with Oracle RDBMS.

            ----

            How it'll work:

            * All variants of {{INSERT}} will work — {{INSERT ... VALUES}}, {{INSERT ... SET}}, {{INSERT ... SELECT}}
            * {{INSERT ... ON DUPLICATE KEY UPDATE}} will work, will return the final row values (inserted or updated)
            * {{REPLACE}} will return what's inserted
            * {{INSERT IGNORE}} will return what it'll try to insert, (ignoring insert errors, as expected)
            * {{INSERT DELAYED}} will, obviously, not work
            serg Sergei Golubchik made changes -
            Description Please add a RETURNING option to INSERT.

            Example from PostgreSQL
            {noformat}
            postgres=# CREATE TABLE t1 (id SERIAL, name VARCHAR(100));
            CREATE TABLE
            postgres=# INSERT INTO t1(name) VALUES('test') RETURNING id;
             id
            ----
              1
            (1 row)

            INSERT 0 1
            {noformat}

            Inspired by: https://evertpot.com/writing-sql-for-postgres-mysql-sqlite/

            This could make it easier to write statements which work with both MariaDB *and* PostgreSQL. And this might improve compatibility with Oracle RDBMS.

            ----

            How it'll work:

            * All variants of {{INSERT}} will work — {{INSERT ... VALUES}}, {{INSERT ... SET}}, {{INSERT ... SELECT}}
            * {{INSERT ... ON DUPLICATE KEY UPDATE}} will work, will return the final row values (inserted or updated)
            * {{REPLACE}} will return what's inserted
            * {{INSERT IGNORE}} will return what it'll try to insert, (ignoring insert errors, as expected)
            * {{INSERT DELAYED}} will, obviously, not work
            Please add a RETURNING option to INSERT.

            Example from PostgreSQL
            {noformat}
            postgres=# CREATE TABLE t1 (id SERIAL, name VARCHAR(100));
            CREATE TABLE
            postgres=# INSERT INTO t1(name) VALUES('test') RETURNING id;
             id
            ----
              1
            (1 row)

            INSERT 0 1
            {noformat}

            Inspired by: https://evertpot.com/writing-sql-for-postgres-mysql-sqlite/

            This could make it easier to write statements which work with both MariaDB *and* PostgreSQL. And this might improve compatibility with Oracle RDBMS.

            ----

            How it'll work:

            * All variants of {{INSERT}} will work — {{INSERT ... VALUES}}, {{INSERT ... SET}}, {{INSERT ... SELECT}}
            * {{INSERT ... ON DUPLICATE KEY UPDATE}} will work, will return the final row values (inserted or updated)
            * {{REPLACE}} will return what's inserted
            * {{INSERT IGNORE}} will return what it'll try to insert, (ignoring insert errors, as expected)
            * {{INSERT DELAYED}} will, obviously, not work
            * rows will be returned as they're being inserted. If the insert fails in the middle and is rolled back — rows might've been sent that didn't end up in the table
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik made changes -
            serg Sergei Golubchik added a comment - see https://github.com/MariaDB/server/compare/bb-10.5-serg...bb-10.5-10014
            serg Sergei Golubchik made changes -
            Assignee Sergei Golubchik [ serg ] Vicentiu Ciorbaru [ cvicentiu ]
            serg Sergei Golubchik made changes -
            Assignee Vicențiu Ciorbaru [ cvicentiu ] Oleksandr Byelkin [ sanja ]
            serg Sergei Golubchik made changes -
            Fix Version/s 10.5 [ 23123 ]

            Generally all is OK except returning non-inserted data, so tests should be added:

            • INSERT IGNORE with some errors and it should not return what was not inserted
            • INSERT with trigger and ON DUPLICATE which change inserted data and return should return changed values
            • INSERT inside stored procedure (with multiple sets return)
            • INSERT inside function and trigger (even if it is an error)
            • "insert t1 set a=(select a+3);" will be turned to "insert t1 set a=(a+3)", so if we allow subqueries it will be better to put something which does not transform and have additional table "insert t1 set a=(select c from t2 where a=b)"
            • in the following query it would be better to return val to see if it was changed

              INSERT IGNORE INTO ins_duplicate(id,val) VALUES (3,'c'),(4,'d') ON DUPLICATE
              KEY UPDATE val='g' RETURNING id;
              

            sanja Oleksandr Byelkin added a comment - Generally all is OK except returning non-inserted data, so tests should be added: INSERT IGNORE with some errors and it should not return what was not inserted INSERT with trigger and ON DUPLICATE which change inserted data and return should return changed values INSERT inside stored procedure (with multiple sets return) INSERT inside function and trigger (even if it is an error) "insert t1 set a=(select a+3);" will be turned to "insert t1 set a=(a+3)", so if we allow subqueries it will be better to put something which does not transform and have additional table "insert t1 set a=(select c from t2 where a=b)" in the following query it would be better to return val to see if it was changed INSERT IGNORE INTO ins_duplicate(id,val) VALUES (3,'c'),(4,'d') ON DUPLICATE KEY UPDATE val='g' RETURNING id;
            sanja Oleksandr Byelkin made changes -
            Assignee Oleksandr Byelkin [ sanja ] Sergei Golubchik [ serg ]
            Status In Review [ 10002 ] Stalled [ 10000 ]
            serg Sergei Golubchik added a comment - post-review fixes: https://github.com/MariaDB/server/commit/b4626e12e2df239051ab6738b4cbfb6c6e10ac3a
            serg Sergei Golubchik made changes -
            Fix Version/s 10.5.0 [ 23709 ]
            Fix Version/s 10.5 [ 23123 ]
            Resolution Fixed [ 1 ]
            Status Stalled [ 10000 ] Closed [ 6 ]
            K Kaspars added a comment -

            With this feature implemented, you can multi-row insert 100 rows and get the corresponding ids back in 1 rtt.

            Is the result set guaranteed to be in the same order as value list?

            K Kaspars added a comment - With this feature implemented, you can multi-row insert 100 rows and get the corresponding ids back in 1 rtt. Is the result set guaranteed to be in the same order as value list?
            serg Sergei Golubchik made changes -
            Workflow MariaDB v3 [ 75500 ] MariaDB v4 [ 132861 ]
            danblack Daniel Black made changes -
            serg Sergei Golubchik made changes -
            bar Alexander Barkov made changes -
            bar Alexander Barkov made changes -
            bar Alexander Barkov made changes -
            bar Alexander Barkov made changes -
            bar Alexander Barkov made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            ralf.gebhardt Ralf Gebhardt made changes -
            bar Alexander Barkov made changes -
            Issue Type Task [ 3 ] New Feature [ 2 ]

            People

              serg Sergei Golubchik
              dveeden Daniël van Eeden
              Votes:
              5 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.