[MDEV-27838] Add handling of correct byte order in WKB geometries Created: 2022-02-14  Updated: 2023-12-11

Status: Open
Project: MariaDB Server
Component/s: GIS, Server
Affects Version/s: 10.2
Fix Version/s: 10.2

Type: Bug Priority: Major
Reporter: Anel Husakovic Assignee: Alexey Botchkov
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Blocks
blocks MDEV-26161 crash in Gis_point::calculate_haversine Stalled

 Description   

All geometries created from Geometry byte string Geometry *Geometry::construct on the server are created using little endian wkb_ndr, although there is the bit for big endian wkb_xdr and error message on the server ER_GIS_DATA_WRONG_ENDIANESS that is not used.
There are also some checks for big endian, not needed at all wkb_get_double () and wkb_get_uint():

wkb_get_uint()
  if (bo != Geometry::wkb_xdr)
    return uint4korr(ptr);

So or we need to remove the endian or we need to conclude when the geometry needs handling of byte order.
Something like this should be the patch for a function

--- a/sql/spatial.cc
+++ b/sql/spatial.cc
@@ -20,6 +20,7 @@
 #include "spatial.h"
 #include "gstream.h"                            // Gis_read_stream
 #include "sql_string.h"                         // String
+#include <mysqld_error.h>
 
 /* This is from item_func.h. Didn't want to #include the whole file. */
 double my_double_round(double value, longlong dec, bool dec_unsigned,
@@ -267,6 +268,13 @@ Geometry *Geometry::construct(Geometry_buffer *buffer,
 
   if (data_len < SRID_SIZE + WKB_HEADER_SIZE)   // < 4 + (1 + 4)
     return NULL;
+
+  // Check for byte order
+  if (uint4korr(data + SRID_SIZE) == Geometry::wkb_xdr)
+  {
+    my_error(ER_GIS_DATA_WRONG_ENDIANESS, MYF(0));
+    return NULL;
+  }
   /* + 

Issue is catched with MDEV-26161, where based on output of MySQL ER_GIS_DATA_WRONG_ENDIANESS is obtained (why?)


Generated at Thu Feb 08 09:55:58 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.