Red Hat Enterprise Linux Server release 6.9 (Santiago)
Linux 2.6.32-696.1.1.el6.x86_64 #1 SMP Tue Mar 21 12:19:18 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Query cache is normally disable by default, but this correspond to an issue detected today : MDEV-13300 in query cache.
Could you verify that your server configuration has query cache enable (SELECT @@have_query_cache, @@query_cache_type, @@query_cache_size) ?

Yup:

MariaDB [(none)]> SELECT @@have_query_cache, @@query_cache_type, @@query_cache_size\G;

*************************** 1. row ***************************

@@have_query_cache: YES

@@query_cache_type: ON

@@query_cache_size: 67108864

1 row in set (0.03 sec)

ERROR: No query specified

closing since it's a server issue : this will be handled with MDEV-13300.
workaround is to disable query cache or using only drivers that have CLIENT_DEPRECATE_EOF (or driver's that haven't CLIENT_DEPRECATE_EOF)

Thanks, Diego, but could you try explaining that workaround again? (It's confusing, as written.) Maybe write it as pseudocode with if statements?

i've changed MDEV-13300 description, but the best is to disable query cache :

SET global query_cache_type=0

RESET QUERY CACHE

that will disable immediatly the query cache.

Server configuration must be changed accordingly (query_cache_type=0) in configuration file, to avoid reenabling it is server is rebooted.

Sorry, I forgot that this was about this version of server (5.5). Since this error corresponded perfectly to the MDEV-13300 error, I was too quick to assign this to this error.

I reopen this issue.

Yes, it's suspicious that my symptoms are also packet errors and/or getting results back from unrelated queries. For what it's worth, I disabled the query cache on 7/13 (when I'd thought MDEV-13300 was the culprit). I don't know whether it's had any effect yet.

The thing is that's very weird : error message with "Packet contents (hex) = 03" means that packet content first byte is 03.
0x03 is the first byte of a column definition packet that describe column.
driver is waiting for an EOF packet then throw an exception, because protocol is not respected.

Before sending column definition packets, server indicate the number of column. This error would indicate that there server send more column than expected. I don't see how that can occur.

So i have a few question :

• Do you use java 8 ? if so, you use driver 2.0.3 and activate option "enablePacketDebug" (i.e. use connection string jdbc:mysql://myhost:3306/my_schema?connectTimeout=60000&socketTimeout=180000*&enablePacketDebug=true*) That won't correct the problem, but at least, stacktrace will contain the last hexadecimal exchanges with server. That would permit to identify this issue.
• do you use any proxy (HAProxy / maxscale ? )
• please join server configuration

• I'm pushing the 2.0.3 driver update & enablePacketDebug=true through our tiers. I'll have it up this evening.
• We don't have any intermediate proxy; the connections are direct.
• Config:

  /etc/my.cnf.d/server.cnf
  #
  # These groups are read by MariaDB server.
  # Use it for options that only the server (but not clients) should see
  #
  # See the examples of server my.cnf files in /usr/share/mysql/
  #
  # this is read by the standalone daemon and embedded servers
  [server]
  # this is only for the mysqld standalone daemon
  [mysqld]
  datadir=/var/lib/mysql
  socket=/var/lib/mysql/mysql.sock
  user=mysql
  group=mysql
  # Disabling symbolic-links is recommended to prevent assorted security risks
  symbolic-links=0
  max_allowed_packet=16384000
  group_concat_max_len=4294967295
  innodb_buffer_pool_size=2560M
  innodb_file_format = Barracuda
  innodb_file_per_table = 1
  innodb_open_files = 4096
  # disable query cache 7-13-2017
  query_cache_type=0
  query_cache_size=0
  tmp_table_size = 16M
  max_heap_table_size = 16M
  max_connections = 1000
  open_files_limit = 4096
  table_cache = 4096
  #changes 03-23-2017
  #changes 03-23-2017
  slow_query_log = 0
  slow_query_log_file = /var/log/mysql/redactedhostname-slow.log
  log_output = FILE
  long_query_time = 30
  log_queries_not_using_indexes = 1
  local-infile = 0
  ignore-db-dir=lost+found
  ignore-db-dir=lost@002bfound
  log_warning = 2
  skip-symbolic-links
  log-error=/var/log/mysql/redactedhostname.err
  secure_auth=1
  expire_logs_days=3
  character-set-server=utf8
  collation-server=utf8_general_ci
  init_connect='SET collation_connection = utf8_general_ci'
  init_connect='SET NAMES utf8'
  skip-character-set-client-handshake
  # this is only for embedded server
  [embedded]
  # This group is only read by MariaDB-5.5 servers.
  # If you use the same .cnf file for MariaDB of different versions,
  # use this group for options that older servers don't understand
  [mysqld-5.5]
  # These two groups are only read by MariaDB servers, not by MySQL.
  # If you use the same .cnf file for MySQL and MariaDB,
  # you can put MariaDB-only options here
  [mariadb]
  [mariadb-5.5]

For what it's worth, we've been running without query caching since 7/13 (when we'd thought this was caused by MDEV-13300), but I see more incidents of the error* from my app since then.

* (conn:3425) Packets out of order when reading field packets, expected was EOF stream. Packet contents (hex) = 03

I pushed the 2.0.3 driver update & enablePacketDebug=true to production last night.

Twenty minutes ago, we suffered another outage due to this bug. Unfortunately, we universally got the "crossed queries" manifestation this time, versus the "EOF" error. When we get crossed queries, we don't get JDBC errors, we just see a slew of downstream effects (e.g., "column not found, `blah`) of the wrong query being returned (because the wrong query's results are returned).

Unfortunately, this manifestation doesn't generate its own exception, so I didn't get anything out of the enablePacketDebug=true setting.

As I mentioned previously, I'd been having a mixture of two kinds of problems:

1. Crossed queries (that don't throw JDBC exceptions, but which cause problems in my app)
2. Packet errors (which do throw exceptions)

Since upgrading the driver, I only seem to get the first problem, which isn't helpful, since there's no exception now for enablePacketDebug to expose.

I need a plan B. Any ideas?

closing as requested

sorry closed wrong task

closing since not reproduced and no news since a long time