[MDEV-18525] mysql client runs out of memory on a box with 154G of free ram Created: 2019-02-09 Updated: 2019-03-11 Resolved: 2019-03-11 |
|
| Status: | Closed |
| Project: | MariaDB Server |
| Component/s: | Scripts & Clients |
| Affects Version/s: | 10.3.12 |
| Fix Version/s: | N/A |
| Type: | Bug | Priority: | Minor |
| Reporter: | Philip orleans | Assignee: | Unassigned |
| Resolution: | Not a Bug | Votes: | 0 |
| Labels: | None | ||
| Environment: |
ubuntu 18.04 |
||
| Description |
|
mysql asterisk -e "select * from npadata" > NAmDB.csv |
| Comments |
| Comment by Elena Stepanova [ 2019-02-09 ] | ||||||
|
Does it happen every time you run the query?
? | ||||||
| Comment by Philip orleans [ 2019-02-12 ] | ||||||
|
It happens after like an hour The table as 1.97BN rows. | ||||||
| Comment by Sergei Golubchik [ 2019-02-12 ] | ||||||
|
It doesn't seem to be crashing, it gracefully exits with an error "MySQL client run out of memory". An easy workaround for you would be to use the mysql -q, it will print results as received, row by row, instead of trying to read the whole result set in memory first. There might be something we can change in the client, but by far the fastest and easiest solution for you is to use -q option. | ||||||
| Comment by Elena Stepanova [ 2019-02-13 ] | ||||||
|
But does it actually use all the memory before aborting with the error? | ||||||
| Comment by Philip orleans [ 2019-02-13 ] | ||||||
|
The tool needs to be rewritten, instead of saving all the result set in memory, as noted earlier, it should write off a smaller buffer, and never request or use all the memory in the box. In any case, the parameter "-g" should be on by default. I would say that this a problem, a big problem. Why would anybody write a tool that behaves like this? It's a gross architectural misdesign. | ||||||
| Comment by Sergei Golubchik [ 2019-02-13 ] | ||||||
|
See
So, the logic here is to read the result from the server and free the server for other tasks as soon as possible. Because normally the client has enough memory to read the whole result set, it's better to use it, instead of blocking the server while the client consumes rows one by one. |