[MCOL-816] Improve O_DIRECT handling Created: 2017-07-18 Updated: 2022-11-05 Resolved: 2022-11-05 |
|
| Status: | Closed |
| Project: | MariaDB ColumnStore |
| Component/s: | N/A |
| Affects Version/s: | None |
| Fix Version/s: | Icebox |
| Type: | New Feature | Priority: | Major |
| Reporter: | Andrew Hutchings (Inactive) | Assignee: | Todd Stoffel (Inactive) |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Epic Link: | ColumnStore Performance Improvements |
| Description |
|
It appears that we don't use O_DIRECT in WriteEngine which probably means writes are getting put into the OS caches but are not being used by PrimProc which seems a waste. |
| Comments |
| Comment by Andrew Hutchings (Inactive) [ 2017-07-19 ] |
|
After a lot of testing I found that using posix_fadvise on small writes is probably the best way to handle this, otherwise for big writes we should be using unbuffered O_DIRECT. We should also make sure that PrimProc's read ahead is set high enough for O_DIRECT not to hurt read performance (I haven't found the sweet spot on my test drives yet). |
| Comment by Andrew Hutchings (Inactive) [ 2017-07-19 ] |
|
My quick hack test code: https://gitlab.com/LinuxJedi/directio |
| Comment by Todd Stoffel (Inactive) [ 2022-11-05 ] |
|
Item is out of date. Closing due to inactivity. If you feel this was done in error please open a new ticket. |