Not only binary data in tab-separated files look bad in any editor,
it is hardly editable, and this is hard to parse it for the LOAD DATA itself

One possible idea is to add escaping to the field, e.g \X for hex and \B for base64, this makes output tab-separated file more self-describing.

Or, SELECT INTO OUTFILE could rely on result set metadata like command-line client or mariadb-dump does, and LOAD DATA INFILE with hexblob option can rely on DDL info to unhex the field

Otherwise, it should be possibly to explicitly provide field names of hex/base64 encoded fields (although, not quite as user-friendly)

MySQL Shell dump-utilities do that (base64), and store the base64-ed fields in the metadata file, but it would be better to have it implemented in server itself.