[MCOL-4266] Clean away Windows carriage return (line ending) characters Created: 2020-08-23  Updated: 2021-05-14

Status: Open
Project: MariaDB ColumnStore
Component/s: None
Affects Version/s: None
Fix Version/s: Icebox

Type: Bug Priority: Major
Reporter: Otto Kekäläinen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: beginner-friendly

Issue Links:
Relates
relates to MCOL-4057 Package ColumnStore 5.x with 10.5 ser... Closed

 Description   

While preparing MariaDB 10.5.5 upload to Debian I noticed that there are many files in the ColumnStore sources that have been committed to the repository with Windows style line endings.

On my Linux machine git handles these files as text files and automatically strips Windows CR chars away, leading to mismatches in file checksums later on:

 mariadb-10.5/storage/columnstore/columnstore/build/Build.bat
 mariadb-10.5/storage/columnstore/columnstore/build/CalpontVersion.txt.in
 mariadb-10.5/storage/columnstore/columnstore/build/CopyToCalweb.bat
 mariadb-10.5/storage/columnstore/columnstore/build/Nightly.bat
 mariadb-10.5/storage/columnstore/columnstore/build/Switch.bat
 mariadb-10.5/storage/columnstore/columnstore/cmake/boost.CMakeLists.txt.in
 mariadb-10.5/storage/columnstore/columnstore/cmake/superbuild.md
 mariadb-10.5/storage/columnstore/columnstore/dbcon/ddlpackage/scanner.bat
 mariadb-10.5/storage/columnstore/columnstore/dbcon/ddlpackage/sql/examples/alter-table.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/ddlpackageproc/libddlpackageproc.rc
 mariadb-10.5/storage/columnstore/columnstore/dbcon/dmlpackage/scanner.bat
 mariadb-10.5/storage/columnstore/columnstore/dbcon/dmlpackageproc/libdmlpackageproc.rc
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/02.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/03.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/04.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/TE_Table_Create_wip.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/demo1_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/demo2_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/demo3a_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/demo3b_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/demo4_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q11_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q12_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q13_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q14_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q16_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q16_plan_o.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q17_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q18_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q20_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q21_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q22_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q22_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q22_plan_o.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q2_plan1.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q2_plan_o.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q3_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q4_plan.txt
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q7_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q8_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/doc/q9_plan1.TXT
 mariadb-10.5/storage/columnstore/columnstore/dbcon/joblist/libjoblist.rc
 mariadb-10.5/storage/columnstore/columnstore/dbcon/mysql/calremoveuserpriority.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/mysql/calsetuserpriority.sql
 mariadb-10.5/storage/columnstore/columnstore/dbcon/mysql/calshowprocesslist.sql
 mariadb-10.5/storage/columnstore/columnstore/ddlproc/DDLProc.rc
 mariadb-10.5/storage/columnstore/columnstore/dmlproc/DMLProc.rc
 mariadb-10.5/storage/columnstore/columnstore/exemgr/ExeMgr.rc
 mariadb-10.5/storage/columnstore/columnstore/oam/oamcpp/liboamcpp.rc
 mariadb-10.5/storage/columnstore/columnstore/oamapps/columnstoreSupport/calpontSupport.bat
 mariadb-10.5/storage/columnstore/columnstore/primitives/primproc/PrimProc.rc
 mariadb-10.5/storage/columnstore/columnstore/tools/dbbuilder/dataGen.pl
 mariadb-10.5/storage/columnstore/columnstore/tools/dbbuilder/srcdata/ordertable.csv
 mariadb-10.5/storage/columnstore/columnstore/tools/dbbuilder/srcdata/partsupptable.csv
 mariadb-10.5/storage/columnstore/columnstore/tools/dbbuilder/srcdata/regiontable.csv
 mariadb-10.5/storage/columnstore/columnstore/tools/dbbuilder/srcdata/suppliertable.csv
 mariadb-10.5/storage/columnstore/columnstore/tools/dbloadxml/colxml.rc
 mariadb-10.5/storage/columnstore/columnstore/tools/qfe/scanner.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/configcpp/libconfigcpp.rc
 mariadb-10.5/storage/columnstore/columnstore/utils/udfsdk/docs/make.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/Calpont.xml
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/EnvVarUpdate.nsh
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/crc.c
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/glob.c
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idb_64_enterprise-Debug.nsi
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idb_64_enterprise.nsi
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idb_64_standard.nsi
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idbmysql.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idbsvsta.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/idbsvsto.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/my.ini
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/poll.c
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/svcwait.bat
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part1.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part2.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part3.1.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part3.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part4.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/win_setup_mysql_part5.sql
 mariadb-10.5/storage/columnstore/columnstore/utils/winport/winfinidb.rc
 mariadb-10.5/storage/columnstore/columnstore/versioning/BRM/controllernode.rc
 mariadb-10.5/storage/columnstore/columnstore/versioning/BRM/libbrm.rc
 mariadb-10.5/storage/columnstore/columnstore/versioning/BRM/workernode.rc
 mariadb-10.5/storage/columnstore/columnstore/writeengine/bulk/cpimport.rc
 mariadb-10.5/storage/columnstore/columnstore/writeengine/libwriteengine.rc
 mariadb-10.5/storage/columnstore/columnstore/writeengine/server/WriteEngineServer.rc
 mariadb-10.5/storage/columnstore/columnstore/writeengine/splitter/splitter.rc

Please do not commit CR/LF in the version control to begin with.

As a temporary workaround, I told my git to manage these files are binary files and thus not touch the line endings by adding to .git/info/attributes:

storage/columnstore/columnstore/** binary



 Comments   
Comment by Otto Kekäläinen [ 2021-01-13 ]

Inside the MariaDB Server codebase there is also WolfSSL that seems to have a lot of Windows developers committing CR/LF line endings. WolfSSL issue: https://github.com/wolfSSL/wolfssl/pull/2784

Comment by Otto Kekäläinen [ 2021-05-11 ]

Faustin had this same issue: https://salsa.debian.org/mariadb-team/mariadb-10.5/-/merge_requests/5#note_239172

Could ColumnStore Windows developers please check your git settings to not import carriage returns into text files, thanks!

Generated at Thu Feb 08 02:49:03 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.