Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-32524

Use named constants for ids used in spider mem alloc functions to avoid collision

Details

    Description

      To avoid more issues like in MDEV-32486...

      Attachments

        Issue Links

          Activity

            ycp Yuchen Pei added a comment - - edited

            Hi holyfoot, ptal thanks (based on 11.0)

            upstream/bb-11.0-mdev-32524 8ce2924e6be5361d1f2e73f54aa3bc53b98f30e5
            MDEV-32524 Use named constants for ids passed to spider_bulk_malloc()
             
            This will avoid issues like MDEV-32486
            

            For a 10.4 version, see

            4ff721cffa6 upstream/bb-10.4-mdev-32524 MDEV-32524 Use named constants for ids passed to spider_bulk_malloc()

            ycp Yuchen Pei added a comment - - edited Hi holyfoot , ptal thanks (based on 11.0) upstream/bb-11.0-mdev-32524 8ce2924e6be5361d1f2e73f54aa3bc53b98f30e5 MDEV-32524 Use named constants for ids passed to spider_bulk_malloc()   This will avoid issues like MDEV-32486 For a 10.4 version, see 4ff721cffa6 upstream/bb-10.4-mdev-32524 MDEV-32524 Use named constants for ids passed to spider_bulk_malloc()

            Great that you fixed this.
            Still i have two comments there
            https://github.com/MariaDB/server/commit/8ce2924e6be5361d1f2e73f54aa3bc53b98f30e5

            holyfoot Alexey Botchkov added a comment - Great that you fixed this. Still i have two comments there https://github.com/MariaDB/server/commit/8ce2924e6be5361d1f2e73f54aa3bc53b98f30e5
            ycp Yuchen Pei added a comment - - edited

            Hi holyfoot, thanks for the comment, and ptal thanks

            b9d74012c02 bb-11.0-mdev-32524 MDEV-32524 Fix long lines
            9b79247b8c4 MDEV-32524 Use enums for ids passed to spider mem alloc functions

            ycp Yuchen Pei added a comment - - edited Hi holyfoot , thanks for the comment, and ptal thanks b9d74012c02 bb-11.0-mdev-32524 MDEV-32524 Fix long lines 9b79247b8c4 MDEV-32524 Use enums for ids passed to spider mem alloc functions

            ok to push.

            holyfoot Alexey Botchkov added a comment - ok to push.
            ycp Yuchen Pei added a comment -

            I have prepared commits generated by a script for all versions up to
            11.0, with the 10.4 almost ready to push. Each version of 10.4-10.11
            seems to be quite different. Will push on Monday, after

            1. Fixing the commits due to a minor issue in the script (not bounding
            regex search by right brace)
            2. Waiting for any comments on slack regarding potential merging
            troubles

            ycp Yuchen Pei added a comment - I have prepared commits generated by a script for all versions up to 11.0, with the 10.4 almost ready to push. Each version of 10.4-10.11 seems to be quite different. Will push on Monday, after 1. Fixing the commits due to a minor issue in the script (not bounding regex search by right brace) 2. Waiting for any comments on slack regarding potential merging troubles
            ycp Yuchen Pei added a comment -

            Pushed 0b36694ff8293ee3e6ae5bffd0c254cced0a4ff0 to 10.4.

            Conflicts and solutions:

            • 10.4->10.5: 9f7da106c2c5dd9f48f8fcf40823a9d4fd655298
            • 10.6->10.11: 6a110e64b908e9d574e50c0f403b02769cfc4052
            • 10.11->11.0: 4d8167563e79f2657454a9f410e9cf2a2673bf76

            Because of the nature of the change, the best way to resolve merge
            conflicts is:

            1. merge until the parent commit of this commit
            2. git merge this commit; presented with conflicts
            3. disregard the changes, i.e. choose "ours" for all files
            4. run the script (see below)
            5. complete the merge of this commit
            6. merge the descendants of this commit

            I can do 2-5 if needed

            elisp script to generate the change for this ticket:

            (defvar my-mdev-32524-ids nil "List of ids used.")
             
            (defun my-mdev-32524-make-id ()
              "Return an id on the current position."
              (let* ((base
                      (replace-regexp-in-string
                       "::" "_"
                       (replace-regexp-in-string
                        "^\\(SPIDER_\\)?" "SPD_MID_"
                        (upcase (which-function)))))
                     (last-id (car my-mdev-32524-ids))
                     (counter (cdr last-id))
                     (result))
                (if (equal (car last-id) base)
                    (progn
                      (setq result (format "%s_%d" base (1+ counter)))
                      (push (cons base (1+ counter)) my-mdev-32524-ids))
                  (setq result (format "%s_1" base))
                  (push (cons base 1) my-mdev-32524-ids))
                result))
             
            (defun my-mdev-32524-replace-ids ()
              "Replace magic id numbers with enum symbols."
              (setq my-mdev-32524-ids nil)
              (dolist (file (directory-files "./storage/spider" t ".\\(cc\\|h\\)$"))
                ;; Don't do this with spd_malloc.{cc,h}
                (unless (string-match "spd_malloc" file)
                  (with-current-buffer (find-file-noselect file)
                    (goto-char (point-min))
                    (while (re-search-forward
                            (concat
                             "\\(spider_alloc_calc_mem_init\\|init_calc_mem\\|spider_malloc\\|"
                             "spider_bulk_alloc_mem\\|spider_bulk_malloc\\)"
                             "\\(([^0-9)]*\\)\\([0-9]+\\)")
                            nil t)
                      (let ((beg (match-beginning 3))
                            (end (match-end 3)))
                        (replace-region-contents beg end #'my-mdev-32524-make-id)))
                    (save-buffer)))))
             
            (defun my-mdev-32524-insert-enum-def ()
              "Insert enum def at point according to `my-mdev-32524-ids'."
              (with-current-buffer (find-file-noselect "./storage/spider/spd_include.h")
                (goto-char (point-min))
                (re-search-forward "#define.* SPIDER_CONN_META_BUF_LEN")
                (beginning-of-line 2)
                (let ((pre
                       "
            /*
              IDs for spider mem alloc functions, including
              - spider_alloc_calc_mem_init()
              - spider_string::init_calc_mem()
              - spider_malloc()
              - spider_bulk_alloc_mem()
              - spider_bulk_malloc()
              In the format of
              SPD_MID_<CALLSITE_FUNC_NAME_SANS_SPIDER_PREFIX>_<NO>
            */
            enum spider_malloc_id {
              ")
                      (middle (string-join
                               (sort
                                (mapcar
                                 (lambda (pair)
                                   (format "%s_%d" (car pair) (cdr pair)))
                                 my-mdev-32524-ids)
                                #'string<)
                               ",
              "))
                      (post "
            };
            "))
                  (insert (concat pre middle post))
                  (save-buffer))))
             
            (defun my-mdev-32524 ()
              "Make changes for MDEV-32524. Assuming we are at the source root."
              (interactive)
              (my-mdev-32524-replace-ids)
              (my-mdev-32524-insert-enum-def))

            ycp Yuchen Pei added a comment - Pushed 0b36694ff8293ee3e6ae5bffd0c254cced0a4ff0 to 10.4. Conflicts and solutions: 10.4->10.5: 9f7da106c2c5dd9f48f8fcf40823a9d4fd655298 10.6->10.11: 6a110e64b908e9d574e50c0f403b02769cfc4052 10.11->11.0: 4d8167563e79f2657454a9f410e9cf2a2673bf76 Because of the nature of the change, the best way to resolve merge conflicts is: 1. merge until the parent commit of this commit 2. git merge this commit; presented with conflicts 3. disregard the changes, i.e. choose "ours" for all files 4. run the script (see below) 5. complete the merge of this commit 6. merge the descendants of this commit I can do 2-5 if needed elisp script to generate the change for this ticket: (defvar my-mdev-32524-ids nil "List of ids used.")   (defun my-mdev-32524-make-id () "Return an id on the current position." (let* ((base (replace-regexp-in-string "::" "_" (replace-regexp-in-string "^\\(SPIDER_\\)?" "SPD_MID_" (upcase (which-function))))) (last-id (car my-mdev-32524-ids)) (counter (cdr last-id)) (result)) (if (equal (car last-id) base) (progn (setq result (format "%s_%d" base (1+ counter))) (push (cons base (1+ counter)) my-mdev-32524-ids)) (setq result (format "%s_1" base)) (push (cons base 1) my-mdev-32524-ids)) result))   (defun my-mdev-32524-replace-ids () "Replace magic id numbers with enum symbols." (setq my-mdev-32524-ids nil) (dolist (file (directory-files "./storage/spider" t ".\\(cc\\|h\\)$")) ;; Don't do this with spd_malloc.{cc,h} (unless (string-match "spd_malloc" file) (with-current-buffer (find-file-noselect file) (goto-char (point-min)) (while (re-search-forward (concat "\\(spider_alloc_calc_mem_init\\|init_calc_mem\\|spider_malloc\\|" "spider_bulk_alloc_mem\\|spider_bulk_malloc\\)" "\\(([^0-9)]*\\)\\([0-9]+\\)") nil t) (let ((beg (match-beginning 3)) (end (match-end 3))) (replace-region-contents beg end #'my-mdev-32524-make-id))) (save-buffer)))))   (defun my-mdev-32524-insert-enum-def () "Insert enum def at point according to `my-mdev-32524-ids'." (with-current-buffer (find-file-noselect "./storage/spider/spd_include.h") (goto-char (point-min)) (re-search-forward "#define.* SPIDER_CONN_META_BUF_LEN") (beginning-of-line 2) (let ((pre " /* IDs for spider mem alloc functions, including - spider_alloc_calc_mem_init() - spider_string::init_calc_mem() - spider_malloc() - spider_bulk_alloc_mem() - spider_bulk_malloc() In the format of SPD_MID_<CALLSITE_FUNC_NAME_SANS_SPIDER_PREFIX>_<NO> */ enum spider_malloc_id { ") (middle (string-join (sort (mapcar (lambda (pair) (format "%s_%d" (car pair) (cdr pair))) my-mdev-32524-ids) #'string<) ", ")) (post " }; ")) (insert (concat pre middle post)) (save-buffer))))   (defun my-mdev-32524 () "Make changes for MDEV-32524. Assuming we are at the source root." (interactive) (my-mdev-32524-replace-ids) (my-mdev-32524-insert-enum-def))

            People

              ycp Yuchen Pei
              ycp Yuchen Pei
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Git Integration

                  Error rendering 'com.xiplink.jira.git.jira_git_plugin:git-issue-webpanel'. Please contact your Jira administrators.