The function uses realloc without checks. More immediately significant, on line 456 the variable "result" is reallocated, but no change is made to the pointer to it. Consequently, the memcpy that follows is liable to write into the freed memory space resulting from the realloc operation.

The function could be more effective if it made two passes, first computing the space required for the transformed string, then making a memory allocation (if necessary) and carrying out the replacements.

But it would probably be better altogether to use the PCRE function "pcre2_substitute" which appears to do the job without any development effort at all.