So playing with this some more suggests this is only a problem in single threaded/connection applications.
Because of how the semi-sync logic works sleeping threads are only woken up when a new write is processed.
Otherwise the full timeout is always observed.
In the case of an infinite timeout this means that unless you do another write on another connection your previous connection will never unblock.
However in the case of a shorter timeout the timeout will expire, the master will switch semi-sync off (even though there is a semi-sync slave available that has replicated all of the available entries) and the very next write will switch it back on.
The reason why this is important is the semantics of the timeout. The following only applies to applications that don't do concurrent writes.
Basically you want to be able to set the semi-sync timeout to the Mean Time To Recovery of your slave in most scenarios, or infinite if you never want to lose any data.
Doing either of these means your single threaded application is always going to be blocked the full duration of the timeout if your slave disconnects.
Which is untenable for obvious reasons. 
Would there be a way to wake connections up when a new slave is connected or better yet when a segment > the wait position has been transferred to a slave?
Another reasonable solution would be to have an option to change the behaviour on timeout from switching semi-sync off to returning an error to the client and rolling back the transaction.
This would eliminate the need for using an infinite timeout to ensure data safety is enforced.
I have found what I believe to be the root cause, which is that when using purely GTID replication the initial binlog dump doesn't result in waiting sessions being woken up because it relies on a comparison of the log file names/positions which may not be set on CHANGE MASTER when using purely GTID replication.
Changing my test case to also set MASTER_LOG_FILE and MASTER_LOG_POS worked around the issue.
Should I update the original issue to reflect the root cause?
When you do SET GLOBAL gtid_slave_pos = MASTER_GTID, which GTID do you set, exactly?
Is it the GTID of the last replicated event, or GTID of the event that is currently hanging (waiting) on the master?
I set the GTID of the most recent entry in physical backup used to create the slave.
It appears ActiveTranx::compare doesn't know how to handle GTID replication, or at least it's not aware of it and something should be converting to the old log file name + pos format before calling it.
Is there a way to get the master file name and position from the GTID during initial semi-sync dump?
I asked because I can easily reproduce the behavior if I use GTID of the hanging event, but not with the GTID of the last executed event.
Anyway, I'll pass it over to the replication expert knielsen for further analysis and comments on how GTID and semi-sync should work together.
Ahh, looks like I had a bug in my test script that was trying to replicate the hang. 
Fixing it so it uses the proper GTID fixes that little hang.
After I got past this one I ran into a different issue which I am not sure is actually solvable.
Basically I have a system that sets up an automated chain replicated MariaDB setup.
Normally it's setup with 3 nodes, one master with a semi-sync slave and an async slave replicating from the semi-sync one.
i.e:
Primary -> Sync -> Async.
Now when this is being setup each pulls a physical backup from it's upstream peer using xtrabackup.
How it works is a innobackupex process is spawned using xbstream mode and streamed into the local data directory.
All of this works fine except in one special case. Which is when someone attempts to call something like CREATE DATABASE while xtrabackup is trying to pull the first physical backup in order to setup the semi-sync slave.
What happens here is that the CREATE DATABASE transaction goes into the active transactions hash table and the database starts waiting on a slave to replicate it.
However the backup isn't yet complete.. so when it goes to finalize the backup and get a consistent snapshot of the binlog is calls FLUSH TABLES WITH READ LOCK.
Which seems to deadlock with the CREATE DATABASE transaction.
But because the semi-sync slave needs this physical backup to start replicating the CREATE DATABASE transaction will never be unblocked.
Is there anyway to avoid this scenario?
The only hunch I have right now is because the underlying filesystem is ZFS is looking into whether I can create a consistent snapshot without needing to call FLUSH TABLES WITH READ LOCK or other massive database lock.
Is it possible to take a consistent backup this way or prevent DDL like CREATE DATABASE from locking up the database until the semi-sync slave is online? Like a stronger version of @@read_only?
The GTID set in gtid_slave_pos is the last event executed on the slave.
So that GTID will never be received by the slave (but all following GTIDs
will).
I am not much familiar with semisync, but I'd expect that then the GTID will
also not be acknoledged (but following GTIDs will).
I think that is what Elena was refering to, so I agree, that seems right.
Seeing as I have gotten to the bottom of this and it's clear it's not actual a MariaDB bug but rather an interaction between semi-sync replication, locks and innobackupex/xtrabackup this should probably be closed as invalid.
Closing as per discussion in comments