The following variations do work, so the system must be trying to be smart somewhere..
select * from trans_test where member_id in (select id from member_test where id < 2 );
select * from trans_test where member_id in (select id from member_test where name = 'member 1');
select * from trans_test where member_id = 1;

I have a suspicion this is to do with aliases, if you update member_test to not have an id column the query behaves correctly (it also behaves the same whether myisam or innodb):

create table member_test2 as select id mt_id, name from member_test;

MariaDB [test]> select * from trans_test where member_id  in (select mt_id from member_test2 where mt_id = 1 );

+------+-----------+---------+

| id   | member_id | name    |

+------+-----------+---------+

|    1 |         1 | trans 1 |

|    3 |         1 | trans 3 |

|    5 |         1 | trans 5 |

+------+-----------+---------+

3 rows in set (0.02 sec)

The reason appears to be that the initial myisam part of the query is optimized as a primary key match rather than a where condition, so there is no where condition pushed down which the cross engine processor is looking for. In the member_test2 example there are no indexes so there is a where condition to push down.

Whilst I work on a solution a possible workaround is to remove the primary key from these tables.

The problem was the where condition wasn't pushed down due to it being an index lookup. I have created a fix which will disable indexes whilst doing the first pass optimization but will use indexes for the cross-join execution or any query on a non-ColumnStore table.

Two possible workarounds if the row count of the MyISAM table is low are to remove indexes or disable them using `USE INDEX()` after the table name in the FROM clause in the subquery.

Build verified: Build from Github

[root@localhost mariadb-columnstore-server]# git show
commit 3795bd4cf42d59b792c473101703911fb53e9297
Merge: 570184c 84714c9
Author: dhall-InfiniDB <david.hall@mariadb.com>
Date: Wed Nov 30 11:42:14 2016 -0600

Merge pull request #18 from mariadb-corporation/MCOL-424

MCOL-424 Disable indexes for cross-engine

[root@localhost mariadb-columnstore-server]# cd mariadb-columnstore-engine/
[root@localhost mariadb-columnstore-engine]# git show
commit 63266938716bd933b80eb03fc8aa4ef10d25eab6
Merge: 691c52c 4efd58d
Author: david hill <david.hill@mariadb.com>
Date: Fri Dec 2 10:03:11 2016 -0600

merge mcol-421 branch

Verified the test case in the bug description
MariaDB [mytest]> SELECT * FROM trans_test t WHERE t.`member_id` IN (SELECT id FROM member_test WHERE id =1);
----------------------

----------------------

----------------------
3 rows in set (0.19 sec)