[MDEV-7580] possible memory leak Created: 2015-02-13  Updated: 2019-08-30  Resolved: 2016-11-23

Status: Closed
Project: MariaDB Server
Component/s: Query Cache, Storage Engine - InnoDB
Affects Version/s: 10.0.15-galera
Fix Version/s: N/A

Type: Bug Priority: Critical
Reporter: bmernz Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: need_feedback
Environment:

3 x gentoo linux vmware vm's with 8GB RAM allocted to each

Attachments: File mariadboomtest.log.gz     File mariadboomtest.php    
Issue Links:
Relates
relates to MDEV-20446 It can not re-joining Galera Cluster ... Closed

 Description   

The apparent leak seems to be proportional to innodb_buffer_pool_size

With innodb_buffer_pool_size = 128M it takes about 2-3 days for the kernel to run out of memory and kill/crash the mariadb node.

With innodb_buffer_pool_size = 1024M it takes about 2-3 hours for the kernel to run out of memory and kill/crash the mariadb node.

Please ask specifically for any/all details, logs, further tests etc. required to assist in investigation.

I'd also be happy to be directed to any related issues discussions etc. that googling has so far not revealed to me.

 Comments   
Comment by Elena Stepanova [ 2015-02-13 ]

Hi,

Did you by any chance try running a single-node "cluster" or a standalone server to see if it has the same problem?

Does it happen on all nodes or on any particular one?

Does the problematic node receive direct updates, or cluster updates, or both?

How many concurrent connections does the cluster serve?

Can you describe the flow that the node/cluster is running – is it mainly read (SELECTs), or write (INSERT/UPDATE/DELETE), or DDL (CREATE, DROP, ALTER and such)?

Would you be able to setup a simple monitoring script that would run 

SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST; 
SHOW INNODB STATUS;
 
 
 
-- and also top (or any analogue of your choice) for the mysqld process

every few seconds, from the node start and until OOM? (Note: it is important to do SELECT from IINFORMATION_SCHEMA rather than SHOW PROCESSLIST, because it displays some more data).

Thanks.

Comment by bmernz [ 2015-02-15 ]

Hi,

I haven't tried single node yet. It seems to happen on all nodes, however apparently only when that node is receiving direct updates - I'd need to do some more aggressive testing to confirm this.

In order to expedite the logs you requested I created an artificial load with representative queries. In normal operation the load is mainly SELECTS, however at this time of year there are a larger than normal number of INSERT/UPDATE/DELETE and DDL so I allowed for that in the tests.

Attached are the script, and the log produced.

In this instance mysql memory use increased dramatically and then it crashed rather than a kernel oom, however I'll see if I can get the test node to oom once it has resync'd and started up again.

This was the mysqld.err tail:

150215  8:46:31 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
 
150215 10:59:03 [ERROR] mysqld got signal 11 ;
 
This could be because you hit a bug. It is also possible that this binary
 
or one of the libraries it was linked against is corrupt, improperly built,
 
or misconfigured. This error can also be caused by malfunctioning hardware.
 
 
 
To report this bug, see http://kb.askmonty.org/en/reporting-bugs
 
 
 
We will try our best to scrape up some info that will hopefully help
 
diagnose the problem, but since we have already crashed,
 
something is definitely wrong and this may fail.
 
 
 
Server version: 10.0.16-MariaDB-wsrep-log
 
key_buffer_size=16777216
 
read_buffer_size=262144
 
max_used_connections=15
 
max_threads=153
 
thread_count=10
 
It is possible that mysqld could use up to
 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 137106 K  bytes of memory
 
Hope that's ok; if not, decrease some variables in the equation.
 
 
 
Thread pointer: 0x0x7f6160894e28
 
Attempting backtrace. You can use the following information to find out
 
where mysqld died. If you see no messages after this, something went
 
terribly wrong...
 
stack_bottom = 0x7f6059aece78 thread_stack 0x48000
 
(my_addr_resolve failure: fork)
 
/usr/sbin/mysqld(my_print_stacktrace+0x39) [0x7f61598aefb9]
 
/usr/sbin/mysqld(handle_fatal_signal+0x426) [0x7f61593a9086]
 
/lib64/libpthread.so.0(+0x11250) [0x7f6157dd2250]
 
/lib64/libc.so.6(fgets+0x1b) [0x7f61569c0f1b]
 
/usr/lib64/mysql/plugin/lib_mysqludf_sys.so(sys_eval+0x70) [0x7f605a3ec670]
 
/usr/sbin/mysqld(udf_handler::val_str(String*, String*)+0x80) [0x7f6159416720]
 
/usr/sbin/mysqld(Item_func_udf_str::val_str(String*)+0x28) [0x7f6159416858]
 
/usr/sbin/mysqld(Item_func_set_user_var::save_item_result(Item*)+0xd8) [0x7f6159417fe8]
 
/usr/sbin/mysqld(select_dumpvar::send_data(List<Item>&)+0x20a) [0x7f61591ffb2a]
 
/usr/sbin/mysqld(JOIN::exec_inner()+0x10e0) [0x7f61592850b0]
 
/usr/sbin/mysqld(JOIN::exec()+0x1d) [0x7f615928652d]
 
/usr/sbin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x1c5) [0x7f6159283535]
 
/usr/sbin/mysqld(handle_select(THD*, LEX*, select_result*, unsigned long)+0x2c4) [0x7f6159286874]
 
/usr/sbin/mysqld(+0x411cb4) [0x7f615922acb4]
 
/usr/sbin/mysqld(mysql_execute_command(THD*)+0x58b8) [0x7f61592371c8]
 
/usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x2c) [0x7f61594ca92c]
 
/usr/sbin/mysqld(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*)+0xa1) [0x7f61594d0741]
 
/usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0x17c) [0x7f61594d0d3c]
 
/usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x6f8) [0x7f61594cddd8]
 
/usr/sbin/mysqld(sp_head::execute_function(THD*, Item**, unsigned int, Field*)+0x29b) [0x7f61594ce5ab]
 
/usr/sbin/mysqld(Item_func_sp::execute_impl(THD*)+0x102) [0x7f6159419b82]
 
/usr/sbin/mysqld(Item_func_sp::execute()+0x36) [0x7f6159419c86]
 
/usr/sbin/mysqld(Item_func_sp::val_str(String*)+0x55) [0x7f6159423f55]
 
/usr/sbin/mysqld(Item::save_in_field(Field*, bool)+0x7a) [0x7f61593bbc2a]
 
/usr/sbin/mysqld(Item_trigger_field::set_value(THD*, sp_rcontext*, Item**)+0x73) [0x7f61593bc3f3]
 
/usr/sbin/mysqld(sp_instr_set_trigger_field::exec_core(THD*, unsigned int*)+0x2f) [0x7f61594ca32f]
 
/usr/sbin/mysqld(sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned int*, bool, sp_instr*)+0x324) [0x7f61594d09c4]
 
/usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x6f8) [0x7f61594cddd8]
 
/usr/sbin/mysqld(sp_head::execute_trigger(THD*, st_mysql_lex_string const*, st_mysql_lex_string const*, st_grant_info*)+0x19f) [0x7f61594ce1ef]
 
/usr/sbin/mysqld(Table_triggers_list::process_triggers(THD*, trg_event_type, trg_action_time_type, bool)+0x133) [0x7f61592bbe73]
 
/usr/sbin/mysqld(fill_record_n_invoke_before_triggers(THD*, TABLE*, List<Item>&, List<Item>&, bool, trg_event_type)+0x70) [0x7f61591edb60]
 
/usr/sbin/mysqld(mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool)+0xb69) [0x7f615921e3c9]
 
/usr/sbin/mysqld(mysql_execute_command(THD*)+0xab4) [0x7f61592323c4]
 
/usr/sbin/mysqld(mysql_parse(THD*, char*, unsigned int, Parser_state*)+0x25c) [0x7f615923b07c]
 
/usr/sbin/mysqld(+0x422151) [0x7f615923b151]
 
/usr/sbin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1c23) [0x7f615923d4c3]
 
/usr/sbin/mysqld(do_command(THD*)+0x21a) [0x7f615923dfba]
 
/usr/sbin/mysqld(do_handle_one_connection(THD*)+0x30c) [0x7f61592fa87c]
 
/usr/sbin/mysqld(handle_one_connection+0x56) [0x7f61592fa926]
 
/lib64/libpthread.so.0(+0x91da) [0x7f6157dca1da]
 
/lib64/libc.so.6(clone+0x6d) [0x7f6156a3fded]
 
 
 
Trying to get some variables.
 
Some pointers may be invalid and cause the dump to abort.
 
Query (0x7f5fecf5d538): is an invalid pointer
 
Connection ID (thread ID): 3346
 
Status: NOT_KILLED
 
 
 
Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
 
 
 
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
 
information that should help you find out what is causing the crash.

Comment by Elena Stepanova [ 2015-02-16 ]

Hi,

Thanks for the log, it's quite interesting.
There are some quite serious queries, and it looks like the memory usage jumps happen around the same time when execution of these queries happen. Maybe you could execute some of such queries manually, and see if it causes immediate memory leaks? See for example lines 296069, 306633 from mariadboomtest.log (I'll also quote the queries below). If it doesn't cause the leak, we'll need to look deeper.

Regarding the stack trace in your previous comment, I see it happened in lib_mysqludf_sys.so, a 3rd-party set of UDFs; so, maybe it's unrelated. Or maybe not, and the leak happens in the library, we cannot rule it out just yet. Still, I find those huge queries more suspicious for OOM. 

INSERT INTO ume_ObjectAttributes SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '24' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepUsername` AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 24 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 24 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 24 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `existingobject`.`value_tinytext` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '61' AS attribute_id, `newobject`.`object_id` AS value_int, NULL AS value_boolean, NULL AS value_text, NULL AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 61 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 61 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 61 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `existingobject`.`value_int` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '6' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepFirstName` AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 6 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 6 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 6 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '13' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepLastName` AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 13 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 13 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 13 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '22' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepRole` AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 22 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 22 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 22 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '1' AS attribute_id, NULL AS value_int, IF( ISNULL(`o`.`mlepSmsPersonId`), NULL, ASCII(`mlepSmsPersonId`)-48 IS TRUE ) AS value_boolean, NULL AS value_text, NULL AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 1 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 1 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 1 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '29' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, LOWER( CONCAT( LEFT(`o`.`mlepFirstName`,1), `o`.`mlepLastName` )) AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o LEFT JOIN `ume_ObjectAttributes` AS allusernames   ON `allusernames`.`attribute_id` = 29   AND `allusernames`.`value_tinytext` = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName` )) LEFT JOIN `ide_Person` AS idep   ON `idep`.`batch_id` = `o`.`batch_id` AND `idep`.`mlepSmsPersonId` != `o`.`mlepSmsPersonId` AND LOWER( CONCAT( LEFT( `idep`.`mlepFirstName`, 1), `idep`.`mlepLastName` )) = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName` )) JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 29 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 29 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 29 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `allusernames`.`id` IS NULL AND `idep`.`id` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '78' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, LOWER( CONCAT( LEFT(`o`.`mlepFirstName`,1), `o`.`mlepLastName`,'@tnis.school.nz' )) AS value_tinytext, NULL AS value_datetime, '5234' AS batch_id FROM `ide_Person` AS o LEFT JOIN `ume_ObjectAttributes` AS allemails ON `allemails`.`attribute_id` = 78 AND `allemails`.`value_tinytext` = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName`,'@tnis.school.nz' )) JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 08:02:52.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5234' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 78 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 78 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`1df2bfcc-b44b-11e4-8c90-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 78 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `allemails`.`id` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL
 

INSERT INTO ume_ObjectAttributes SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '24' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepUsername` AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 24 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 24 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 24 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `existingobject`.`value_tinytext` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '61' AS attribute_id, `newobject`.`object_id` AS value_int, NULL AS value_boolean, NULL AS value_text, NULL AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 61 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 61 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 61 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `existingobject`.`value_int` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '6' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepFirstName` AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 6 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 6 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 6 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '13' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepLastName` AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 13 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 13 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 13 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '22' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, `o`.`mlepRole` AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 22 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 22 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 22 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '1' AS attribute_id, NULL AS value_int, IF( ISNULL(`o`.`mlepSmsPersonId`), NULL, ASCII(`mlepSmsPersonId`)-48 IS TRUE ) AS value_boolean, NULL AS value_text, NULL AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 1 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 1 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 1 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '29' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, LOWER( CONCAT( LEFT(`o`.`mlepFirstName`,1), `o`.`mlepLastName` )) AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o LEFT JOIN `ume_ObjectAttributes` AS allusernames   ON `allusernames`.`attribute_id` = 29   AND `allusernames`.`value_tinytext` = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName` )) LEFT JOIN `ide_Person` AS idep   ON `idep`.`batch_id` = `o`.`batch_id` AND `idep`.`mlepSmsPersonId` != `o`.`mlepSmsPersonId` AND LOWER( CONCAT( LEFT( `idep`.`mlepFirstName`, 1), `idep`.`mlepLastName` )) = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName` )) JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 29 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 29 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 29 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `allusernames`.`id` IS NULL AND `idep`.`id` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL UNION SELECT NULL, COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AS object_id, '78' AS attribute_id, NULL AS value_int, NULL AS value_boolean, NULL AS value_text, LOWER( CONCAT( LEFT(`o`.`mlepFirstName`,1), `o`.`mlepLastName`,'@tnis.school.nz' )) AS value_tinytext, NULL AS value_datetime, '5240' AS batch_id FROM `ide_Person` AS o LEFT JOIN `ume_ObjectAttributes` AS allemails ON `allemails`.`attribute_id` = 78 AND `allemails`.`value_tinytext` = LOWER( CONCAT( LEFT(`o`.`mlepFirstName`, 1), `o`.`mlepLastName`,'@tnis.school.nz' )) JOIN `ide_Batch` AS b ON `b`.`id` = `o`.`batch_id` AND `b`.`timestamp` = '2015-02-15 09:02:33.000000' LEFT JOIN `ume_ObjectAttributes` AS newobject ON `newobject`.`value_tinytext` = `o`.`mlepUsername` AND `newobject`.`attribute_id` = 24 AND `newobject`.`batch_id` = '5240' LEFT JOIN `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS existingobject ON `existingobject`.`value_tinytext` = `o`.`mlepUsername` AND `existingobject`.`attribute_id` = 24 LEFT JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS snapobjid JOIN `ume_Batch` AS snapbatch ON `snapbatch`.`id` = `snapobjid`.`batch_id` ) ON `snapobjid`.`object_id` = COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) AND `snapobjid`.`attribute_id` = 78 LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspi JOIN `ume_AttributeTypes` AS uathisi ON `uathisi`.`id` = `sothisaspi`.`attribute_id` AND `uathisi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisasps JOIN `ume_AttributeTypes` AS uathiss ON `uathiss`.`id` = `sothisasps`.`attribute_id` AND `uathiss`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `sothisasps`.`object_id` = `sothisaspi`.`object_id` AND `sothisasps`.`value_int` = 3
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspa JOIN `ume_AttributeTypes` AS uathisa ON `uathisa`.`id` = `sothisaspa`.`attribute_id` AND `uathisa`.`description` = 'AttributeSourcePriority action Default' )
 
                ON `sothisaspa`.`object_id` = `sothisaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS sothisaspr JOIN `ume_AttributeTypes` AS uathisr ON `uathisr`.`id` = `sothisaspr`.`attribute_id` AND `uathisr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `sothisaspr`.`object_id` = `sothisaspi`.`object_id`
 
            ) ON `sothisaspi`.`value_int` = 78 AND `sothisaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) LEFT JOIN (
 
                ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspi JOIN `ume_AttributeTypes` AS uaprevi ON `uaprevi`.`id` = `soprevaspi`.`attribute_id` AND `uaprevi`.`description` = 'AttributeSourcePriority attribute_id Default' )
 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevasps JOIN `ume_AttributeTypes` AS uaprevs ON `uaprevs`.`id` = `soprevasps`.`attribute_id` AND `uaprevs`.`description` = 'AttributeSourcePriority source_id Default' )
 
                ON `soprevasps`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspa JOIN `ume_AttributeTypes` AS uapreva ON `uapreva`.`id` = `soprevaspa`.`attribute_id` AND `uapreva`.description = 'AttributeSourcePriority action Default' )
 
                ON `soprevaspa`.`object_id` = `soprevaspi`.`object_id` 
                JOIN ( `ume_ObjectAttributeSnapshots` PARTITION (`34973a88-b319-11e4-adc6-005056962ff9`) AS soprevaspr JOIN `ume_AttributeTypes` AS uaprevr ON `uaprevr`.`id` = `soprevaspr`.`attribute_id` AND `uaprevr`.`description` = 'AttributeSourcePriority rank Default' )
 
                ON `soprevaspr`.`object_id` = `soprevaspi`.`object_id` 
            ) ON `soprevaspi`.`value_int` = 78 AND `soprevasps`.`value_int` = `snapbatch`.`source_id`  AND `soprevaspa`.`value_tinytext` = IF( ISNULL( `snapobjid`.`id` ), 'Create', 'Update' ) WHERE ( COALESCE( `sothisaspr`.`value_int`, 0 ) >= COALESCE( `soprevaspr`.`value_int`, 0 ) ) AND ( COALESCE( `sothisaspr`.`value_int`, 0 ) != -1 ) AND `o`.`mlepRole` = 'NonTeachingStaff' AND `allemails`.`id` IS NULL AND COALESCE(`newobject`.`object_id`,`existingobject`.`object_id`) IS NOT NULL

Comment by bmernz [ 2015-02-18 ]

Hi,

I suspect you're correct, those queries do seem to coincide with the crashes. I haven't been able to lab the queries to provide the requested logs yet.

We have been restarting mysql regularly to minimise the risk/frequency of an oom crash, however today it appears that as soon as the rsync completes it crashes with kernel OOM. - below

We had also had to increase the frequency of the script that runs the query you flagged - I have now suspended that script to see if it reduces the symptoms.

This has become critical for us now so I will work with the team to lab the query this afternoon. If there are any other specifics that would assist in diagnostic please reply with details.

Feb 18 10:49:06 www02 kernel: [17729] 81 17729 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17731] 81 17731 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17733] 81 17733 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17734] 81 17734 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17735] 81 17735 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17736] 81 17736 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17737] 81 17737 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17738] 81 17738 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17739] 81 17739 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17740] 81 17740 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17741] 81 17741 55535 268 56 1218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17742] 81 17742 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17743] 10028 17743 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17744] 10028 17744 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17745] 10028 17745 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17746] 10028 17746 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17747] 10028 17747 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17748] 10028 17748 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17749] 10028 17749 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17750] 10028 17750 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17751] 10028 17751 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17752] 10028 17752 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17753] 10028 17753 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17754] 10028 17754 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17755] 10028 17755 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17756] 10028 17756 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17757] 10028 17757 55535 268 56 1219 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17758] 10028 17758 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17759] 10028 17759 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17760] 10028 17760 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17761] 10028 17761 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17762] 10028 17762 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17763] 10028 17763 76355 258 70 3165 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17764] 10028 17764 81451 258 80 8318 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17765] 10028 17765 76355 258 70 3165 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17766] 10028 17766 76355 258 70 3164 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17767] 10028 17767 80550 258 78 7431 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17768] 10028 17768 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17769] 10028 17769 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17770] 10028 17770 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17771] 10028 17771 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17772] 10028 17772 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17773] 10028 17773 55535 268 56 1220 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17774] 10028 17774 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17775] 10028 17775 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17776] 10028 17776 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17777] 10028 17777 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17778] 10028 17778 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17779] 10028 17779 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17780] 10028 17780 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17781] 10028 17781 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17782] 10028 17782 81451 259 80 8316 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17783] 10037 17783 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17784] 10037 17784 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17785] 10037 17785 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17786] 10037 17786 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17787] 10037 17787 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17788] 10037 17788 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17789] 10037 17789 55535 268 56 1221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17790] 10037 17790 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17791] 10037 17791 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17792] 10037 17792 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17793] 10037 17793 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17794] 10037 17794 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17795] 10037 17795 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17796] 10037 17796 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17797] 10037 17797 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17798] 10037 17798 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17799] 10037 17799 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17800] 10037 17800 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17801] 10037 17801 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17802] 10037 17802 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17803] 10037 17803 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17804] 10037 17804 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17805] 10037 17805 55535 268 56 1222 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17806] 10037 17806 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17807] 10037 17807 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17808] 10037 17808 55535 302 57 1249 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17809] 10037 17809 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17810] 10037 17810 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17811] 10037 17811 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17812] 10037 17812 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17813] 10037 17813 55535 302 57 1249 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17814] 10037 17814 55535 302 57 1249 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17815] 10037 17815 55535 302 57 1249 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17816] 10037 17816 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17817] 10037 17817 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17818] 10037 17818 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17819] 10037 17819 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17820] 10037 17820 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17822] 10037 17822 55535 268 56 1223 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17823] 10037 17823 55535 268 56 1224 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17824] 11524 17824 77403 258 84 4229 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17825] 11524 17825 58062 258 79 3840 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17826] 11524 17826 77157 258 84 4075 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17827] 11524 17827 58001 258 79 3851 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17828] 11524 17828 58001 257 79 3799 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17829] 11524 17829 78349 258 87 5198 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17830] 11524 17830 58268 275 79 4121 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17831] 11524 17831 57547 275 78 3405 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17832] 11524 17832 57986 258 79 3797 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17833] 11524 17833 76643 258 84 3560 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17834] 11524 17834 78349 258 87 5262 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17835] 11524 17835 57226 258 77 3081 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17836] 11524 17836 57986 258 79 3841 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17837] 11524 17837 57483 258 78 3335 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17838] 11524 17838 58059 258 79 3840 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17839] 11524 17839 77170 258 84 4085 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17840] 11524 17840 57226 258 77 3081 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17841] 11524 17841 57515 258 78 3370 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17842] 11524 17842 76947 258 83 3801 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17843] 11524 17843 58059 258 79 3838 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17844] 11524 17844 78676 285 88 5492 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17845] 11524 17845 77296 285 86 4218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17846] 11524 17846 77653 275 86 4574 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17847] 11524 17847 58761 286 82 4621 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17848] 11524 17848 79187 275 89 6108 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17849] 11524 17849 77924 285 87 4782 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17850] 11524 17850 78704 377 88 5523 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17851] 11524 17851 78709 478 88 5437 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17852] 11524 17852 78494 483 88 5218 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17853] 11524 17853 78624 1163 88 4668 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17854] 11524 17854 77667 285 86 4588 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17855] 11524 17855 79333 285 90 6221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17856] 11524 17856 77225 285 86 4147 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17857] 11524 17857 77740 284 87 4663 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17858] 11524 17858 76985 285 85 3907 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17859] 11524 17859 77387 285 86 4308 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17860] 11524 17860 78594 285 88 5513 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17861] 11524 17861 77684 285 86 4606 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17862] 11524 17862 77299 285 86 4221 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17863] 11524 17863 77219 286 85 4140 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17864] 11386 17864 76177 637 81 3025 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17865] 11386 17865 76246 636 82 3102 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17866] 11386 17866 76814 636 84 3670 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17867] 11386 17867 77065 636 83 3976 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17868] 11386 17868 57354 636 78 3094 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17869] 11386 17869 76306 636 82 3125 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17870] 11386 17870 57294 636 78 3085 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17871] 11386 17871 57154 636 77 2950 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17872] 11386 17872 57354 636 78 3087 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17873] 11386 17873 76305 636 82 3126 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17874] 11386 17874 75537 636 80 2393 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17875] 11386 17875 57355 636 78 3095 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17876] 11386 17876 76242 636 82 3039 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17877] 11386 17877 76462 636 83 3318 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17878] 11386 17878 57154 636 77 2950 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17879] 11386 17879 76242 636 82 3098 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17880] 11386 17880 74961 636 79 1817 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17881] 11386 17881 76750 636 84 3606 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17882] 11386 17882 57294 636 78 3086 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17883] 11386 17883 57154 636 77 2948 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17884] 11386 17884 77940 1857 87 3562 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17885] 11386 17885 77951 1770 88 3665 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17886] 11386 17886 76928 1879 86 2572 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17887] 11386 17887 76474 1913 84 2076 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17888] 11386 17888 77998 1921 87 3553 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17889] 11386 17889 77869 1853 87 3535 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17890] 11386 17890 81042 2527 93 6074 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17891] 11386 17891 76474 1783 84 2209 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17892] 11386 17892 76987 2362 85 2145 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17893] 11386 17893 77812 1852 87 3476 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17894] 11386 17894 76753 1872 85 2401 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17895] 11386 17895 78003 1784 87 3695 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17896] 11386 17896 76923 1836 85 2610 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17897] 11386 17897 77883 1825 87 3541 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17898] 11386 17898 76923 1774 85 2664 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17899] 11386 17899 76987 1793 85 2709 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17900] 11386 17900 77435 1723 86 3112 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17901] 11386 17901 77889 1761 87 3607 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17902] 11386 17902 77754 1800 87 3472 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17903] 11386 17903 76990 6412 85 1896 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17904] 19988 17904 63036 257 74 8873 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17905] 19988 17905 56699 259 61 2535 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17906] 19988 17906 56699 259 61 2535 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17907] 19988 17907 81863 259 79 8759 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17908] 19988 17908 84483 259 86 11279 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17909] 19988 17909 82119 259 79 8955 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17910] 19988 17910 56699 259 61 2535 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17911] 19988 17911 81803 276 78 8699 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17912] 19988 17912 82538 259 81 9476 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17913] 19988 17913 62848 259 73 8681 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17914] 19988 17914 81863 259 79 8758 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17915] 19988 17915 81992 259 79 8895 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17916] 19988 17916 81799 259 78 8692 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17917] 19988 17917 81799 259 78 8702 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17918] 19988 17918 81991 259 79 8894 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17919] 19988 17919 82402 259 81 9279 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17920] 19988 17920 81863 259 79 8759 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17921] 19988 17921 63041 259 74 8884 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17922] 19988 17922 77212 258 71 4110 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17923] 19988 17923 85634 259 88 12572 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17924] 19988 17924 63110 259 75 8907 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17925] 19988 17925 63042 259 75 8885 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17926] 19988 17926 62273 259 73 8118 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17927] 19988 17927 82324 259 80 9192 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17928] 19988 17928 88672 259 93 15551 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17929] 19988 17929 60097 259 69 5940 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17930] 19988 17930 82323 259 80 9200 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17931] 19988 17931 82475 259 81 9415 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17932] 19988 17932 63106 259 75 8952 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17933] 19988 17933 62338 259 73 8183 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17934] 19988 17934 63042 259 75 8885 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17935] 19988 17935 63108 259 75 8907 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17936] 19988 17936 63238 259 75 9083 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17937] 19988 17937 82324 276 80 9190 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17938] 19988 17938 84422 276 85 11214 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17939] 19988 17939 60160 276 69 6004 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17940] 19988 17940 63113 259 75 8909 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17941] 19988 17941 82334 259 80 9230 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17942] 19988 17942 82475 259 81 9414 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17943] 19988 17943 60161 259 69 6005 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17944] 19989 17944 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17945] 19989 17945 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17946] 19989 17946 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17947] 19989 17947 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17948] 19989 17948 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17949] 19989 17949 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17950] 19989 17950 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17951] 19989 17951 55568 268 56 1231 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17952] 19989 17952 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17953] 19989 17953 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17954] 19989 17954 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17955] 19989 17955 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17956] 19989 17956 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17957] 19989 17957 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17958] 19989 17958 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17959] 19989 17959 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17960] 19989 17960 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17961] 19989 17961 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17962] 19989 17962 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17963] 19989 17963 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17964] 19989 17964 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17965] 19989 17965 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17966] 19989 17966 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17967] 19989 17967 55568 268 56 1232 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17968] 19989 17968 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17969] 19989 17969 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17970] 19989 17970 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17971] 19989 17971 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17972] 19989 17972 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17973] 19989 17973 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17974] 19989 17974 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17975] 19989 17975 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17976] 19989 17976 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17977] 19989 17977 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17978] 19989 17978 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17979] 19989 17979 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17980] 19989 17980 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17981] 19989 17981 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17982] 19989 17982 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17983] 19989 17983 55568 268 56 1233 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17984] 19999 17984 78902 274 75 5770 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17985] 19999 17985 81900 257 81 8768 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17986] 19999 17986 82613 214 81 9433 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17987] 19999 17987 80379 257 76 7249 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17988] 19999 17988 78037 257 73 4928 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17989] 19999 17989 80380 257 77 7250 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17990] 19999 17990 79094 257 75 5968 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17991] 19999 17991 80407 257 76 7298 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17992] 19999 17992 78034 257 73 4949 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17993] 19999 17993 61561 256 70 7396 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17994] 19999 17994 62089 256 71 7945 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17995] 19999 17995 80421 256 75 7335 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17996] 19999 17996 82063 257 80 8940 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17997] 19999 17997 80590 256 76 7504 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17998] 19999 17998 78484 257 74 5375 0 php-fpm
Feb 18 10:49:06 www02 kernel: [17999] 19999 17999 82626 255 81 9503 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18000] 19999 18000 62774 257 72 8573 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18001] 19999 18001 80196 257 75 7090 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18002] 19999 18002 78902 257 75 5793 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18003] 19999 18003 82133 257 81 8972 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18004] 19999 18004 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18005] 19999 18005 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18006] 19999 18006 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18007] 19999 18007 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18008] 19999 18008 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18009] 19999 18009 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18010] 19999 18010 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18011] 19999 18011 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18012] 19999 18012 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18013] 19999 18013 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18014] 19999 18014 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18015] 19999 18015 55568 268 57 1235 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18016] 19999 18016 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18017] 19999 18017 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18018] 19999 18018 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18019] 19999 18019 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18020] 19999 18020 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18021] 19999 18021 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18022] 19999 18022 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18023] 19999 18023 55568 268 57 1236 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18024] 20000 18024 55586 427 59 1234 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18025] 20000 18025 55586 476 59 1194 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18026] 20000 18026 82485 401 80 9259 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18027] 20000 18027 81468 564 80 8047 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18028] 20000 18028 80660 429 78 7334 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18029] 20000 18029 79293 507 73 5862 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18030] 20000 18030 55586 386 59 1275 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18031] 20000 18031 55586 362 59 1308 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18032] 20000 18032 55586 358 59 1312 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18033] 20000 18033 79291 433 73 5933 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18034] 20000 18034 79291 396 73 5959 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18035] 20000 18035 55587 390 59 1281 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18036] 20000 18036 55587 394 59 1277 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18037] 20000 18037 79291 418 73 5948 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18038] 20000 18038 60242 414 68 5852 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18039] 20000 18039 81327 422 79 8048 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18040] 20000 18040 60242 409 68 5857 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18041] 20000 18041 55587 463 59 1199 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18042] 20000 18042 55587 465 59 1206 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18043] 20000 18043 55587 481 59 1181 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18044] 20000 18044 57542 1365 74 1289 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18045] 20000 18045 61478 965 81 5645 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18046] 20000 18046 57869 1368 75 1710 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18047] 20000 18047 57736 1096 74 1837 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18048] 20000 18048 61999 735 83 6477 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18049] 20000 18049 84932 477 94 11733 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18050] 20000 18050 80189 394 85 6488 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18051] 20000 18051 57419 999 73 1620 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18052] 20000 18052 57752 1104 73 1853 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18053] 20000 18053 58113 1197 75 2091 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18054] 20000 18054 57158 1398 72 1495 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18055] 20000 18055 57727 951 74 2002 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18056] 20000 18056 56902 805 71 1836 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18057] 20000 18057 57781 891 73 2060 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18058] 20000 18058 57862 457 74 2529 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18059] 20000 18059 57869 1213 74 1859 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18060] 20000 18060 61288 883 80 5639 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18061] 20000 18061 57672 989 72 1828 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18062] 20000 18062 57673 568 74 2316 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18063] 20000 18063 61677 733 80 6133 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18070] 10028 18070 76355 259 70 3165 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18071] 10037 18071 55568 268 56 1239 0 php-fpm
Feb 18 10:49:06 www02 kernel: [18072] 19999 18072 80398 257 76 7269 0 php-fpm
Feb 18 10:49:06 www02 kernel: [ 3604] 11386 3604 76815 1704 85 2628 0 php-fpm
Feb 18 10:49:06 www02 kernel: [27184] 0 27184 22502 0 45 217 0 sshd
Feb 18 10:49:06 www02 kernel: [27196] 1000 27196 31375 3 50 359 0 sshd
Feb 18 10:49:06 www02 kernel: [27205] 1000 27205 5542 12 12 77 0 bash
Feb 18 10:49:06 www02 kernel: [23032] 81 23032 519421 1355 129 1917 0 apache2
Feb 18 10:49:06 www02 kernel: [ 531] 0 531 21981 24 44 203 0 sshd
Feb 18 10:49:06 www02 kernel: [ 1200] 1000 1200 30828 0 48 312 0 sshd
Feb 18 10:49:06 www02 kernel: [ 1215] 1000 1215 5576 0 13 111 0 bash
Feb 18 10:49:06 www02 kernel: [ 1461] 1000 1461 15129 17 32 112 0 su
Feb 18 10:49:06 www02 kernel: [ 3131] 0 3131 5609 73 14 75 0 bash
Feb 18 10:49:06 www02 kernel: [15704] 81 15704 519421 1018 129 1971 0 apache2
Feb 18 10:49:06 www02 kernel: [28847] 60 28847 2615975 1906357 4458 231982 0 mysqld
Feb 18 10:49:06 www02 kernel: [28951] 0 28951 1525 0 6 25 0 tail
Feb 18 10:49:06 www02 kernel: [29696] 0 29696 24556 18 48 213 0 sshd
Feb 18 10:49:06 www02 kernel: [29746] 1502 29746 32882 17 49 355 0 sshd
Feb 18 10:49:06 www02 kernel: [29755] 1502 29755 8691 17 17 117 0 sh
Feb 18 10:49:06 www02 kernel: [29813] 1502 29813 18747 17 34 113 0 su
Feb 18 10:49:06 www02 kernel: [29822] 0 29822 5575 0 14 124 0 bash
Feb 18 10:49:06 www02 kernel: Out of memory: Kill process 28847 (mysqld) score 691 or sacrifice child
Feb 18 10:49:06 www02 kernel: Killed process 28847 (mysqld) total-vm:10463900kB, anon-rss:7625428kB, file-rss:0kB

Comment by Elena Stepanova [ 2015-02-18 ]

Hi,

I suspect you're correct, those queries do seem to coincide with the crashes. I haven't been able to lab the queries to provide the requested logs yet.

Alternatively, if you can provide the schema dump, I can try it locally. If it's possible, it would be even better since we'll be able to experiment locally. The dump can be uploaded to ftp.askmonty.org/private (only MariaDB developers will have access to it).
If you cannot provide the complete data due to confidentiality, the schema structure (CREATE statements for all involved objects) alone might help, although it's less likely.

We have been restarting mysql regularly to minimise the risk/frequency of an oom crash, however today it appears that as soon as the rsync completes it crashes with kernel OOM. - below

But did it have a chance to get actual queries, or did it crash even before that?

This has become critical for us now so I will work with the team to lab the query this afternoon. If there are any other specifics that would assist in diagnostic please reply with details.

As soon as we locate the guilty queries, it's possible that tweaking optimizer_switch or some other options might help, as a workaround until the fix. But we need to know first what causes the problem. The only ways to do it is to experiment either on your side, or on ours. Experiments on your side are good in terms of keeping the data private, if it's critical. Experiments on ours would probably be more efficient if we can reproduce it locally, but they'll require a lot of data, since the query involves so many objects.

Comment by Elena Stepanova [ 2015-06-02 ]

Hi,

What was the outcome of your experiments? Are you still experiencing the issue? Did you have a chance to collect the data – to run the suspicious queries, etc.?

Comment by Elena Stepanova [ 2016-11-23 ]

Please comment to re-open if you have further information on the issue.

Generated at Thu Feb 08 07:20:41 UTC 2024 using Jira 8.20.16#820016-sha1:9d11dbea5f4be3d4cc21f03a88dd11d8c8687422.