Fix some bugs in mouse tracking logic