When you select worker which only stand.
If in the menu of buildings press Cancel,
the worker will be unselected.
But if worker does something, then you press Cancel in the menu of buildings you just come out of this menu, it is not unselecting worker.