Most Muslims in west Bengal (and elsewhere, e.g. Bihar) were/are working off the land, and they stayed put.
Stop applying your own Bengali situation to the rest of India. We all know that Hindu Zamindars in Bengal were keeping Muslim Bengalis as labourers. I’m confused as to how Muslim Bengali descendants of Arab, Turk and Afghan nobility ended up working as labourers for Hindus?