Baggy Shalwar are associated with Muslims. Even Hindus in India and Nepal wear non-baggy type Pyjamas. Also, South Asian or Afghan type Kurta or Kameez is rarely seen being work more Northward in Central Asia or Iran.
Shalwar Kameez is basically a dress introduced into the subcontinent by Muslim invaders, especially the Turks. Shalwar allows men and women to ride the horses with ease when they travelled. At the same time it also fits the Muslim dress code. Punjabis, being in the invasion path, were the first Indians who adopted it, with a lighter version to fit the hot Indian climate - hence 'Punjabi dress'.
During the last few decades, this dress has spread across South Asia, since its people are still comfortable with their Victorian modesty standards, and unable to adapt to the modern dress of European origin that exposes more skin. Even in the lands of its origin, Central Asia, people now wear modern dress, and not Shalwar Kameez. The Turkish lady who served me, as I lived in a yurt (Turkish tent) in Central Asia to experience the medieval Turkish life, wore a modern dress. But one can find Shalwar Kameez dress worn in remote backward Turkish villages.
the shalwar worn in pakistan and india is not the baggy shalwar which is worn in lets say balochistan, yes the indians did wear baggy pants, i have worn payjama which is essentially like a legging which is tightly fit, so indians did wear shalwar or the baggy pants before the islamic invasion. The shalwar worn in india and pakistan is just a misnomer, they may have been called loose payjama or something like that before, but it doesnt mean it derived from outside. here is the painted proof
this painting is from bihar/east india rather than Punjab 1015 AD, notice the loose pants as depicted in the body line and the line of the pants, clearly indicating baggy trousers
legging type payjama, this is from gilgit
here is one shalwar type payjama from Kashmir painting 600 AD
legging type payjama here from gupta period
buddha, chandraketugarh, bengal, 1-2 BC bengal, notice the folds in his pants
parthian statue, same pants
different shalwar, same parthian ethnicity
national shalwar kameez from india/pakistan
balochistan, baggy compared to pashtun version
so the conclusion is, baggy trousers were worn in pakistan/india before islamic invasion and the baggy trousers like the baloch for instance was never worn in the sub continent, the baggy shalwar of the western asia has no resemblance with the shalwar fitting of south asia as pashtun shalwar is not as baggy as baloch one, pashtun shalwar resembles indian shalwar and not iranian shalwar. the tailoring is also a matter of dispute, since the pashtun tailored dress resembles indian dresses especially kameez, which also resembles kurta, but this is not the case with lets say persian shirts/kurtas etc.