After Diocletian became Emperor in 285, the empire was truly administratively divided and then united, and then divided, and then united, and then divided....
The first Tetrarchy was formed when Diocletian split the empire into Eastern and Western halves, sharing his rule with his friend Maximiam. Diocletian did this because of the difficulties administrating and defending such a large empire. It seems that they were intended to be administratively separate, but act as partners towards a common goal (with Diocletian remaining as pater familias). From the top down there were two separate bureaucracies. They further divided administrative duties by appointing two Caesars as VP's, each of whom were to become Augustus upon dealth or abdication of their respective ruling emperor. This was supposed to form a system of succession, but never truly took hold.
The initial Eastern and Western empires were held together by the cooperation and friendship of the two emperors. Within another generation all parties went back to fighting over sole control - which occurred when Constantine I became sole Emperor, uniting the empire until again splitting it among his three sons after his death. I assume that most of the lower-level administrative divisions remained during all of the fighting at the top, but this is not clear in the sources that I have read.
I really find the whole thing somewhat confusing, so maybe someone with an expertise in Roman history might want to comment?