Opposition to the Caesars did not worry overly about legality. Since no office actually existed ('Emperor' is a modern word the Romans did not use and the Caesars of Rome did not have a single authority, but instead, held control via packages of titles, powers, and honours the Senate assigned them) there could not be by definition a legal means to oust someone controlling them.
You then say, and I'm still trying to wrap my mind around this, that the opposition to the Caesar/Emperor did "not [need to] worry overly about legality". Well, if there are no legal means to oppose your wishes, and the people who want to oppose them need to act outside the law, that's basically the definition of a dictatorship. Normally when people oppose the power of the person running the country they can at the very least vote against them at the ballot box, etc, or the other governing bodies who share power can try to sack them. Your implicit concession that none of that was possible is the knife in the heart of your argument the emperors were not dictators.
Your paradoxical argument, that "since no office existed there was nothing to overthrow" is clearly not true; there is the de facto office the person with all these powers is holding, whatever they chose to call themselves. For instance, if the Senate (as you claimed) held the same or more powers as in the republic then they should have been able to simply vote to remove the powers from this person. In reality the Senate was a rubber stamp who had no legal redress against this person, and who could be compelled to do more or less anything by them. Power in Rome was not just "about Social Status", some of the Emperors were tremendously unpopular with the public and their peers at large; but because they had de facto power it didn't matter. The client/patron relationship is a complete red herring. In the old days of the Republic there were a bunch of families who formed the ruling class, and the power was divided among them more or less, however the balance ebbed and flowed. In the case of Imperial Rome the power was in the hands of one person, and everyone who wanted something had to ally themselves in some capacity with that person; because there weren't any alternatives. If the one guy in charge wanted to exile you or kick you out of the Senate, or make sure you didn't get any government contracts, etc, to cut you off from the source of your power; then he could. Clients enlisted with a patron because the patron could do stuff for them. If the patron can't do stuff for the client then they're a useless patron and will lose their clients. Similarly, if the patron needs to go through another patron to get things for their clients, then their clients are in effect the clients of someone else, just like different branches of a family under the family head are all under that family head on a chart. In the days of the republic powerful families could get together politically to oppose rival factions. In the Empire the guy in charge would just take the power from anyone trying to oppose him (which would both end the opposition, and prevent opposition from ever happening in the first place because you can see it won't end well).
Your counter point of "well they can assassinate him" only serves to show the vast differences that existed between the power structure in the Empire and Republic, where there were many legal means of opposition, power was shared and balanced to some degree, and there wasn't one guy with insurmountable life powers that neutered all legal opposition, and a host of other powers and structural support that made illegal opposition pretty tough too (for the Senate certainly).
Also the idea that the Senate would remove someone is subject to them debating and voting in that way. The Senate was not a unified body that acted to command. it was composed of Rome's elite who were wealthy enough to qualify, and the majority wanted nothing more than to be important and to enjoy senatorial perks than actually run the city. Even the men sent to oversee provinces did not rule over them directly. Local government remained in power and the governor was there as Rome's representative, the final word in both Roman and native law (Rome did not wipe away local laws when they brought a new provice into the fold - they simply added their own to the mix). Few would have the courage to stand up and declare opposition overtly. It was a very dangerous thing to do because the Caesar knew his position was insecure and thus needed protecting. However, the idea that a veto from a Caesar was meaningfully powerful in the face of a senatorial decree against him is ridiculous. If senators felt safe enough to make such a move, no veto from the offending Caesar could possibly matter. After all, once Nero was left exposed, the vengeful senators declared him 'Enemy of the State' thus he could be legally killed by anyone.