While reality star Phil Robertson may never reappear on A&E's hit reality series "Duck Dynasty," due to expressing his views on Homosexuality being a sin, the other cast members may have to move forward legally without their leader.

Despite the Robertson's family apprehension to continue the show without its "Duck Commander," they cannot legally take a United We stand, Divided We Fall posture.

"It's not as easy as just not returning to the show, since they all have binding agreements. It's the network's option, not theirs," said a source connected to the production to FOX411. "Plus the family makes far too much money selling merchandise because of the show. It would be a bad business decision to leave it."

Even without Phil Robertson, there is just no way A&E would ever cancel "Duck Dynasty" right now. With 14 million viewers each week, including reruns, it is still the most viewed program on television.

"There is almost no way in the bayou that A&E would dump 'Duck Dynasty.' The show is a massive hit for them with 11.8 million viewers tuning in for [the] 'Duck Dynasty' Season 4 premiere earlier this year, and 8.9 million watching the Robertson clan's December 11 Christmas Special," said Deadline Hollywood's Dominic Patten. "Season 5 is set to debut on January 15 and you can bet it'll be a record breaker for a record-breaking show."

According to Julian Chan, an entertainment attorney in Los Angeles, each Duck Dynasty family member has a separate deal when it comes to this sort of agreement. So each cast member could be subject to a breach of deal claim regardless of what happens to anyone else on the show, no matter what his or her role is.

"Often a show will suspend a cast member who does something controversial. This gives them the chance to disassociate themselves from the controversy, see how the wind blows before deciding what to do, then either fire or reinstate the person after the media pressure has passed," said Chan to FOX411. "We have seen in the past where someone has been rehabilitated or forgiven, and other times where they have been moved out."

Last week, the Robertson family issued an announcement stating that while they have had a "successful working relationship with A&E... [We] cannot imagine the show going forward without our patriarch at the helm."