The 81-year-old, who is accused of organizing sex parties where male models were paid for sex and plied with drugs, had initially been deemed unfit when he was hospitalized back in May because of the ailments.