The activation of the human transforming growth factor (TGF-beta) system begins with the cytokine-induced association of the extracellular domains of two structurally related receptor subunits. To study the protein-protein interactions between TGF-beta and the ligand-specific receptor subunit, the extracellular domain of the human TGF-beta receptor type II (T beta R-II) has been expressed as an intracellular protein in insect cells using the baculovirus expression system. The cDNA construct was engineered to encode amino acids 24-159 (the signal sequence 1-23 was lacking) preceded by one initiator methionine residue and six histidine residues added at the carboxy terminus. The soluble receptor accumulated in the cytoplasm of infected cells and was purified by one-step nickel-chelate affinity chromatography. The purified protein was not glycosylated; it migrated as a single band of apparent mass 19.5 kDa in SDS/polyacrylamide gels, and had a homogeneous N-terminal sequence. We have established a solid-phase binding assay using radioiodinated TGF-beta 3 and capture antibodies to immobilize the soluble receptor. In this assay, the apparent dissociation constant of the TGF-beta type-II receptor ectodomain for TGF-beta 3 was approximately 150 nM (this value is approximately 1000-fold higher than that of the cell-membrane receptor complex of living cells). The affinity of TGF-beta 3 for the unglycosylated ectodomain of T beta R-II from insect cells was lower than the affinity for the recombinant glycosylated ectodomain T beta R-II from mouse cells. The novel assay has been used to characterize affinities and specificities of TGF-beta 3, TGF-beta 2, corresponding mutants and hybrid proteins, as well as a related protein, BMP-2. The assay could also be used to search for inhibitors.