I picked up and 04 STI center diff that was burned out from a local shop in hopes of reselling my 06 center diff and came across the same issue that you have now. I went ahead and used the 06 diff since that was the easier way out.

In addition to bolting the adapter in, I also welded it around the entire perimeter of the flange after V grooving it.

It sounds like youre on the right path just make sure that center smaller shaft is long enough to have sufficient engagement into the adapter.