I don't trust those rubber caps and usually drive the nipple out then unstall a freeze plug. The nipples don't come out easily so in your situation with the water pump already installed the cap is probably the only reasonable option.

Jeff