Riv Nuts holding hinges to trunk lid keep falling out

[Dodgeman]

I am having problems getting the riv nuts to hold the hinge bracket to the fiberglass trunk lid, They continue to pull out when I put a little pressure on them while tightening the 1/4-20 hex head bolts.
The slightest pressure caused them to tear out.
Have anyone had this problem and what did you do to fix it.
Thinking of bolting thru the lid.
I would add photos but don't know how to.

Thanks,
Bob