You are probably right about that!
Make sure the bore is nice and clean before you put it in. If it were me, just for some redundancy I would think about making some type of retainer bracket utilizing the two bolt holes on either side of it. Or possibly just oversize washers utilizing those same two holes that could overlap it on the edges. Might be overkill, but I usually tend to try to alleviate possible reoccurrences.