A poster in the Apple Forums has pointed out that none of the standard keyboards used for Vietnamese input will work in OpenOffice, because it doesn't recognize their deadkeys, which are essential for typing the large number of diacritics used in this language. It turns out that you have to make a custom X11 keymapping file in order to do Vietnamese in this app. The basic info is contained in this note:
Making a Keyboard for OpenOffice/X11
And more details can be found here.