[XeTeX] Using --, ---, ``, '' and ' with CM

[Atip Asvanund]

There are times when I would like to use Computer Modern along with OTF
fonts. Since, the preferred method for entering these punctuations is via
Unicode, they only work with the fonts that XeTeX loads in. However, I
cannot get those punctuations to show up in Computer Modern at all,
whether I enter them them using unicode or with the --, ---, ``, '' and '.

Is there a work around for this?

Thank you,

Atip