If the pages (from whatever type/extension) are static (not dynamic like PHP, ASP, etc.), the caching mechanism should be pretty automatic. The web server is supposed to add Last-Modified or ETag headers for you and the browser (or "user agent") is supposed to understand these.

You can check these headers are present or not with a tool such as Fiddler2 (on Windows).

If they are not present, then you would have to use a HTTP equivalent META tag, like this:

<meta http-equiv="last-modified" content="Sun, 27 Jan 2012 11:52:12 GMT" />