Hi Aussie Boy,
You can find some interesting Jasper on the beaches around there. Caves Beach is pretty good but not where you swim - you have to head North right up to the next headlands you can see to find a fair bit.
A bit further but there is also some interesting stuff in the Barrington Tops - small sapphire etc but you'll need to google it and do your research. Strongly recommend checking out your local lapidary club as they'll have info but also often do organised trips you can go on and learn a lot.