You can use your selection tool set to rounded rectangle, make your selection and if you need to move your marching ants, use your right mouse button to move the selection around.
Invert selection and cut, make sure this is a raster layer
Invert selection again, so it goes back to just around your photo and then expand your selection say 10-20 pixels,
choose your background color or image,
Create new raster layer for the mat,
use your bucket fill to fill in your selection that is on the new layer.
Your new layer will be on top of your photo image so move them to the right position in the layers pallet.
Then you can use drop shadows to make them look real, and layered!!
