Your masks should be made in standard 256 grey colors. It is preferable to
use a format supporting the color mode indexes (GIF, PNG, BMP, TIFF) in
order to conserve the original quality of the mask without loss of information,
particularly if the latter contains degraded content. But, it is possible to use
the JPEG format with a compression ratio sufficient for limiting loss due to
compression.