Flatness is a PostScript parameter that is used to tell a RIP how accurately it has to process vector data like drawings made in Illustrator or CorelDraw, and masks that are saved in PhotoShop EPS-files. All of these applications allow the user to define the exact flatness value. Setflat is the operator used to define the flatness.
Photoshop bug
PostScript errors ‘typecheck’, offending command ‘setflat’ are caused by a bug in older (pre-2005) versions of Adobe Photoshop. They reflect a real problem that should have been reported as a ‘limitcheck’ error. A mask or cutout is too complicated. Open the image in PhotoShop and simplify the mask or try setting flatness to a higher value.