The spot separation preview can be isolated by using selections. Only the selected areas will be shown and separated in the window and if your selections are tonal masks or have feathered edges, they will be simulated correctly.

If the design you are separating includes motifs or specific areas of different colours (rather than an all over mix of colours), it may be useful to use the Automatic Tonal Separation, or HSL first, in order to create mask layers. These mask layers can then be selected (by going to Edit Menu > Select All (⌘A) and then Edit Menu > Select > Snap (⌘/)) and the selections will help to limit and define the areas to be separated using the Spot Window.

To use a selection with Spot Separations, simply select your area before you click Preview in Spot.