how to animate a mask?
 

I need to mask an irregular section that changes both shape and position. Movie Studio doesn't allow making a mask by drawing, but I can make one in Gimp, and import png image.
Now about animation of my mask: I can use track motion for a single mask, but I'm not sure how to do it when my masking image needs to be updated every second or so. The transition won't be smooth.
Any suggestions?

A related question: can the mask be of different size than my video, or this would be a problem? This is important as during animation the masking image becomes smaller than my 1080x1920.

P.s. I know some basic shapes of masks can be entered and animated through track motion, but it's not good enough. I haven't figured out a way of combining these shapes, e.g. half-blocked and a circle (a cookie cutter FX cannot be animated).

Cookie Cutter shapes certainly can be animated. Unless I'm misunderstanding you. If you drop a shape on the track, then move 10s in, change the size, position, feather, etc. All those are key-frameable. The shape itself isn't, well... it is, but it simply jumps to the new shape at the key-frame.

As for sizing, if your image is too small, then the missing parts are essentially not there, or totally transparent. So, you might not have trouble there. If you're making a white on alpha mask, white being the bits to keep, then child that mask to the source track, setting its composite mode (not the parent mode, but the mode on the parent track) to multiply will show the parent image where there was white, and everything below as transparent, no matter if the source or mask tracks are smaller or different sizes from the project.

I really think you're in for a whole lot of pain though... manually rotoscoping is painful enough when you have an animateable mask right in front of you to work with, but to try and make it in gimp, then bring it in is probably enough to make you cry. And there really are no other options that I can think of to smooth it out.

You might try cinegobs, I've only played with it, but it does have animateable mattes, as well as many other interesting keying features. It's here:

CineGobs Keyer

I honestly have only played around with it though, but it did look interesting.

Hope this helps.

thanks, I'm giving the cookie cuter another try. Indeed it can be animated: I was using 'event pan/crop' instead of panning from within 'video event fx'. And it looks like I can do satisfactory masking by linking several 'circles with feathered edges' into a chain.

Funny I had a need to make a moving mask in a production just last week! I shot footage in a hospital environment and I needed to blur the information on a PC. It was your basic over the shoulder shot, and the person seated at the computer kept bobbing in front of a portion of the monitor.

After much trial and error, I duplicated the scene and put it on a track above the primary. I opened the Pan/Crop tool, selected the mask checkbox, then used the pixel tool to draw a custom mask around the screen of the PC monitor. Using the timeline, I dragged to the right just to the point where her head moved onto the screen and created an event point. I added a few points to the edge of the mask, and kept dragging the timeline and adjusting every other second or so. Vegas turned this into a smooth animation that followed the movement of her head. The key learning for me was to make lots of keyframes, especially if there was no movement for a portion of the scene. That keeps the animated mask on track with the footage.

Hope this helps?

The cookie cutter effect also works well. Put a normal version on track 1, a blurred version on track 2, and use a cookie cutter on track 1 to let the blurred section show through the desired area.

If you don't want to use multiple tracks, the NewBlue Pixelator in Video Essentials is worth looking into.