Is this common ? Why does it do this...

If i'm idling forward and go to trim my engine down... it kind of 'jerks' itself down one movement every half second till it I pull my finger off the trim switch.

What is causing this and is this something that needs a fix and what is the fix ?

Thx