Smaller tablets are way harder to control than decent sized once. Also the surface + pen combination is more often than not having less friction than a regular pen or pencil, while your hand tending to switch between gliding and sticking to the surface of your tablet.
Older tablets also tend to have a larger delay, which leads to overcorrecting all the time, making things worse.
Easiest solution to all those problem; use a stabilizer to varying degree.
Seriously, most people use them.