The tossed (or hit) ball follows a projectile motion trajectory -- the trajectories
, for the most part, are all parabolas
. Note that the straight line (up & down) is a special case of the parabola. If the launch angle it precisely 90 degrees (perpendicular to the ground), then the trajectory will be a straight line (up & down) assuming that the wind is not a significant factor.
Try if for yourself with the following java simulation of projectile motion. If you move the middle slider control all the way to the right, then the launch angle is 90 degrees.