more mass would be pulling on you at an angle, rather than directly toward the center

Is this true? I don’t think a faraway object accelerates faster than a near one.