Mix peanut butter, milk, and vanilla until well blended. Gradually add powdered sugar. Mix until smooth and of desired consistency.