Preheat oven to 450°F.
Rinse the potatoes and cut them into halves or quarters depending on the size. The more exposed starchy surface area (non-skin side), the crispier the potatoes will be.
In a large pot, bring water to a boil over high heat. Add a big pinch of salt, vinegar, and red potatoes. Cover and return to a boil, then reduce heat to a simmer and cook until potatoes are barely tender, about 5 minutes. Potatoes should be met with some resistance when pierced with a paring knife.
Strain all water and return the potatoes to the pot. Add olive oil, rosemary, salt, and pepper. Stir with a metal spoon to fully coat potatoes in oil and spices.
Distribute potatoes evenly on a greased rimmed baking sheet. Ensure exposed potato flesh is facing down on the pan.
Bake for 15 minutes and then rotate/flip potatoes to ensure additional sides are facing down. Bake another 15-20 minutes until potatoes are deep golden brown and thoroughly crisped.
In the last 10 minutes of cooking, melt butter in a small saucepan over low heat with rosemary and garlic. Bring to a light simmer and turn the heat down low to prevent burning or browning. Keep on low for about 5 minutes, stirring occasionally.
Baste potatoes all over with garlic butter as soon as they come out of the oven and serve.