Place potatoes in a pot of cold salted water. Bring to a boil and cook for 10-15 minutes, or until fork tender. Set aside in a large bowl.
While the potatoes are cooking, add diced bacon to a heavy skillet and cook until crisp, about 5 minutes. Remove from skillet with a slotted spoon and add to potatoes.
Drain all but 2 tablespoons of bacon grease and return skillet to medium heat. Add onion to skillet and cook over medium-low heat until caramelized and brown, about 20 minutes. (Don't skip this step, and be careful not to burn the onion!)
When the onions are a deep, caramel brown, add them to the bacon and potatoes.
Combine apple cider vinegar, water, and Old No. 1 in a small bowl and mix with a wire whisk. Pour this mixture into the skillet with the bacon drippings and cook over medium-high heat for about 2 minutes, or until the sauce is thickened. Be sure to scrape all of the browned bits from the bottom of the skillet.
Pour sauce over the potato mixture and toss until well-coated. Serve warm. Garnish with a sprig of fresh rosemary.