I did the same - I had a stainless Sub for years as my 1 watch, and then I got a dress watch and after that purchase, it was all over... I started adding to the collection and fell in love with AP's design language, PP and Cartier, etc. Then I had to have various color dials, various shapes, etc. I still have the one stainless steel Sub and I am not looking for any more Rolexes lolThat's exactly what I did. Wore the Sub as a One Watch. JLC dress watch I got last year was the first real luxury watch purchased since 2000.
Then again I don't move in Patek circles or have the expendable cash to collect them.