Many people learn the hard way that very old bourbons are not necessarily better, they're just older. This is different from scotch, where older generally does mean better. (Though, even there, not always.)
Very old bourbons have a distinctive taste that is wood-heavy and not to everybody's liking. I like them okay, but generally in very small doses.
The other thing a lot of noobs don't understand is that the profile of very old bourbons--good ones and not-so-good ones alike--is very different from the profile of standard bourbons. If you go around mostly trying very olds, you don't know what bourbon really tastes like.
Sometimes, people come to the bourbon category and want to try 'the best,' so they gravitate to the most expensive bottles, which generally are the very olds. They are often disappointed.
For example, the Old Rip Van Winkle 23-year-old. If you are very familiar with Stitzel-Weller whiskey then you might enjoy tasting an example of that whiskey at 23 years, and what the family thinks is a really great example of the whiskey at that age. You can appreciate it and be glad you tasted it, even if you don't particularly like it. If you come to it without that experience, you don't have a basis for enjoying it and instead experience all the reasons why bourbon producers normally don't leave their spirit in the barrel for 23 years.