Like any investment, it can be risky. Location and current market conditions are extremely important when deciding whether to buy.
Other things to consider:
Will you be able to afford repairs, maintenance, insurance, and utilities?
What about fees to pay agents who rent the property for you?
If you live several miles away from your vacation home, who will clean up between tenants and take an inventory of household items once the tenants leave?
What if you are unable to rent your second home? Can your pocketbook withstand the strain of paying the mortgage?