Then there's also eBay. eBay can be a good place to buy a large lot of mixed figures for a low price, but it can also be a place where people resell stuff that you can buy cheaper elsewhere, or where you can find an old set of figures that is out of production and hard to find, which can sometimes get pricey. So you do need to know a bit about what you are bidding on, most importantly its fair market price. And then you also need a bit of luck so that the listing does not attract too many bidders.
Then there are also local hobby shops, but these are becoming less common given the competition from the online world. They sometimes they carry a few plastic figures. If you are looking for a good starter set, some of the BMC soldiers are fairly decent and affordable, and can usually be found at these shops.
So whether you are a seasoned collector or new to the hobby, good luck with your shopping and enjoy the hunt!