Don't be a tourist!
Find the restaurants that are built into the alleys at night and take the Sunday bike path ride with the locals. I can assure you that you will be the only foreigner for miles. Visit the parks before the city wakes up to see the early birds doing Tai Chi and other forms of martial arts or even dance. Hong Kong is an expensive city if you're a backpacker, but it's a fun city if you know what to do. Sky car to see the Buddha on the hill is worth the wait to disappear into the clouds.
Insider Tip:The Octopus Card can be used everywhere! No need to carry a ton of cash unless you're eating at the smaller, less touristed restaurants. Also, it discounts your travel around Hong Kong significantly over day passes.