The very first thing you need to do is get a Tax File Number (or TFN) which is similar to a NI national insurance number. Without a TFN, you will be taxed at the highest personal income tax rate including money earned from your job, any government funds or on savings account interest. You essentially need a Tax File Number when doing a number of things such as filing tax returns, starting a new job, changing jobs and applying for pension benefits.
In order to get a TFN, you need to fill out an application form. You can get an application form mailed to you by telephoning 13 2861 in Australia. You can also apply for a TFN online by going to www.ato.gov.au and clicking on the left-sided tab labelled "Individuals." There you'll find a step-by-step guide to applying for the TFN, including an application form.
This website is also the fount of all knowledge to do with tax in Australia. You may want to spend some time there to familiarise yourself with all Australian tax matters. Many of the other institutions that you'll deal with will ask you for your TFN, so the sooner you get it, the better.