Trains link Tunisia's major cities, Tunis, Gabes, Sousse, Sfax and Gafsa (buy tickets in advance -it's much cheaper) and most towns also have bus service links, so a backpacking holiday in Tunisia is a viable option. Away from the touristic zones, the cost of living is also cheaper. Start in the north from Tunis and Carthage. The less touristy beaches at Cap Serrat may suit backpackers on holiday in Tunisia.
Inland Tunisia holds more of cultural interest to backpackers and will attract more similarly-minded travellers: the oases like Tozeur, El-Jem and Roman Thysdrus, the salt flats of Chott el-Jerid and troglodyte dwellings at Matmata (but their Star Wars fame brings crowds). The bigger cities like Gabes and Sfax and are largely industrial and commercial, but the sheer age and varied history of towns like Kairouan (the fourth holiest city in Islam) have appeal for adventurous travellers, as does the lure of almost any area bordering the Sahara.
If you're thinking of backpacking in Tunisia, you will most likely spend time inland, so don't assume English will be spoken. Away from heavy tourist regions it very likely won't be as Arabic and French are the local languages. Private health insurance is also essential if you are backpacking in Tunisia away from the main regions.
Away from the tourist resorts on the coast, budget accommodation can be very cheap indeed, but also rather basic and may not always have hot water. Tunisia does have a network of youth hostels accredited by the Association Tunisienne des Auberges et Tourisme de Jeunes and open to IYHA members, but some youth hostels are independent and most close during the day.