Joomla has it inbuilt. You can simply switch it on and you should have pretty decent url structure for your complete website.
Here is how to do it :
- Goto Global Configuration in the admin backend
- Check the SEO Settings Panel
- As in the above screenshot, just switch on the above two options
- Before you do that, goto your FTP File manager and rename the htaccess.txt file to .htaccess in the root location.
- Refresh your frontend and you are done.
This is the basic Joomla SEF Support. If you are an advance user, and want to have a control on your SEF URLs, then you should better use sh404sef.