I want to delete categories sidebar(on the main page) and make my own sidebar on the right side, and show this on all pages(including main page).
How do I do this?

The easiest would be to change the main_header.html template so that it works on all pages. Otherwise, several templates would have to be changed.

I'd say, you'd need to modify templates/main_modern_layout.html, where {$list_forums} stays for what you possibly want to replace.