Does it cost to start my own business?

For getting the documents needed to set up a company through the Tax Office Agency (Skatteverket) is for free and you do not have to pay anything. 

Here is a basic guidance you can use to start your company through verksamt.se. If you want to register a name for your company through bolagsverket, there is a fee for that.