It's just talking, you said. It's the easiest job in the world, you assured yourself. You love talking in goofy voices, and look, you even own a microphone! Easy peasy.
Not so fast there, buddy. There's a lot more to consider.
Do you know how to make an invoice? How about creating an invoicing system that can track client names, pull up anything from any year you've been in business in a couple clicks? Do you like tracking all of your expenses and keeping reports on them? How about marketing and advertising, writing your own blogs, building your own web site (with samples, client list, rate sheet, a call to action, effective layout......), or developing the engineering skills to make demo after demo after demo as needed? Do you have enough liquid cash on hand to be an exhibitor at a convention, or to build a soundproof booth in your house, or to buy the equipment and software you need to be competitive? Does dealing with delinquent clients, renegotiating/auditioning/arguing with longtime clients every time they have a new project, or creating an LLC or corporation complete with accounts, books, payroll, and quarterly taxes sound fun?
This doesn't scratch the surface of what your actual job is as a voice actor. That bit behind the mic is icing on the cake. Hell, I barely mentioned auditioning, which you'll be doing a ton. If you're lucky enough to suss out all the various sources of auditions, convincing them to put you on their rolls, so you can START trying to get actual 'work'.
If you've ever been unemployed, looking on jobs boards, filling out online resumes and writing cover letters ad nauseum, know that looking for voiceover work is basically a cooler version of that, but forever. You get all the freedom and anxiety and night terrors that come with unemployment, but you're actually working the whole time. There are peaks and valleys, and it can really screw with you mentally and emotionally. Plus, you know, you're going to be isolated often.
That said, I'm over five years in, and I keep hearing that I have to starve for ten years before this thing really busts wide open. I guess I'll report back in five years and see if there's validity to that statement. And it's not to say I'm starving, exactly. I wouldn't call this the most stable job I've ever had, but I've never been as invested in anything I've ever done before in my life. And it's the most rewarding job I could ever do. And so far, somehow, my house is still standing and my daughter is still well-fed and happy, so I guess I'm doing my job. My wife pointed out that I am a voice actor. Not someone trying to break into voiceover, not a guy looking to do something...I am a voice actor. That feels really good.
So no, it's not all fun and games, but is it worth it? Only you can answer that for yourself. I say it every few posts, but I'll say it again - this isn't a part time gig you flit in and out of. This is a lifestyle. It's got to be part of who you are, not just one of the things you do. You have to put up with a mountain of BS, then shove it all aside to perform. Otherwise, what good are you to your client?