You should quit your day job... when the time is right.
For me that is a very scary thing. I'm always playing it safe. I like to know that things are going to work out before I even do them. The biggest thing that frightens me is the fear of the unknown. What if something happens, what if I get really sick, what if a shooting star falls from the sky and lands on our house and our insurance doesn't cover it. Yup, that has gone through me head.
What I've found is that things work out. They always do. If you have your heart set on something bigger than yourself and your mind open to the possibilities that adventure can bring, it will work out.