Years ago, I worked at a furniture store that sold, among other things, reclining chairs.
Since my job back then was to move these chairs around the showroom floor, I got very familiar with just how many there were. The choices were endless, and that was intentional. A customer could go in and find just the right chair, with just the right features. They could even special order the fabric if they didn't like what they saw on the floor.
Churches can be the same way. Don't like the music or building at one church, go down the street until you find just the right one for you.
Unfortunately, in doing so we have created so many churches that it's easy to forget there is really only one.
"There is one body and one Spirit, just as you were called to one hope when you were called; one Lord, one faith, one baptism; one God and Father of all, who is over all and through all and in all." Ephesians 4:4-6N (NIV)
By all means, go where you feel comfortable, but let's never forget we are all members of one church, the Body of Christ.
