Every state is different, and in Missouri, every county has different rules. I cannot produce out of my house for some of the same reasons, so I am not at all surprised. I probably wouldn't fight it - there's little benefit to cornering a bureaucrat on a seemingly nonsensical decision (I know; I've tried), as it may only make your life more difficult. (Usually you have an assigned bureaucrat depending on where you are, and you may have to live with that person for a while.)
Have you talked with him about your options for becoming legit? He ought to be able to shed light there. If it's just investing in a commercial oven, then I would do that. If it's the inspection aspect, then , , , you may not be able to fight that. Part of the reason Missouri (my county, anyway) doesn't allow home-based food businesses is so they can do spot (non-scheduled) inspections. The theory being that people may really be doing things other (unapproved) ways during times not scheduled for inspections. The reason they may have approved the stone oven is because it is outside, and accessible for inspections at any time.
Your options (like renting space) are not necessarily bad ones - I found that when I started renting (and ultimately bought), I was forced to become more of a legit business by accounting for all of my real overhead costs and scaling up to cover them. Forced expansion out of the home may, in fact, be a blessing in disguise.