[Saint]

If no farms, then no farmers. Also, no irrigation, so maybe urban planning is a bit backward - at least in the early days of the world. Maybe these people have no idea what large-scale land clearance is about. You just take the forest path from one village to the next. Fantasy games love spooky forests.

If no pastures, then not much use for animal husbandry. They might still figure out how to breed animals for desirable qualities - they just don't have millennia of experience. If no need to raise food animals, maybe they haven't tamed the wolf into a house pet or guard animal. Come to think of it, if no granaries then they probably don't have vermin like real-world civilizations, and no need to domesticate a cat to chase the vermin away.

Maybe they develop the watermill a little late, since there's no grain to grind.

There might not be any spice trade to speak of, at least not like in history.

But if I read the main point right, if the total effort that goes into planting, harvesting, preserving and preparing food is about the same as the effort that goes into conjuring it up as needed, then I don't see why you can't justify populations in your game world roughly equal to IRL examples.

You might have a whole class of conjurors who do nothing all day but support a much larger group of people by taking care of food needs. A restaurantocracy. Let's see Jack Vance top that.