What is meant by not hurting the oil and the wine, could be that the Olive Tree and Grapevine don't bear their fruit until some time after the wheat and barley harvests.
And they grow without much attention.
These crops wouldn't be affected by war.
So the Olive Trees and Grapevines weren't to be ruthlessly destroyed by invaders because they were needed for medicinal reasons.