It also seems to be a known thing with populations of animals that just haven't encountered humans for a few generations,
such as the notorious Arctic terns on the Farne Islands, which also seems to be a stay-away-from-my-nest scenario. (Although the terns
do encounter humans fairly often, but not hostile ones, since it's a nature reserve... so they've got used to the idea that you can bully humans).
You possibly
could scare them off if you really tried, just due to being so much bigger than them. The rules for Intimidation do already cover that - you get a bonus corresponding to your Size Modifier minus the subject's. (When dealing with animals, it might be appropriate to give a penalty if the subject can fly and knows you can't!)