Leaving Barn Causes FPS and Performance to Drop

So, as the title says: whenever I leave the barn I go from a stable 60FPS to 50 and lower, and the gane stutters. Closing the game and relaunching it fixes the issue, until I enter and leave the barn again. Friendly heads-up, and didn't see it posted elsewhere.