"My plants offer me respite from the world around us, as well."

Which is probably why he's there more often than not, and never to be found in Sorrow's orchard. One glance around his garden is enough to see it's fairly different from the orchard -- aside from the glyphs being used to keep everything watered, the plants themselves aren't the same type at all, each carefully labeled and arranged, with some vines crawling and coiling along constructed eaves and arches in the various rows. He's spent a lot of time here -- perhaps he feels he needs a lot of the aforementioned respite.