Determining when to use lazy loading

You have an object model that has a relational hierarchy three levels deep. In ad-dition to one chain that nests three levels deep (parent, child, and grandchild), you have four other child relations. In total, you have a parent and five related children, and the first child has two children of its own.

Should you pull back the entire chain and all the data, or pull back only a portion and then make roundtrips for the additional data as needed?
If you pull back all the data, what problems might you face?
If you choose to lazy load, what problems might you face?
How should you go about determining which approach should be used?