I love my husband, and I adore my child.
They are the only two men I have ever had in my life on a really personal level that fulfill my need for male companionship. With my son, it is all about having fun, teaching him, learning from him (because we can and should learn about the beauty of innocence, love, and joy from our children), and helping him become a better and more successful person than either I or his father have become.
With my husband, it is about growing with someone...figuring out life and how it works when you are sharing it with someone.
To me...that is harder.
It is hard to figure out how to balance your own self-interests, because they do not end when you get married, and those that are best for the unit. It is hard to have to deal with and accept someone else's faults, personality, opinions, issues, whatever...
With a child, of course, the difficulty comes in the unknown...each of us raises each child the best we can at the time (hopefully) but there is always that factor of knowing when to protect, when to allow the child to explore, when to step in, when to allow the child to handle the situation, etc.

So what do you think...which is harder. Cuz these are just MY opinions.