I found the three-part inventions harder to play than the two-part inventions, if that helps. 
 
Just from listening to them:  definitely the Goldberg variations. 
 
Justin, can you comment on the violin concertos?  They sound pretty hard to me, but I wouldn't know where they rank in the large scale of things. 
 
I have to admit to not being that familiar with him, other than (a) what I was required to play by my teacher, which was mostly the two- and three-part inventions, IIRC; and ( the more famous pieces, with would be that famous organ fugue, the Goldberg variations, and some of the concertos.