It’s a rare word these days, but as it has a grand sound, and its sense — adulterous sexual intercourse — is of such universal application, perhaps somebody should begin a campaign to restore it to common usage.
One well-known appearance is in Vladimir Nabokov’s book Pale Fire: “She would have preferred him to have gone through a bit of wholesome houghmagandy with the wench”.
We do know the word was originally Scots, as the guttural gh indicates. The first part is the same word as hock, the joint in a four-legged animal that matches the human ankle, sometimes still spelt that way (as in the Scots’ hough soup). It can also refer to the hollow part behind the human knee joint (didn’t you always want a word for it? Actually it’s better known to medicine as the popliteal area) as well as the nearby thigh. The second element of the word is problematic; it could be from canty, a Scots and northern English dialect adjective for someone who is lively or cheerful, or perhaps active or brisk. So, a bit of active thigh work — you can see how the word could have arisen.
There seems to be no link with the similar-sounding but obsolete Australian word for a thin and unpalatable stew, hashmagandy, which comes from salmagundi.