One piece of the evidence is they have a king.
The fact that the Houyhnhnm have king means that they develop a complex social structure and separate each roles to all creatures within their community.
This means that they already develop a civilization just like its used in human civilization.
