All the things that Hamlet said after he asked Ophelia where her father was is already directed to Polonius. He already knew that Polonius was in the room, eavesdropping.
"Are you honest?", "Are you fair?" are the lines which Ophelia did not really understand why he was asking. This means that these lines are directed to Polonius.
"It hath made me mad" was said for them to believe that he's really crazy.
