I can think of a few ways I might have handled the situation, but not sure they would result in any different result – with an Agent in your home. That’s bad. I might have asked for ID and then stepped out onto the porch, but then how would you know the knocking Agent is indeed from the government.
Given the fact that you may not be able to immediately verify the person’s identity, I might have told them to please go out to the road – off my property, while I then worked to verify.
Or just ignored him. But then that is sometimes how criminals “check ” to see if no-one is home.